|
|
c70b42c9d7
|
Made a lot of changes. Enhanced memory usage.
|
2025-02-16 09:38:00 +01:00 |
|
|
|
7abe904997
|
Testing search query
|
2025-01-05 15:37:18 +01:00 |
|
|
|
e9fd738f1f
|
Title and description extraction now only look in the head section of the HTML document
|
2024-12-23 14:09:51 +01:00 |
|
|
|
580d687f62
|
Add user-agent
|
2024-12-21 21:57:01 +01:00 |
|
|
|
5fe35192c6
|
Misc changes such as performance tuning, added support for more than 64 threads for the scanner.
|
2024-12-04 13:20:45 +01:00 |
|
|
|
3034e66126
|
Changed GET to HEAD for checking robots txt.
|
2024-11-29 11:02:51 +01:00 |
|
|
|
f02203c14c
|
Fixed duplicate insertions into the filtered database.
|
2024-11-27 15:53:00 +01:00 |
|
|
|
f3c6338a6a
|
Initial commit
|
2024-11-24 12:40:51 +01:00 |
|
