Commit Graph
20 Commits
Author SHA1 Message Date
owner 45b6d8ecab Preparing for multithreaded content filter 2025-02-07 23:30:25 +01:00
owner 83897c72cf Way too much happened for me to create a commit message 2025-02-07 18:44:22 +01:00
owner 7abe904997 Testing search query 2025-01-05 15:37:18 +01:00
owner a6804be9f1 Implement database compression 2024-12-30 18:36:04 +01:00
owner 0cfbcea252 Testing database compression. 2024-12-24 14:23:23 +01:00
owner e9fd738f1f Title and description extraction now only look in the head section of the HTML document 2024-12-23 14:09:51 +01:00
owner 580d687f62 Add user-agent 2024-12-21 21:57:01 +01:00
owner 5fe35192c6 Misc changes such as performance tuning, added support for more than 64 threads for the scanner. 2024-12-04 13:20:45 +01:00
owner 2cff817044 Make it so the filter does not iterate over already filtered IPs. 2024-11-29 19:13:27 +01:00
owner f2ace6f571 Reworked the queue items. 2024-11-29 12:59:07 +01:00
owner 3034e66126 Changed GET to HEAD for checking robots txt. 2024-11-29 11:02:51 +01:00
owner ec9cca59da Duscarded object is now a struct. 2024-11-28 14:36:36 +01:00
owner 3d8f903d30 Unfiltered object is now a struct. 2024-11-28 14:24:33 +01:00
owner 69a22fb3eb Fixed. 2024-11-28 10:26:44 +01:00
owner f02203c14c Fixed duplicate insertions into the filtered database. 2024-11-27 15:53:00 +01:00
owner 4eaaf0f244 Fix communication between applications. 2024-11-27 11:33:53 +01:00
owner be07cdb13e Create runtime variables and remove MessagePack. 2024-11-26 19:40:27 +01:00
owner 9afddf803a Remove redundant comment. 2024-11-25 19:20:08 +01:00
owner 96b5ce4ff5 Fix NetMQ not stopping. 2024-11-25 19:13:37 +01:00
owner f3c6338a6a Initial commit 2024-11-24 12:40:51 +01:00