Commit Graph
25 Commits
Author SHA1 Message Date
owner c7fcc3297f Reduced the memory usage, as well as hopefully fixing the memory leak. Also implemented database compression across rows, including Brotli compression and bitshifting. 2025-04-05 11:28:54 +02:00
owner b0318b7759 Removed .txt 2025-02-16 09:38:57 +01:00
owner c70b42c9d7 Made a lot of changes. Enhanced memory usage. 2025-02-16 09:38:00 +01:00
owner 8d659d4261 Implement autosave 2025-02-08 13:02:18 +01:00
owner 45b6d8ecab Preparing for multithreaded content filter 2025-02-07 23:30:25 +01:00
owner 83897c72cf Way too much happened for me to create a commit message 2025-02-07 18:44:22 +01:00
owner 7abe904997 Testing search query 2025-01-05 15:37:18 +01:00
owner a6804be9f1 Implement database compression 2024-12-30 18:36:04 +01:00
owner 0cfbcea252 Testing database compression. 2024-12-24 14:23:23 +01:00
owner e9fd738f1f Title and description extraction now only look in the head section of the HTML document 2024-12-23 14:09:51 +01:00
owner 580d687f62 Add user-agent 2024-12-21 21:57:01 +01:00
owner 5fe35192c6 Misc changes such as performance tuning, added support for more than 64 threads for the scanner. 2024-12-04 13:20:45 +01:00
owner 2cff817044 Make it so the filter does not iterate over already filtered IPs. 2024-11-29 19:13:27 +01:00
owner f2ace6f571 Reworked the queue items. 2024-11-29 12:59:07 +01:00
owner 3034e66126 Changed GET to HEAD for checking robots txt. 2024-11-29 11:02:51 +01:00
owner ec9cca59da Duscarded object is now a struct. 2024-11-28 14:36:36 +01:00
owner 3d8f903d30 Unfiltered object is now a struct. 2024-11-28 14:24:33 +01:00
owner 69a22fb3eb Fixed. 2024-11-28 10:26:44 +01:00
owner f02203c14c Fixed duplicate insertions into the filtered database. 2024-11-27 15:53:00 +01:00
owner 4eaaf0f244 Fix communication between applications. 2024-11-27 11:33:53 +01:00
owner be07cdb13e Create runtime variables and remove MessagePack. 2024-11-26 19:40:27 +01:00
owner 9297cf4358 Remove obj folder from GIT. 2024-11-25 19:21:15 +01:00
owner 9afddf803a Remove redundant comment. 2024-11-25 19:20:08 +01:00
owner 96b5ce4ff5 Fix NetMQ not stopping. 2024-11-25 19:13:37 +01:00
owner f3c6338a6a Initial commit 2024-11-24 12:40:51 +01:00