Commit Graph

57 Commits

Author SHA1 Message Date
c7fcc3297f Reduced the memory usage, as well as hopefully fixing the memory leak. Also implemented database compression across rows, including Brotli compression and bitshifting. 2025-04-05 11:28:54 +02:00
b0318b7759 Removed .txt 2025-02-16 09:38:57 +01:00
c70b42c9d7 Made a lot of changes. Enhanced memory usage. 2025-02-16 09:38:00 +01:00
8d659d4261 Implement autosave 2025-02-08 13:02:18 +01:00
45b6d8ecab Preparing for multithreaded content filter 2025-02-07 23:30:25 +01:00
83897c72cf Way too much happened for me to create a commit message 2025-02-07 18:44:22 +01:00
680b295f9e Update gitignore 2025-01-16 12:27:23 +01:00
03e7ebe421 Merge pull request 'SearchFunction' (#45) from SearchFunction into main
Reviewed-on: #45
2025-01-16 10:12:41 +00:00
41dfb5809a Add stuff 2025-01-16 11:10:43 +01:00
7abe904997 Testing search query 2025-01-05 15:37:18 +01:00
8256d6a681 Merge pull request 'CompressDatabase' (#43) from CompressDatabase into main
Reviewed-on: #43
2024-12-30 17:47:33 +00:00
a6804be9f1 Implement database compression 2024-12-30 18:36:04 +01:00
0cfbcea252 Testing database compression. 2024-12-24 14:23:23 +01:00
3f647a3e68 Merge pull request 'Add search page' (#41) from CreateSearchPage into main
Reviewed-on: #41
2024-12-24 10:11:15 +00:00
53289db102 Add search page 2024-12-24 11:10:38 +01:00
f4d94a3336 Merge pull request 'Title and description extraction now only look in the head section of the HTML document' (#40) from OptimizeTitleAndDescriptionExtraction into main
Reviewed-on: #40
2024-12-23 13:10:49 +00:00
e9fd738f1f Title and description extraction now only look in the head section of the HTML document 2024-12-23 14:09:51 +01:00
e581d947f8 Merge pull request 'Add user-agent' (#39) from AddUserAgent into main
Reviewed-on: #39
2024-12-21 21:19:28 +00:00
580d687f62 Add user-agent 2024-12-21 21:57:01 +01:00
af7b385641 Merge pull request 'Add cached progress response' (#36) from ImplementAPICache into main
Reviewed-on: #36
2024-12-19 17:37:01 +00:00
7562bbf7d1 Add cached progress response 2024-12-19 18:36:13 +01:00
ac645b01b9 Merge pull request 'OptimiseFrontend' (#35) from OptimiseFrontend into main
Reviewed-on: #35
2024-12-18 13:28:03 +00:00
97321492ba Add headers 2024-12-18 14:27:31 +01:00
8628d31bec Add nuxt-purgecss to further minimize bundle size. 2024-12-18 12:02:35 +01:00
23d0a8b978 Merge pull request 'UseDictionary' (#34) from UseDictionary into main
Reviewed-on: #34
2024-12-17 20:33:08 +00:00
8b55cf7bb6 Reworked the /progress page 2024-12-17 13:15:47 +01:00
3822339dd9 Add a time-based cache for the progress result from the API 2024-12-17 10:46:10 +01:00
1c74ae9de5 Added frontend and working fetching. 2024-12-14 13:26:21 +01:00
bab8fa6c25 Added frontend project with some basic api fetching implemented. 2024-12-06 12:33:48 +01:00
cd49d32dd7 Merge pull request 'Misc changes such as performance tuning, added support for more than 64 threads for the scanner.' (#24) from UseForeignKeys into main
Reviewed-on: #24
2024-12-04 12:24:34 +00:00
5fe35192c6 Misc changes such as performance tuning, added support for more than 64 threads for the scanner. 2024-12-04 13:20:45 +01:00
a1f5b08ac3 Merge pull request 'Make it so the filter does not iterate over already filtered IPs.' (#22) from OptimizeContentFilter into main
Reviewed-on: #22
2024-11-29 18:18:00 +00:00
2cff817044 Make it so the filter does not iterate over already filtered IPs. 2024-11-29 19:13:27 +01:00
407bbc1556 Merge pull request 'Reworked the queue items.' (#21) from SplitUpQueueItem into main
Reviewed-on: #21
2024-11-29 13:21:44 +00:00
f2ace6f571 Reworked the queue items. 2024-11-29 12:59:07 +01:00
3034e66126 Changed GET to HEAD for checking robots txt. 2024-11-29 11:02:51 +01:00
701ffff27e Merge pull request 'Duscarded object is now a struct.' (#17) from ConvertDiscardedToStruct into main
Reviewed-on: #17
2024-11-28 13:37:01 +00:00
ec9cca59da Duscarded object is now a struct. 2024-11-28 14:36:36 +01:00
90d1d6b429 Merge pull request 'Unfiltered object is now a struct.' (#15) from ReworkUnfilteredObject into main
Reviewed-on: #15
2024-11-28 13:25:35 +00:00
3d8f903d30 Unfiltered object is now a struct. 2024-11-28 14:24:33 +01:00
eedca27a11 Merge pull request 'Fixed.' (#14) from FixGetProgress into main
Reviewed-on: #14
2024-11-28 10:32:39 +00:00
69a22fb3eb Fixed. 2024-11-28 10:26:44 +01:00
d3270d3117 Merge pull request 'Fixed duplicate insertions into the filtered database.' (#11) from IgnoreAlreadySavedIpInTheFilteredDatabaseAfterResuming into main
Reviewed-on: #11
2024-11-27 15:09:13 +00:00
f02203c14c Fixed duplicate insertions into the filtered database. 2024-11-27 15:53:00 +01:00
7ad691c439 Merge pull request 'CreateRuntimeVariable' (#8) from CreateRuntimeVariable into main
Reviewed-on: #8
2024-11-27 10:40:12 +00:00
4eaaf0f244 Fix communication between applications. 2024-11-27 11:33:53 +01:00
a160992c08 Remove random sqlite file. 2024-11-26 19:44:55 +01:00
1a361a9524 Create runtime variables and remove MessagePack. 2024-11-26 19:41:12 +01:00
be07cdb13e Create runtime variables and remove MessagePack. 2024-11-26 19:40:27 +01:00
28c5918a83 Update README.md 2024-11-25 18:32:32 +00:00