Optimize Content Filter Inner Loop #18

New Issue

owner · 2024-11-28T16:44:16Z

owner commented

2024-11-28 16:44:16 +00:00

		Backend/Handler/ContentFilter.cs
		Line 54 in 701ffff27e
	
				            for (long i = 0; i < indexes; i++)

This way of filtering is rather inefficient, since the number only grows with each IP inserted to the unfiltered database, so the loop has to go from 0 --> check if that id has been filtered --> continue to next iteration because it has already been filtered, and then start filtereing when it's done looking through the whole unfiltered database.

It would be better if we instead just returned an array of Id's that hasn't been filtered yet, and loop over them.

This should also help lessen the pressure on the GC, since _dbHandler.ReadUnfilteredWithId(id); has to be called less. And the less we call it, the fewer SqliteConnection and SqliteDataReader objects we have to create. Plus it also gives a little speed-boost to loop through less rows in the database.

https://git.rbwr.dk/owner/RSE/src/commit/701ffff27e54926c6fd6042f90c1f9ad01c955b4/Backend/Handler/ContentFilter.cs#L54 This way of filtering is rather inefficient, since the number only grows with each IP inserted to the unfiltered database, so the loop has to go from 0 --> check if that id has been filtered --> continue to next iteration because it has already been filtered, and then start filtereing when it's done looking through the whole unfiltered database. It would be better if we instead just returned an array of Id's that hasn't been filtered yet, and loop over them. This should also help lessen the pressure on the GC, since `_dbHandler.ReadUnfilteredWithId(id);` has to be called less. And the less we call it, the fewer `SqliteConnection` and `SqliteDataReader` objects we have to create. Plus it also gives a little speed-boost to loop through less rows in the database.

owner added this to the Minimize GC Pressure milestone 2024-11-28 16:44:16 +00:00

owner added the

enhancement

label 2024-11-28 16:44:16 +00:00

owner self-assigned this 2024-11-28 16:44:16 +00:00

owner added reference OptimizeContentFilter

2024-11-29 18:13:45 +00:00

owner closed this issue

2024-11-29 18:17:50 +00:00

Sign in to join this conversation.