Header Filters

Technical support and discussion of Newsbin Version 6 series.

Header Filters

Postby bobberly » Thu May 18, 2017 1:16 pm

Thank you for the built-in option to specify a header filter, I've made very good use of it to reduce spam.

Should I be aware of any performance considerations as I add more and more entries to clean up headers? Are more complex entries better than the total number?

Regarding filters themselves, can you mix the accept-if, reject-if if they have overlapping matches? I noticed there wasn't a way to order, so I'm guessing it isn't a first-match, do action type system?

And I figured i'd ask if we can do this existing or not before a feature request, the existing filters look like we can only specify one set of size restrictions AND multiple (subject or email) entries. Any way to have a regex for just the size along with the subject? Mostly I'm trying to filter crap but preserve cases where an .nfo file or .par2 was posted separate.

I write SQL code all day long for work, so I keep wishing I could construct a query to include/exclude headers. The regex is powerful, just not sure how to handle multiple criteria.
bobberly
n00b
n00b
 
Posts: 4
Joined: Sun Mar 03, 2013 12:31 am

Registered Newsbin User since: 12/04/09

Re: Header Filters

Postby Quade » Thu May 18, 2017 3:52 pm

Should I be aware of any performance considerations as I add more and more entries to clean up headers? Are more complex entries better than the total number?


The more filters you have, the slower it'll run. You might be better off filtering IN the results you want instead of trying to filter out everything.

Regarding filters themselves, can you mix the accept-if, reject-if if they have overlapping matches? I noticed there wasn't a way to order, so I'm guessing it isn't a first-match, do action type system?


Basically it has to pass both to be accepted. It needs to match on the accept from the accept list and not match in the reject list. It's all or nothing.

And I figured i'd ask if we can do this existing or not before a feature request, the existing filters look like we can only specify one set of size restrictions AND multiple (subject or email) entries. Any way to have a regex for just the size along with the subject? Mostly I'm trying to filter crap but preserve cases where an .nfo file or .par2 was posted separate.


Size filtering doesn't work at the header filtering level. They're 99% 600-750K per header and they haven't been combined into files at this level. You could do size filtering at the watch list level. You could have a watch list you feed all the NFO's to for example and then feed the files to a different watch list.

I write SQL code all day long for work, so I keep wishing I could construct a query to include/exclude headers. The regex is powerful, just not sure how to handle multiple criteria.


Sqlite is popular and many tools exist for it. You could write something to post process the header DB's too after the import completes.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to V6 Technical Support

Who is online

Users browsing this forum: Google [Bot] and 2 guests