Version 6.73 Release Candidate 2 Available...

This is the place to help test and discuss Version 6 Beta releases.

Version 6.73 Release Candidate 2 Available...

Postby dexter » Wed Jan 25, 2017 10:53 pm

We have implemented an important change to help combat the current spam flood on Usenet. While Newsbin already had the capability to filter header downloads prior to insertion to the database, it was difficult to configure. Now you can apply a filter profile to header downloads from the Properties screen for either a Group Folder or a newsgroup itself. Instructions are in the Newsbin Help Wiki.

Change Summary:

  • Added a GUI setting for header filters in group options. Right-click the group or the group folder and select Properties to use it.
  • Made change to allow header filtering with "downloadfilter" per group or per group folder. Previously only worked on group folders.
  • Now removes filters that no longer exist from the the display when saving options for the group.
  • Fixed issue where Search on Poster setting is not maintained if using multiple search tabs.
  • Added full date info into logging timestamp for people who run Newsbin across multiple days.
  • Fixed issue when using multiple NZB tabs when Newsbin is in single tab mode.
  • Fixed issu with NZB Watch folder moving everything into the loaded folder, not just NZB's.
  • Fixed issue downloading files with illegal filenames in the subject.

The download link is on the Newsbin Beta Page. If you find any issues, please reply to this thread or use our Technical Support Contact Form.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby tl » Thu Jan 26, 2017 7:33 am

dexter wrote:We have implemented an important change to help combat the current spam flood on Usenet. While Newsbin already had the capability to filter header downloads prior to insertion to the database, it was difficult to configure. Now you can apply a filter profile to header downloads from the Properties screen for either a Group Folder or a newsgroup itself. Instructions are in the Newsbin Help Wiki.

Unfortunately the specific filter example on the webpage is still wrong and does nothing to stop the current attack.
It says: "nEwZ\[NZB\].*prd3@net.world" (without the quotes).
Should say: "nEwZ\[NZB\].*pr3d@net.world" (without the quotes).

IE, it should be pr3d instead of prd3. It's correct before and after, the "short" version of the filter without the nEwZ\[NZB\] part does work as written but it will also filter out significant number of posts from a real poster that has been using that email to post long before this spammer came along.

Also, at least with RC1 the Subject filtering doesn't seem to actually work with Header Filters, but I need to wait some hours at least for additional headers before I try to verify if that's still the case with RC2.
User avatar
tl
Seasoned User
Seasoned User
 
Posts: 109
Joined: Tue Jul 15, 2003 1:55 pm

Registered Newsbin User since: 04/01/03

Re: Version 6.73 Release Candidate 2 Available...

Postby Fisherking » Thu Jan 26, 2017 9:19 am

Dear Dex & Quade

The bug below still exists in this 673RC2

viewtopic.php?f=44&t=41264

Thanks
Fisherking
Active Participant
Active Participant
 
Posts: 57
Joined: Tue Oct 24, 2006 3:34 pm

Registered Newsbin User since: 10/24/06

Re: Version 6.73 Release Candidate 2 Available...

Postby dexter » Thu Jan 26, 2017 10:03 am

We were unable to reproduce this. If you can send us a step by step procedure to reproduce this problem every time, we can try again.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby Fisherking » Thu Jan 26, 2017 12:08 pm

dexter wrote:We were unable to reproduce this. If you can send us a step by step procedure to reproduce this problem every time, we can try again.


OK, where can I send the example .nzb and pics ?

Thanks
Fisherking
Active Participant
Active Participant
 
Posts: 57
Joined: Tue Oct 24, 2006 3:34 pm

Registered Newsbin User since: 10/24/06

Re: Version 6.73 Release Candidate 2 Available...

Postby dexter » Thu Jan 26, 2017 1:59 pm

ts at newsbin dot com
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby Fisherking » Thu Jan 26, 2017 3:34 pm

dexter wrote:ts at newsbin dot com


Mail Sent
Fisherking
Active Participant
Active Participant
 
Posts: 57
Joined: Tue Oct 24, 2006 3:34 pm

Registered Newsbin User since: 10/24/06

Re: Version 6.73 Release Candidate 2 Available...

Postby grantboucher » Thu Jan 26, 2017 4:28 pm

Awesome! Having the ability to filter Downloads everywhere (if needed) is fantastic. Thanks.

I think I found a bug though. When I choose any option in the "Look in Groups" dropdown in any Watch Topic, it doesn't remember the Group chosen and defaults to searching all newsgroups, no matter what Group Folder I select.
grantboucher
Occasional Contributor
Occasional Contributor
 
Posts: 25
Joined: Wed Aug 20, 2003 5:07 pm

Registered Newsbin User since: 04/10/03

Re: Version 6.73 Release Candidate 2 Available...

Postby Quade » Thu Jan 26, 2017 5:25 pm

I think I found a bug though. When I choose any option in the "Look in Groups" dropdown in any Watch Topic, it doesn't remember the Group chosen and defaults to searching all newsgroups, no matter what Group Folder I select.


I'll check it out, thanks.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42579
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby vspazv » Fri Jan 27, 2017 3:38 am

I thought it treated the name and the email as separate fields so I just use partial matches. Using 'Reject' 'Poster' 'pr3d@net\.world' worked for me.
vspazv
Occasional Contributor
Occasional Contributor
 
Posts: 31
Joined: Thu May 23, 2002 12:45 am

Registered Newsbin User since: 04/03/03

Re: Version 6.73 Release Candidate 2 Available...

Postby tl » Fri Jan 27, 2017 7:41 am

vspazv wrote:I thought it treated the name and the email as separate fields so I just use partial matches. Using 'Reject' 'Poster' 'pr3d@net\.world' worked for me.

It's one field, but if the poster name contains regular expression characters or spaces they need to be escaped or replaced by wild cards.

I'm using the fixed version of what Quade mentions on the new FilterDownloads page (fixing the "prd3" spelling error). I can confirm that this version definitely works too.
IE, the filter is: "Reject if", "Poster Contains", "nEwZ\[NZB\].*pr3d@net.world" (without the quotation marks).
User avatar
tl
Seasoned User
Seasoned User
 
Posts: 109
Joined: Tue Jul 15, 2003 1:55 pm

Registered Newsbin User since: 04/01/03

Re: Version 6.73 Release Candidate 2 Available...

Postby Quade » Fri Jan 27, 2017 12:29 pm

I thought it treated the name and the email as separate fields so I just use partial matches. Using 'Reject' 'Poster' 'pr3d@net\.world' worked for me.


It's a single regular expression to match an arbitrary string. It doesn't even know that it's an email address at that level. It's just looking for a match.

So, if you reject poster, if it matches the RE, it rejects the post. With accept poster, if it matches, it'll pass it on through. Same for the subject type field. You can match on as little or as much as you want. That's why partial matches work.

I want to repeat that matching IN (by accept filtering) the posters and subjects you like is superior to rejecting things you don't like. The problem with adding a bunch of rejects is that there's no upper limit to how many you reject and the more filters you add, the slower the process will be.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42579
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby grantboucher » Sun Jan 29, 2017 1:26 am

Just a heads up. The spammer seems to be back with a new script. He's bombarding the same forums with files in the format of X (X@X.X) where X is a random string of digits. The X changes per post.

Short of choosing a DownloadFilter by Min File Size (which seems like a VERY bad idea, yes?), I'm not sure how to set a filter for this expression, if at all possible.

Clearly this person isn't actual a spammer with a sales agenda, but a troll trying to bring down Usenet Readers and, I assume, indexing services.
grantboucher
Occasional Contributor
Occasional Contributor
 
Posts: 25
Joined: Wed Aug 20, 2003 5:07 pm

Registered Newsbin User since: 04/10/03

Re: Version 6.73 Release Candidate 2 Available...

Postby Bismark » Sun Jan 29, 2017 2:03 am

Yes, spammer is back...

One more problem, with millions of files, if one selects them and tries to delete them, Newsbin seems to stop working and has to be killed. I've waited half an hour and nothing. I have a very fast and high memory windows system. It should be possible to select a few million records and just delete them.
Bismark
Seasoned User
Seasoned User
 
Posts: 159
Joined: Fri Feb 26, 2010 4:03 am

Registered Newsbin User since: 02/27/10

Re: Version 6.73 Release Candidate 2 Available...

Postby don_colchon » Sun Jan 29, 2017 4:03 am

This regex profile (reject if / poster contains) should be able to filter out the new form of the spammer before inserting the headers into the database:

([A-Za-z0-9]{9})\s<\1@\1\.\1>

Locally it worked with UNIX shell:

$ echo "4df7a1578 <4df7a1578@4df7a1578.4df7a1578>"|sed -r "s/([A-Za-z0-9]{9})\s<\1@\1\.\1>/xxx/"
xxx

It doesn't work with newsbin. I found this viewtopic.php?f=4&t=39645 and tried this filter (reject if / poster contains):

(?<first>[A-Za-z0-9]{9})\s<\k<first>@\k<first>\.\k<first>>

I positively tested it against some groups with a lot of posts. This seems to be working. Or killing too much during database load. I don't know yet.

What do you think?

During testing I found huge read I/O (650 MByte/s) for some minutes against my SSD. Which is understandable.
BTW: The test Result in the filter Profile is always "Test text was Accepted - Will be included" regardless of what status/source I selected during testing.
..come un gatto vorrei vivere..
User avatar
don_colchon
Occasional Contributor
Occasional Contributor
 
Posts: 28
Joined: Sat Jun 07, 2003 11:04 pm

Registered Newsbin User since: 10/17/02

Re: Version 6.73 Release Candidate 2 Available...

Postby don_colchon » Sun Jan 29, 2017 5:33 am

The spammer filled/fills the groups with a combined speed of ~10000 headers per second.

With newsbin's toolset I'm unable to clean the dirt.

With sqllitestudio, etc. it is possible to join the two databases Storage.db3 & StorageData.db3 and go ahead with some SQL's... Group by group. Not the best way for every user, I guess.

Is it possible to do it with some new newsbin function, too? E.g. Post Storage -> Delete Stored Posts with filter -> "enter some filter regex" ?
..come un gatto vorrei vivere..
User avatar
don_colchon
Occasional Contributor
Occasional Contributor
 
Posts: 28
Joined: Sat Jun 07, 2003 11:04 pm

Registered Newsbin User since: 10/17/02

Re: Version 6.73 Release Candidate 2 Available...

Postby petewagner2002 » Sun Jan 29, 2017 3:13 pm

I get an error message when I click on the download button for this beta:

"\AppData\Local\Temp\WFcmdL+e.exe.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator."

Is this me or you?

Thanks!
petewagner2002
n00b
n00b
 
Posts: 8
Joined: Fri Feb 10, 2012 12:36 am

Registered Newsbin User since: 05/07/08

Re: Version 6.73 Release Candidate 2 Available...

Postby dexter » Sun Jan 29, 2017 5:54 pm

petewagner2002 wrote:I get an error message when I click on the download button for this beta:

"\AppData\Local\Temp\WFcmdL+e.exe.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator."

Is this me or you?

Thanks!


I'd say it's you since we've seen lots of downloads and no other complaints. Perhaps some security software on your machine is interfering or something.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9297
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby petewagner2002 » Mon Jan 30, 2017 12:33 pm

It was my antivirus (Bitdefender) blocking the download. Had never happened on any former release of Newsbin nor on any other software download (including exe files). Anyway, thanks for the response so that I knew where to start looking for the solution.
petewagner2002
n00b
n00b
 
Posts: 8
Joined: Fri Feb 10, 2012 12:36 am

Registered Newsbin User since: 05/07/08

Re: Version 6.73 Release Candidate 2 Available...

Postby Co6aka » Wed Feb 01, 2017 1:24 pm

FWIW, there's still a long-existing minor bug where posts added to the wishlist don't indicate so after closing and reopening Newsbin. They're still in the wishlist, but that they are is no longer indicated so in the group or GoG.
Co6aka
Active Participant
Active Participant
 
Posts: 64
Joined: Wed Nov 09, 2005 2:06 pm
Location: Sobachi Dom

Registered Newsbin User since: 04/12/05

Re: Version 6.73 Release Candidate 2 Available...

Postby Rolo71 » Wed Feb 01, 2017 2:18 pm

Fixed issu with NZB Watch folder moving everything into the loaded folder, not just NZB's.


Still moving .rar to the loaded folder here. It leaves the .par files
Rolo71
n00b
n00b
 
Posts: 2
Joined: Tue Nov 14, 2006 12:14 pm

Registered Newsbin User since: 10/27/04

Re: Version 6.73 Release Candidate 2 Available...

Postby chilly » Thu Feb 02, 2017 3:29 pm

Rolo71 wrote:
Fixed issu with NZB Watch folder moving everything into the loaded folder, not just NZB's.


Still moving .rar to the loaded folder here. It leaves the .par files


I can confirm I'm observing same behavior.
chilly
n00b
n00b
 
Posts: 3
Joined: Mon Feb 08, 2016 4:12 pm

Registered Newsbin User since: 09/05/11

Re: Version 6.73 Release Candidate 2 Available...

Postby Quade » Thu Feb 02, 2017 4:42 pm

It moves files that can contain NZB files. That means RAR/ZIP/GZ and NZB. You really should make the NZB watch/black hole folder exclusively for NZB's and not dump other files into the folder. Firstly it has to scan inside the files so, you get a performance hit for every RAR and ZIP. Secondly it will move them to loaded.

The change in RC2 prevents it from moving all your shortcuts if someone is...unwise enough to make it watch on the desktop (yes a real thing) but it assumes the folder is for NZBs, not some dumping ground for 1000's of files.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42579
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby chilly » Sat Feb 04, 2017 4:14 pm

Quade wrote:It moves files that can contain NZB files. That means RAR/ZIP/GZ and NZB. You really should make the NZB watch/black hole folder exclusively for NZB's and not dump other files into the folder. Firstly it has to scan inside the files so, you get a performance hit for every RAR and ZIP. Secondly it will move them to loaded.

The change in RC2 prevents it from moving all your shortcuts if someone is...unwise enough to make it watch on the desktop (yes a real thing) but it assumes the folder is for NZBs, not some dumping ground for 1000's of files.


I'm downloading all my files to one folder. It's just habit but difficult one to change. :(
In previous versions NewsBin worked great. Picked just NZBs and RARs containing NZBs and moved them to Loaded folder (FANTASTIC). Now this is gone. :roll:
chilly
n00b
n00b
 
Posts: 3
Joined: Mon Feb 08, 2016 4:12 pm

Registered Newsbin User since: 09/05/11

Re: Version 6.73 Release Candidate 2 Available...

Postby Quade » Sun Feb 05, 2017 5:39 am

I'm downloading all my files to one folder. It's just habit but difficult one to change.


You can download them to any folder you want. You just want to make the NZB's go to a special NZB folder. You make it sound like you're downloading the NZB's from usenet instead of downloading the actual files.

There is a hidden feature that makes Newsbin automatically download any downloaded NZBs. I'll have to look it up though.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 42579
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.73 Release Candidate 2 Available...

Postby bob1324 » Mon Feb 13, 2017 1:21 pm

Still have this problem:

viewtopic.php?f=44&t=41099

Everything goes to a nzbs folder.
bob1324
Occasional Contributor
Occasional Contributor
 
Posts: 28
Joined: Wed Dec 17, 2003 7:53 pm

Registered Newsbin User since: 12/16/03

Re: Version 6.73 Release Candidate 2 Available...

Postby vspazv » Wed Feb 15, 2017 11:06 pm

The "New Files" column in the Groups List window keeps displaying 0. Hitting the +/- next to the groups makes them go back and forth randomly from 0 to normal numbers. If I expand and collapse a group of newsgroups repeatedly it toggles between showing 0s for everything, normal numbers on the selected subgroup only, normal numbers for everything.

Image
vspazv
Occasional Contributor
Occasional Contributor
 
Posts: 31
Joined: Thu May 23, 2002 12:45 am

Registered Newsbin User since: 04/03/03


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 8 guests