Version 6.80 Beta 1 Available...

This is the place to help test and discuss Version 6 Beta releases.

Version 6.80 Beta 1 Available...

Postby dexter » Mon Apr 24, 2017 3:00 pm

Version 6.80 Beta 1 is now available for public beta testing. It has an improved download engine that will do a better job plowing through some of the funkiness that has been showing up on Usenet lately. It may even be a tick faster than it used to be. Other than that, mostly fixes that have been reported in the 6.7x versions.

It also has changes to better handle posts coming in from Sonarr or SickBeard with obfuscated subjects. We would like to hear from people using Newsbin with Sonarr and SickBeard to see if Newsbin is working better with these style posts.

Change Summary:

  • Fixed issue where Newsbin couldn't download if there was an illegal filename in subject.
  • Fixed issue where the "Look in Groups" dropdown in a Watch Topic wasn't remembering the Group chosen and defaulted to searching all newsgroups.
  • Removed confirmation prompt when performaing a "Mark All Old" since it is non-destructive and reversable.
  • Fixed issue in AutoPar when the first par2 file is corrupt. Now triggers other par downloads.
  • Now compacts all NZB records that come from Sonarr or SickBeard to solve issue displaying post sets with obscured filenames.
  • Fixed issue where automatic assemble incompletes wasn't working correctly.
  • Resolved intermittent stalling issue when retries fail.
  • Reduced logging in some cases. Caused the disks to get hammered.
  • NZB Autoload no longer looks at rar files larger than 5MB. There is no limit in the size of NZB files
  • Change AutoPAR to go ahead and unrar 0 byte files. Sometimes they are meaningful files.
  • The performance option "Fail Chunks with bad yEnc Formats" is now on by default. Most are correct these days. Bad ones are usually spam.
  • In cases where the yEnc filename doesn't contain an extension now uses the filename from the Subject.
  • Change in the download engine to download raw data and send to decoder after the fact instead of processing on the fly.
  • Speed and reliabilty enhancements to yEnc, Mime, and UU decoding functions since they are now decoupled from the download engine.

The download link is on the Newsbin Beta Page. If you find any issues, please reply to this thread or use our Technical Support Contact Form.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9511
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby mimauk » Tue Apr 25, 2017 3:20 am

Different par behaviour when downloading.

Loaded 22 new files to the Download list from different groups/gog and NBPro downloaded them all in sequence - all progress bars turned blue - without reassembling them.

Then went back to the top of the list and counted all the blocks downloaded for each file in sequence and then went back to the top of the list and started adding any needed par files and reassembling the posts.
mimauk
Seasoned User
Seasoned User
 
Posts: 327
Joined: Sun Aug 20, 2006 12:35 pm

Registered Newsbin User since: 03/29/06

Re: Version 6.80 Beta 1 Available...

Postby mimauk » Tue Apr 25, 2017 7:33 am

Just tried another series of downloads a few hours later and the par and reassembling were back to the usual behaviour of previous versions.

Maybe something was reset when the new beta was installed and after NBPro was shut down for a few hours??? - puzzling!
mimauk
Seasoned User
Seasoned User
 
Posts: 327
Joined: Sun Aug 20, 2006 12:35 pm

Registered Newsbin User since: 03/29/06

Re: Version 6.80 Beta 1 Available...

Postby Quade » Tue Apr 25, 2017 9:39 am

You can see this if the download speed is really fast and the program is set to not pause the download during unrar/repair. Basically you run out of disk bandwidth so assembling files and handling the par processing is delayed.

In the performance options, if you tell it to pause download while doing unrar/repair this probably won't happen.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby mimauk » Tue Apr 25, 2017 11:04 am

Thanks for the reply - I've got a 200MB connection so that would explain it.
mimauk
Seasoned User
Seasoned User
 
Posts: 327
Joined: Sun Aug 20, 2006 12:35 pm

Registered Newsbin User since: 03/29/06

Re: Version 6.80 Beta 1 Available...

Postby johnisme » Wed Apr 26, 2017 5:33 pm

I installed 6.80b1 and went to d/l a file i had previously downloaded,it wouldn't d/l no matter did,right clicked on it and selected the bypass filter and had dupe file unchecked.Reinstalled 6.73 4821 and it downloaded the file with no issues,tried this with several files with 6.8 and no go.

Is there a setting I'm missing in 6.8b1?
User avatar
johnisme
Active Participant
Active Participant
 
Posts: 92
Joined: Mon Sep 24, 2007 12:00 am

Registered Newsbin User since: 09/22/07

Re: Version 6.80 Beta 1 Available...

Postby Quade » Wed Apr 26, 2017 6:15 pm

Is the file still sitting on disk?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby johnisme » Wed Apr 26, 2017 6:22 pm

Nope,was deleted a while ago.
User avatar
johnisme
Active Participant
Active Participant
 
Posts: 92
Joined: Mon Sep 24, 2007 12:00 am

Registered Newsbin User since: 09/22/07

Re: Version 6.80 Beta 1 Available...

Postby Quade » Wed Apr 26, 2017 11:22 pm

I'll have to try it then. Sounds like it's not bypassing the dup check. As a work around you could disable the dup check in the advanced options.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby johnisme » Wed Apr 26, 2017 11:35 pm

I did uncheck the dupe check in options/settings/advanced so i could d/l files that got caught in the dupe check because they had the same first few minutes of video as the previous week.
User avatar
johnisme
Active Participant
Active Participant
 
Posts: 92
Joined: Mon Sep 24, 2007 12:00 am

Registered Newsbin User since: 09/22/07

Re: Version 6.80 Beta 1 Available...

Postby johnisme » Sat Apr 29, 2017 1:35 am

I did some further test on 6.80 and found that it would d/l files which have been rared but not files which are whole(not rared) such as a mp4.
User avatar
johnisme
Active Participant
Active Participant
 
Posts: 92
Joined: Mon Sep 24, 2007 12:00 am

Registered Newsbin User since: 09/22/07

Re: Version 6.80 Beta 1 Available...

Postby Quade » Sat Apr 29, 2017 1:49 am

If we're talking search based, I have a fix in the works for this. It'll be in B2. Hopefully early next week.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby grantboucher » Sat Apr 29, 2017 9:47 pm

[edit: solution to problem presented in Edit3 at the end]

Hey man!

This broke something weird for me. When I download a group of headers (18+ x2 servers for overlap) it completely freezes my entire system until the downloading data clears whatever its clogging up.

Yes, the CPU (4 cores + HT = 8 ) pegs at 100%, but no other multithreaded program gives me the same system freeze. And I do a lot of video compression, 3d rendering, etc.

The best way to describe it is that it "feels" like one of those full DMA system freezes from many years ago.

I tried SSD and sata HD and nothing changed. Went back to the previous release version and the problem went away, so it's clearly something odd with this new beta.

My best guess is something in the beta itself is freezing up when CPU usage is maxed.

This is W10x64 CU, 12 GB ram, Intel 975 extreme CPU. The only thing really customized on this would be the settings for the Realtek PCIe gigabit network adapter, which have been optimized to reduce bufferbloat across my 350 megabit DL connection. But, again, this freeze does not happen except with the beta.

Do you have an obvious culprit?

I hope this is useful beta feedback. :)

Edit: Back the release version, I can make the same thing happen for a few fractions of a second, so there's something about the beta that more efficient about gumming up the works. hehe I'll try making some network card changes to see if some kind of network offloading is being hammered. Will advise.

Edit2: GOT IT! It was X-features. No change on the LAN card seems to have made any difference. Based on the previous comments, I figured I'd try this option. The system comes to a complete crawl with BOTH giganews and astraweb and xfeatures enabled (disable option unchecked), in BOTH the release version and this new beta, so maybe that can help you track this down (if possible). This also explains why there's no problem with downloading actual files or any other network access. I hope this helps. Thanks again for all of your amazing work. :)

Edit3: I was able to find a solution at my end. Moving Newsbin to a BelowNormal task priority (via task manager/details pane) solved the problem with xfeatures on header import. The best way I can describe the problem is that the system freezes as if Newsbin's header import suddenly jumps to realtime task priority, causing everything else in my system to fight for resources. Not sure if you need to look at anything at your end (or if this is just something wrong with my system at a core level), but I thought I would post a solution in case you or anyone else finds this. In the interim, I have created a shortcut that can launch Newsbin at the lower task priority. Best regards.
Last edited by grantboucher on Wed May 03, 2017 1:05 am, edited 3 times in total.
grantboucher
Occasional Contributor
Occasional Contributor
 
Posts: 28
Joined: Wed Aug 20, 2003 5:07 pm

Registered Newsbin User since: 04/10/03

Re: Version 6.80 Beta 1 Available...

Postby Biship » Tue May 02, 2017 10:00 am

I get a hard crash when downloading headers, but only for some (one?) group. I can't tell which group it is.
Logfile.txt is empty.

Also newsbin shows: Cache 200/200 (1), when there are 2 (~45MB) gz files in the Import folder. Interestingly it also showed the same when there was only 1 gz file. Each file took 30-60s to process.

Also, I have 3 providers, and I have one group (from only provider usenetexpress), that doesn't seem to download the headers. Every time I perform Download Latest Headers, it shows 0/292,467 for about 15s, then 0/157,352 for about 10s, then it goes away (for that one provider).
This occurs everytime I download latest for that group. The other providers download (just the 100 XOVER) almost instantly.

I renamed the group to alt.test in the log file:
Code: Select all
[05/02 08:43:00]  HIGH  PAR Pausing:  Disabled
[05/02 08:43:00]  HIGH  Par2Repair - Multi-core VC2015 2.0
[05/02 08:43:00]  HIGH  MessageId Filter: Disabled
[05/02 08:43:00]  HIGH  FileTypeId Version: 3.00
[05/02 08:43:18]  DEBUG Connecting: news.frugalusenet.com Index:2
[05/02 08:43:18]  DEBUG Connecting: news.usenetexpress.com Index:3
[05/02 08:43:18]  DEBUG Connecting: ssl-us.astraweb.com Index:5
[05/02 08:43:18]  DEBUG Connection Status: news.frugalusenet.com SSL Version: TLSv1.2 TLS-RSA-WITH-AES-128-CBC-SHA
[05/02 08:43:18]  DEBUG Connection Status: news.usenetexpress.com SSL Version: TLSv1.2 TLS-RSA-WITH-AES-128-CBC-SHA
[05/02 08:43:18]  DEBUG Connected: news.frugalusenet.com Index:2
[05/02 08:43:18]  DEBUG Stored Range: Min: 480373 Max: 480373 alt.test news.frugalusenet.com
[05/02 08:43:18]  DEBUG Group Numbers: alt.test Min: 432477 Max: 480373 news.frugalusenet.com
[05/02 08:43:18]  DEBUG Server:news.frugalusenet.com No new records since last header update :432477 : 480373 - alt.test
[05/02 08:43:18]  DEBUG Connected: news.usenetexpress.com Index:3
[05/02 08:43:18]  DEBUG Stored Range: Min: 135215 Max: 135215 alt.test news.usenetexpress.com
[05/02 08:43:18]  DEBUG Group Numbers: alt.test Min: 1 Max: 292467 news.usenetexpress.com
[05/02 08:43:18]  DEBUG Total XOVER Size:alt.test - 135,115:292,467
[05/02 08:43:18]  DEBUG Group Numbers: alt.test Reps: 1 news.usenetexpress.com
[05/02 08:43:18]  DEBUG XOVER Numbers: alt.test Min: 135115 Max: 292467 news.usenetexpress.com
[05/02 08:43:18]  DEBUG Connection Status: ssl-us.astraweb.com SSL Version: TLSv1.0 TLS-RSA-WITH-AES-256-CBC-SHA
[05/02 08:43:18]  DEBUG news.usenetexpress.com alt.test XOVER: alt.test 135115:292467
[05/02 08:43:19]  DEBUG Connected: ssl-us.astraweb.com Index:5
[05/02 08:43:19]  DEBUG Stored Range: Min: 683599 Max: 683599 alt.test ssl-us.astraweb.com
[05/02 08:43:19]  DEBUG Group Numbers: alt.test Min: 208177 Max: 683599 ssl-us.astraweb.com
[05/02 08:43:19]  DEBUG Server:ssl-us.astraweb.com No new records since last header update :208177 : 683599 - alt.test
[05/02 08:45:19]  DEBUG Download: Idle Connection Closed:news.frugalusenet.com
[05/02 08:45:19]  DEBUG Download: Idle Connection Closed:news.usenetexpress.com
[05/02 08:45:20]  DEBUG Download: Idle Connection Closed:ssl-us.astraweb.com


EDIT: Still occurs in B2
Biship
Active Participant
Active Participant
 
Posts: 88
Joined: Sun Jul 27, 2003 10:14 am

Registered Newsbin User since: 04/01/03

Re: Version 6.80 Beta 1 Available...

Postby ddaniel51 » Sun May 07, 2017 1:11 am

Both 1 & 2 crashed during updating headers so fast I didn't have a chance to tweak settings or get any clues.
Had to install 6.73 and rebuild everything to get running again.
ddaniel51
Occasional Contributor
Occasional Contributor
 
Posts: 39
Joined: Fri Aug 15, 2003 10:44 pm
Location: Arizona

Registered Newsbin User since: 04/05/03

Re: Version 6.80 Beta 1 Available...

Postby Quade » Sun May 07, 2017 1:16 am

Not getting any other reports so it's going to be hard to track down. If I had to guess, I'd guess there's some issue reading the GZ files out of the data folder. The fact the beta's crash and not the release makes me wonder if you've told you security package to ignore All three versions when reading and writing data to the data folder?
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby Biship » Sun May 07, 2017 7:02 am

Security package?
I also went back to 6.73 and it is fine.
6.80 B2 still crashes for me.
Biship
Active Participant
Active Participant
 
Posts: 88
Joined: Sun Jul 27, 2003 10:14 am

Registered Newsbin User since: 04/01/03

Re: Version 6.80 Beta 1 Available...

Postby Quade » Sun May 07, 2017 10:02 am

The GZ filename will tell you which group the problem might be from. It clearly might be group specific so it might be a group I don't use.

If you figure out what group you think it is, I can try it here.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby ddaniel51 » Sun May 07, 2017 12:09 pm

Disabled X features in nbi and now it works.

Tnx
Dave
ddaniel51
Occasional Contributor
Occasional Contributor
 
Posts: 39
Joined: Fri Aug 15, 2003 10:44 pm
Location: Arizona

Registered Newsbin User since: 04/05/03

Re: Version 6.80 Beta 1 Available...

Postby Quade » Sun May 07, 2017 6:07 pm

Disabled X features in nbi and now it works.


Yeah, typically this means your security package is messing with the compressed connection. New versions of Newsbin mean you have to tell your security software (Virus scanners/Firewalls) to ignore Newsbin again. New versions look like a new program to many security suites so, you have to reset them to also ignore the new version.

Avast and Kaspersky see to be the most aggressive about messing with the compressed header connections.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby nethead » Thu Jun 01, 2017 3:15 pm

finding "Fixed issue where automatic assemble incompletes wasn't working correctly" is not working using v6.80 b2

having this same issue for months with 6.7 and now 6.8;

files loaded via nzb - no passwords/no disk space issue

example:
a part of a rar file has an error downloading
no auto 'assemble incompletes' happens
expanding the file shows the rar/par files, one rar (or more) shows as incomplete
right click file and select 'assemble incompletes'
assemble incompletes finishes
auto-unrar happens

starting forcing myself to use nzbget to dl overnight vs letting newsbin dl overnight so i don't come back to see long list of incompletes (lol)

-----
oh, confirmed turning on pause download during unrar helps when moving to a FAST Dl connection, went from 150 to 1Gb and had A LOT of Blues cached (lol)
nethead
Occasional Contributor
Occasional Contributor
 
Posts: 44
Joined: Sat Feb 12, 2011 3:03 pm

Registered Newsbin User since: 02/25/06

Re: Version 6.80 Beta 1 Available...

Postby Quade » Thu Jun 01, 2017 5:43 pm

When I download in debug mode, I purposely damage every file (in software) so I force it to assemble and repair every set on its own.

You can control when it gives up and assembles with the retry setting and with "aggressive mode". I'd suggest 2 retries and turning on aggressive mode and see what happens.

Your issue isn't "assemble incomplete". The fact you can assemble them from the right click, shows that it works. You issue is letting Newsbin know when to give up and assemble the files to repair on it's own. This is gated by the retries. "Aggressive Mode" pulls the smarts out of it. It becomes mechanical. After retries expire, it assembles what it has. If the PARS are also missing chunks, they go through the same process. They retry 2 times then they're assembled too till eventually the download has enough blocks to repair or all the PARS download.

New beta should be up today. It has a work around for some obscured files that are purposely posted damaged.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Version 6.80 Beta 1 Available...

Postby nethead » Thu Jun 01, 2017 10:26 pm

thanks! :)
will make the change and report back
years of using (has been over a decade now? (lol)) NewbinPro and it's the best there is....

Update:
Upgraded to v6.80b4
Found Aggressive Mode was already turned on(i kind of remember reading a long ago post about turning it on to help unrar and guess i turned it on then), turned it off to see if a difference, closed newsbin, tried dl again, same issue when turned off (as expected since the idea was to turn it on). turned Aggressive Mode back on.
Thoughts?

Update 2:
with Quade's help, i found the re-tries were the issue, changing the setting to 2, with aggressive mode on was the charm...i now bow my head and walk away :)
nethead
Occasional Contributor
Occasional Contributor
 
Posts: 44
Joined: Sat Feb 12, 2011 3:03 pm

Registered Newsbin User since: 02/25/06


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 2 guests

cron