Automatic "Assemble Incompletes" for RARs

Make suggestions for new features.

Moderator: Quade

Automatic "Assemble Incompletes" for RARs

Postby Kaigi » Sun Jun 21, 2020 10:38 pm

Hello Quaid,

I think the most long-term frustrating thing I've found with Newsbin (possibly the only major one with the program!) is something I have written about before. But it is possible I wasn't clear in expressing my thinking, or else not clear in what I was referring to.

In many, many collections, there are RARs missing pieces (the red-stripe ones, though the black-stripe ones may also need it). These remain forever described by Newsbin as "DownlodING" in the Downloading Files list - i.e. they never are saved to disc as a RAR missing pieces and switch to "DownloadED".

So, if one is not in the same room, watching the program and manually marking each RAR with red-stripe-missing-pieces in it as "Assemble Incompletes", the ENTIRE file is ultimately dumped to the "Failed Files" area. This is true regardless of if there are more than enough PAR2 files to fix the incomplete segments.

Could Newsbin have a switch so that one can set it to "Assemble incomplete RAR files that have missing pieces if all dowload retries have been attempted"?

It would be SO nice to realize I can leave the program unattended without coming back to potentially find multiple actually-salvageable files in the "Failed Files" area, where I then have to re-download them, manually marking any with missing pieces with the "Assemble Incompletes" switch.

Thank you!


[I also need to seriously consider changing my news server to one with a higher completion rate!]
User avatar
Kaigi
Seasoned User
Seasoned User
 
Posts: 109
Joined: Tue Mar 04, 2003 11:23 am

Registered Newsbin User since: 10/24/97

Re: Automatic "Assemble Incompletes" for RARs

Postby Quade » Mon Jun 22, 2020 9:42 am

In 6.82, when a file gets to 10% failed, it automatically fails to the failed list. The problematic once are the once between 1-10%. In 6.82 you can enable "aggressive mode". With aggressive mode and 2 retries, just about everything should assemble and repair if it can or fail when it runs out of PAR files.

With headers, you used to have to wait to see if the file would complete. With NZB's or search, there's no point waiting. In the new code I'm working on, I'm experimenting with "one and done" meaning it tries all the servers for the parts and if all servers fail and file is as downloaded as it can be, it's just assembled right there with no delays. I suspect this would do what you're asking for.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44648
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Automatic "Assemble Incompletes" for RARs

Postby Kaigi » Tue Jun 23, 2020 6:30 pm

Yes, if it tries them all, then saves the files to the hard drive and downloads appropriate PAR2 files to fix any missing pieces, that will be great! Thanks!


For detail on how it plays out as is:
I have had "aggressive mode" enabled, but had it set to just 1 retry for missing pieces. I also always use NZBs. I had been manually selecting them before (to avoid downloading extraneous things like Samples), but with the other problem (new/odd obscured files) I have set Newsbin and left it on automatic download of double-clicked NZBs.

I've changed the setting to 2 retries, and I am still getting the same thing.

I just tested it with an NZB I suspected was incomplete. And, again, this is what happened:

The incompletes are never written to the hard drive (unless the pieces with the red stripes are manually selected and the "Assemble Incompletes" switch is used for them). I suspect from the way it acts that Newsbin appears to ignore their actual content. With that, it doesn't even TRY to download PAR2 files to fill in missing pieces, because without the incomplete pieces saved, it apparently doesn't realize there's enough overall to complete the file.

So, without manually selecting "Assemble Incompletes" for the file parts, Newsbin would just sit there, and after a time move the file from the Downloading Files list to the Failed Files list, having never downloaded any PAR2 files.
User avatar
Kaigi
Seasoned User
Seasoned User
 
Posts: 109
Joined: Tue Mar 04, 2003 11:23 am

Registered Newsbin User since: 10/24/97

Re: Automatic "Assemble Incompletes" for RARs

Postby jh20001 » Sun May 15, 2022 1:13 am

Kaigi wrote:Yes, if it tries them all, then saves the files to the hard drive and downloads appropriate PAR2 files to fix any missing pieces, that will be great! Thanks!


For detail on how it plays out as is:
I have had "aggressive mode" enabled, but had it set to just 1 retry for missing pieces. I also always use NZBs. I had been manually selecting them before (to avoid downloading extraneous things like Samples), but with the other problem (new/odd obscured files) I have set Newsbin and left it on automatic download of double-clicked NZBs.

I've changed the setting to 2 retries, and I am still getting the same thing.

I just tested it with an NZB I suspected was incomplete. And, again, this is what happened:

The incompletes are never written to the hard drive (unless the pieces with the red stripes are manually selected and the "Assemble Incompletes" switch is used for them). I suspect from the way it acts that Newsbin appears to ignore their actual content. With that, it doesn't even TRY to download PAR2 files to fill in missing pieces, because without the incomplete pieces saved, it apparently doesn't realize there's enough overall to complete the file.

So, without manually selecting "Assemble Incompletes" for the file parts, Newsbin would just sit there, and after a time move the file from the Downloading Files list to the Failed Files list, having never downloaded any PAR2 files.


I too find this to be a big pain in the arse. I started using Easynews because of it. When my subscription for search in Newsbin comes back up, I am going to let it go as I can't see myself using it anymore as it takes too much babysitting to get most things to download because of this.
jh20001
Occasional Contributor
Occasional Contributor
 
Posts: 17
Joined: Wed May 05, 2010 12:30 pm

Registered Newsbin User since: 11/30/06

Re: Automatic "Assemble Incompletes" for RARs

Postby Quade » Sun May 15, 2022 3:13 pm

6.90 doesn't wait. It's one and done. It tries all the servers and if the chunk isn't forthcoming the file is assembled.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44648
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Automatic "Assemble Incompletes" for RARs

Postby Kaigi » Sun May 15, 2022 7:12 pm

That isn't quite the issue. The issue is that - if there is a red (missing) block in any piece - the program won't automatically "Assemble Incompletes" and use what blocks ARE present in that piece to assemble the file. It simply ignores that entire RAR part. e.g. "___.part43.rar" would be ignored if ANY blocks are missing from it, even if 99% of that .rar are present. And if there are enough single pieces missing, despite there being enough PARs to compensate - IF what blocks are present are taken into account - the program rejects the entire download.

If the program cannot find a missing block in a segment, my thought was that it should "Assemble Incompletes" for that segment when it has all the blocks it CAN get, so that it uses what IS there. That has been my point/request. It still does not do that, and I have 6.90 Build 5524 installed.
User avatar
Kaigi
Seasoned User
Seasoned User
 
Posts: 109
Joined: Tue Mar 04, 2003 11:23 am

Registered Newsbin User since: 10/24/97

Re: Automatic "Assemble Incompletes" for RARs

Postby Quade » Sun May 15, 2022 11:46 pm

I just now took an NZB and forced one chunk of the first rar to fail and removed one chunk.

So I got a red and a black block (except 6.90 doesn't show blocks). It downloaded, repaired and unrared with no delay. I downloaded it a second time and paused before it was done, looked in the download folder and the file I damaged is sitting there assembled. Newsbin knows it's incomplete so the filename is upper-case.

[05/15 22:41:12] ERROR Downloader: Command Exception NEWS SERVER ERROR: Tweak SendCommand - Failed: 430 no such article Cmd: BODY <1321729915.63355.8@eu.news.astraweb.com12345> Tweak

Both of my servers gave this error.

I don't have any files hanging out in the download list.

If you want to email me another NZB that fails, I'm happy to try it. Your MUST feed the NZB's to the download list though.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44648
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to Feature Requests

Who is online

Users browsing this forum: No registered users and 1 guest