newsbin.com

http://downloads.newsbin.com/downloads/ ... 7-5317.exe

- Mostly auto-par fixes.
- Reduced memory usage because an optimization was being bypassed.
- Some pathing fixes.

It's all in the release notes.

Fixed the lockout poster issue I had.
Will test more tomorrow.

More:
OK it is still broken!
It is showing as Post Filtered in status but not removing them.

Hmm Ok. I thought it was still broken.

This was also happening in B6 - when downloading MP3 files, there are extra sub-folders being created and random files being put there (as opposed to being in the Album folder).

What's the best way to trigger this?

Quade wrote:What's the best way to trigger this?

I just double-clocked on the large rar file for this discography from the file list.
I can send you a nzb if it helps.
I have the filename settings to be "artist\album". (Prepend is not checked off.)

Sure getting the NZB would be great. ts at newsbin dot com.

After "upgrade" to 6.90B7 selected a group; replied with a short "Thank You" to a poster. Does not work. Reverted to 6.81 final. Same group same thank you, went right to it. No changes to settings from one to the other and back again. Minor thing and wanted to let you know....Cheers

Yeah posting is still broken. It's in the release notes. I haven't had a chance to look at it.

This was also happening in B6 - when downloading MP3 files, there are extra sub-folders being created and random files being put there (as opposed to being in the Album folder).

I was working with someone on IRC about an issue like that and the problem was unraring and MP3 mode. With MP3 mode disabled, you got the folder layout inside the RAR set. We were discussing whether MP3 mode should apply to RAR or PAR sets.

Quade wrote:

This was also happening in B6 - when downloading MP3 files, there are extra sub-folders being created and random files being put there (as opposed to being in the Album folder).

I was working with someone on IRC about an issue like that and the problem was unraring and MP3 mode. With MP3 mode disabled, you got the folder layout inside the RAR set. We were discussing whether MP3 mode should apply to RAR or PAR sets.

Gotcha. I trust whatever you decide will be best. I just hope its one or the other, and now this current hybrid that puts things in random places.

I'm noticing that when I use "Read post body", the texts now all say "RAW Data". And when downloaded and opened some have text. Some are saved without an extension. And some are blank.
Haven't read the release notes, but hoping this isn't part of that.
Mesa

I have the new beta (I used it yesterday) and this morning I just selected Download Latest Headers and the Critical Error Messages box popped up with a lot (341 to be exact) of this message:
[02/25 05:43:28] ERROR ReadFullDatabase: No filename or the NZB Data is missing

I've encountered that a couple times (after the beta was out, I didn't see it before) but I'm not sure why yet. It seems header import partially fails for some headers. So the download data doesn't get written to disk.

I don't have a solution yet. As a work around you could try right clicking the group "Post Storage Use Download Age" then set the "download age" in the options to 1 day, then download headers. This will make it re-download the last day's worth of headers and re-import. This would be my #1 issue but I'm in the midst of re-doing how the download workers work so, at the moment it's #2 (because without the workers, I can't trouble-shoot this).

Is there a way to add the Download Path column to the files list? This seemed to disappear after I upgraded.

I didn't think it was needed since you can right click "Open Folder" on any of the files.

There is only one group that I have disabled unrar. It does disable unrar in the folder created for the a.b.group as always. What I am saying with this build, it takes those rar files and unrars them into a separate folder, whereas previously if you did not disable unrar, they would get extracted to the same folder a.b.group. It appears disabling unrar doesn't disable it, just places unrared files in another separate folder.

Pleased to report this issue is now fixed with B7. No secondary folder being created. It now unrars to the same folder a.b.group even though the option is set to disable unrar. Disabling unrar fails to work.

Regards,

I'm getting issues with file pathing and pause status changes being lost after restarting NBP.

1) Add files to download list
2) In the download list, change the 'Assign a New path' and 'Assign New Unrar Path'
3) Select All in download list, pause
4) Close NBP
5) Open NBP
6) The download path have reverted back to the initial download folder
7) Select All in download list, resume
Close NBP
9) Open NBP
10) Files in download list are still in Paused state

How many files in the list? That status gets saved on exit.

Quade wrote:I didn't think it was needed since you can right click "Open Folder" on any of the files.

OK. Not a big deal. I just use assign a new unrar path a lot so my download folder varies. No worries.

Quade wrote:How many files in the list? That status gets saved on exit.

Around 1k

Is it possible to have Newsbin monitor the space in the UnRAR folder? Since I usually have that set to a removable drive, it runs out of space first and Newsbin keeps running on until the download drive is full. Even just adding the status to the bottom would be helpful.

It'll automatically stall the unrar to the full unrar drive but that doesn't stop it from downloading additional files to the download drive. The unrar's will stack up behind the stalled unrar.

What you seem to be asking for is for any drive being low on space to trigger Newsbin to stop downloading?

Quade wrote:What you seem to be asking for is for any drive being low on space to trigger Newsbin to stop downloading?

Yeah, that would be good.

I know you are truncating path names again because some program is incompatible with long file names, but it'd be nice if you could add a switch to allow long names because I'm sure not everyone uses that program, but would like to see full path names like "Black.Lightning.S04E04.The.Book.of.Reconstruction.Chapter.Four.A.Light.in.the.Darkness.CW.WEB-DL.AAC2.0.H.264-TEPES." I use the path name to rename the obfuscated file name, so it would be nice to have the full name.

Downloads have been stop-and-go on the 6.90 Betas. I keep having to go back to a non 6.9 version to get things back to normal.

Basically, grab a large post to download (2+ GB for example)... it starts to download... then stops... in logging I see:

ALWAYS Feeder: Waiting for last both to finish....

This repeats many times... eventually it gets going again... then repeat all of the above. What normally should take just a couple of minute to grab and unrar, now could take an hour...

Thoughts?

ALWAYS Feeder: Waiting for last both to finish....

This is a message related to header importing. Header import is multi-threaded now but to keep it from running away, it'll only stack 2 jobs at the worker before it'll stall and wait for the worker to finish. What this message suggests is that header import has stalled out to the disk with the data folder. I've never actually seen this message before. It might indicate you're out of disk space or the header DB is corrupted for this groups. It might indicate that the disk is busy too. It's unrelated to downloading though.

This repeats many times... eventually it gets going again... then repeat all of the above. What normally should take just a couple of minute to grab and unrar, now could take an hour...

It suggests to me that writing to a disk has stalled out. You have symptoms that both writing to the header DB and writing for unrar and download are stalling out. The question is why. It could be that multithreaded header feeding has simply run you out of disk performance.

If you look down at the bottom of Newsbin where it says "Cache:" do you see a number in parens? If yes, what's that number?

I have been testing the beta and found its absolute not looking at the password file
It constant gives also download errors while it actually really did download the package complete and did fix it also pretty good with mulitpar
But whatever i do it fails to read the prefilled .txt
Further want to say it works like a charm in many ways seems much quicker than before in usage
Ofcourse it has not influence on the download speeds
But seems to work fine for me in general besides the weird failed errors and the password issue

I noticed with the harddrive storage it indeed stalls but its just because the disk is too slow for the speed my downloads get to the disk especially if its busy parring the previous task or unrar them.
For me that is normal and not an issue, if i let all the stuff i download do on my nvme ssd it would not stall anymore, but it will wear out my ssd's
So i choose to get it done a bit slower and see the stall warnings, it simply means newsbin want a faster disk
Which i do not have because it is also doing other tasks and the hdd are ofcourse healthy huge
I have 3 disks for storage a 6 TB internal one a HGST ale640 and a 8TB usb 3.0 a 6 TB usb c and a 4 TB usb 3.0 one, in total 24 TB which are for backup, news items , and plain massive games install.
The internal takes care of the incoming news material and does unpack the files there as well.
Overall the disk performs around 200 MB/ 175 MB writes so when it has todo several tasks ate once its really no wonder that it stalls from time to time.
I could speed it up by pumping the files it unpacks to one of the usb drives but in general that would be slower than doing it on the internal
The rarpasswords.txt belongs in the data folder right, just to be sure its not messed up somehow.

The reason is simple the external drives take more overhead to handle when i do that, so i keep using the same drive as it will do the job similar fast.

But whatever i do it fails to read the prefilled .txt

I'll put this on my list to check again. I did test this for B7 before I put it up. Passwords from the NZB Filenames won't show up the password list. They're stored internally and applied directly to the downloads. You can manually assign the password to the download from the download list.

Quade wrote:
This is a message related to header importing. Header import is multi-threaded now but to keep it from running away, it'll only stack 2 jobs at the worker before it'll stall and wait for the worker to finish. What this message suggests is that header import has stalled out to the disk with the data folder. I've never actually seen this message before. It might indicate you're out of disk space or the header DB is corrupted for this groups. It might indicate that the disk is busy too. It's unrelated to downloading though.

Interesting, this is on an SSD.. I'm wondering if there may be some QEMU changes that are affecting it. Any other useful info on this new handling that I should read so that I can do some further troubleshooting?

Quade wrote:

This repeats many times... eventually it gets going again... then repeat all of the above. What normally should take just a couple of minute to grab and unrar, now could take an hour...

It suggests to me that writing to a disk has stalled out. You have symptoms that both writing to the header DB and writing for unrar and download are stalling out. The question is why. It could be that multithreaded header feeding has simply run you out of disk performance.

If you look down at the bottom of Newsbin where it says "Cache:" do you see a number in parens? If yes, what's that number?

FWIW, the 'disk' (SMB mount) for the download/unrar has for years been different drive (a multi-drive array) from the SSD that holds the database and import.
I'll reinstall and run more tests soon and check the Cache values for you.

and check the Cache values for you

The point of this is some people get the header import SO behind there's no real way to recover so, purging and starting over is sometimes the best method. The number I want just tells me how much header import backlog you have.

Quade wrote:If you look down at the bottom of Newsbin where it says "Cache:" do you see a number in parens? If yes, what's that number?

Typically Cache shows 400/400 (###) with ### being the import folder file count at the time...

I'm going to tweak some QEMU disk settings to see if I get any performance changes...

Quade wrote:

and check the Cache values for you

The point of this is some people get the header import SO behind there's no real way to recover so, purging and starting over is sometimes the best method. The number I want just tells me how much header import backlog you have.

Ah, I watch the import folder looking for any stuck files...

I ran procmon yesterday and noticed a lot of work on a -wal file. The file was 13GB in size, so I moved it out of the way for now.

So reran Beta 7... this time I had a good backlog of files in import folder... it processed it fairly quickly.. but it also add a ton of weird files/posts into my queue (had it paused) based upon one of my watch list... the strange thing is that these posts do not show up with I pull up the Folder and apply the filter that is set for the watch list.

Any changes in this area? I can't quickly rationalize why it is matching these strange posts...

Yes, changes in this area. Is there any pattern to it? I use watch lists set and they SEEM to be pulling the correct files.

ack9 wrote: but it also add a ton of weird files/posts into my queue (had it paused) based upon one of my watch list...

This happened to me as well when I ran the latest beta for the first time yesterday. And it continued today. I left it running, until everything was done. The majority of files had already been downloaded at some point.

Here's another weird thing: for example, double clicking on a file from watch list will add it to the download list, but with a wrong size. Something like 793kb, when it should be 1.2GB. It then gets removed and it'll neither be marked downloaded or old. Sometimes it shows in the failed list (when it actually had been downloaded before), sometimes it doesn't.
I also have some similar weird issues with certain NZBs, or how they are handled. Can't put my finger on it yet, or can't rule out bad NZBs.

Please, Please restore the file path to the Filename tab. I used that all the time. It tells me when I have things in the wrong place. I keep 1,000's of files there. Thank you. If not, I will probably revert to an earlier version if that is possible.

I just tried move file and rename file using a right click on the file and it neither renamed or moved the file. What am I missing. I am operating in the File List tab. Thanks

It doesn't seem to work - rename I mean. I could probably re-add a path column.

Thanks.

Ok path is back. It has it's own sortable column and can be moved around.

Quade wrote:Yes, changes in this area. Is there any pattern to it? I use watch lists set and they SEEM to be pulling the correct files.

Not a clue. I load up the group, select the filter, and the strangest items are shown.

For example: "[244/253] - "u5JB0hyCIGGHs9kIhkbakHQ" yEnc 10000000"

Most of my filters are on subject, and in this particular case, the filter is mostly real words with a little .* or . usage.

Getting this error message an awful lot, too.

Code: Select all

[03/14 19:59:45]  ERROR File Assembly Chunk Size 2 doesn't match size of size of current data buffer


Some posts lie about how much data is in the chunk. The Yenc spec says the size of the chunk is to be listed in the post, in the yEnc headers. If the size of the yEnc header and the size of data Newsbin actually downloaded doesn't match you'll see this.

When you download, repair should really be the exception. If you seem to be repairing every download that shows this then that suggests there's a problem. I've been downloading 100's of gigs a day lately and I'm not seeing this very often and I seldom repair.

Quade wrote:When you download, repair should really be the exception. If you seem to be repairing every download that shows this then that suggests there's a problem. I've been downloading 100's of gigs a day lately and I'm not seeing this very often and I seldom repair.

Seems to be a problem with the posts I download then. Most of them are pretty old. Thanks!

Old files weren't yEnc encode but I thought I worked around that.


I tested the older style password list way of unraring encyrypted rars and it worked for me so, at least in what will be B10, I can't reproduce it. Keep in mind you can only add passwords from the Options menu. The password window in the download list is for selecting or typing in a specific password to the download, it won't add them to the list.

Another thing I noticed: "create NZB from post" doesn't work for my like it should, or at least like it has worked before. The purpose is in this case, to create a smaller NZB from another.
The NZB used to be created from what you had selected - either in a group or even a different NZB. Now it does not work from an opened NZB - the whole list will be put into the new NZB. It does work from an opened watchlist, or a group however.

*edit

One more weird thing regarding NZBs: at first I thought certain files were just faulty (downloaded via NZBKing), when loaded into NB there's stuff in there, that's impossible to be in there (posted after the NZB was created, for example!) or just files from groups that I download from. Trying to download will just remove the file from the download list, without actually downloading or giving an error. But then I realized, when double clicked, the NZB loads fine.
Could there be something wrong with some database in my case? Thanks.

How do I turn off auto-unrar/unzip? I just want to download the files as is and I cannot seem to find this option in Settings. It did not do this before I installed the beta so I am unsure where the option is located.

Thanks.

Let me verify it still works. You out to be able to disable unrar in the autopar options.

Mindflayer wrote:How do I turn off auto-unrar/unzip?

Go into Options / AutoPAR Options -> turn off automatically unRAR complete files.

Let me verify it still works. You out to be able to disable unrar in the autopar options.

Reported that does not work in B7 earlier Quade.