Newsbin Beta 6.90RC1 - build 5424

This is the place to help test and discuss Version 6 Beta releases.

Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Thu Jul 29, 2021 11:21 am

http://downloads.newsbin.com/downloads/ ... 1-5424.exe

Getting close to finished once more. I can only think of one more thing it needs to release, which is hook up "hide incompletes".

Code: Select all

- Sonarr: Change Sonarr interface to first try to set the NZB Specific download folder, then use the main download folder if it doesnt; exist.

- NZB: NZB Autoload will not default to the NZB specific download folder but will fall back to the main NZB folder if there’s no NZB Specific folder set.
- NZB: Some modern NZB’s lie about how many chunks the files has.  It looks like I need to ignore maxindex in the subject. They still work but the download looks ugly.
    A unnamed user kindly provided a class that demonstrated the fix.

- Display: Calendar control for load range dialog now goes as far back in time as the “storage age” setting.
- Display: After doing DB compaction there’s no indication of space freed.  Should log it.
- Display: Add pause state display for children in download list.  User request.

- Display: When you use Move in the files list, the display doesn’t reflect the new path. Can also make files vanish from the files list on restart (not from disk)
- Display: Report that Newsbin’s icon doesn’t update when downloads complete.
- The NewsBin Pro icon in the Windows taskbar is not resetting when a
    task it down. It still has the little play arrow like when its
    downloaded something .Even if its not doing anything.
- Display: GUI stalling still happens from time to time. Particularly in release build.

- AutoPAR: Fix for problem where a RAR set contains a file with a leading space AND you're downloading to the same folder you intend to unrar to.  The leading space prevented copy to final destination.
- AutoPAR: Behavior change when unraring to download folder.  Before, Newsbin would rename the duplicate files out of the way when copying to the final folder. Now Newsbin detects the conflict and copies to a "Decode Copy (N)" folder instead so, the user can choose which copy he wants.
- AutoPAR: Can only download a set once per session
- AutoPAR: Report that it can't be disabled. In fact it was true. It still counts blocks but won't repair or unrar when disabled.

- Download: Changing server configuration should ungulag all connections.  Failed connections get sent to jail for 5 minutes.  In cases where the user changes the password to the correct one, we really want to release the connections from jail to try again.



More stuff in the release notes but these are the high spots. I've attempted to make the remote NZB link NZB Filename finder more robust. It's important for Newsbin to know the Sonarr NZB Filename for pathing to work properly.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby dane » Sat Jul 31, 2021 8:36 pm

Hi Quade - thanks for the Release Candidate. I queued a file to download from Sonarr - the decoded, unrared file ended up in the correct folder for Sonarr to take it, but the rar files downloaded initially to a folder named for the group it was downloaded from. This was left as an orphan folder after it was unrared to the correct nzb-designated folder from which Sonarr took it. Not in and of itself a big hassle, but if you're downloading a number of files, cleanup of the multiple orphans can be tedious.

I appreciate the line by line progress bars so it's possible to see where there are hiccups when they occur. I was also hoping for line by line status ("downloading" or "paused") as the status was displayed in earlier versions of newsbin.

Looking forward to the release version
DaDane
dane
Seasoned User
Seasoned User
 
Posts: 193
Joined: Sat Mar 26, 2005 7:47 pm

Registered Newsbin User since: 03/25/05

Re: Newsbin Beta 6.90RC1 - build 5424

Postby tongueboy » Sun Aug 01, 2021 3:07 am

YES, Thank you.
Autopar behaves much better now when disabled.
Thank you for all your hard work.
tongueboy
Active Participant
Active Participant
 
Posts: 60
Joined: Wed Aug 01, 2012 2:07 am

Registered Newsbin User since: 07/27/12

Re: Newsbin Beta 6.90RC1 - build 5424

Postby aneurysm » Sun Aug 01, 2021 7:01 am

I also want to thank you for the autopar fix. Disabling it works fine again.
One question I do have though, par files don't automatically go to the wish list anymore. And when manually selecting them and then assigning then to wish list doesn't do anything either. I know you said that with the new posting method it was needed for the pars to auto download because of the new obfuscation posting method.
But a lot of stuff is still being posted normally. There it wouldn't make a lot of sense to download all pars automatically if not needed.
Would it be possible that when a nzb is loaded, newsbin checks the method posted and therefore sees if pars need to be downloaded automatically yes or no ?
aneurysm
Occasional Contributor
Occasional Contributor
 
Posts: 10
Joined: Sun Feb 09, 2014 10:44 am

Re: Newsbin Beta 6.90RC1 - build 5424

Postby kirm » Sun Aug 01, 2021 7:51 pm

Not something I have done in some time but tried to Reply to Group to thank a Poster. Is this not connected or working?
kirm
Seasoned User
Seasoned User
 
Posts: 249
Joined: Sun Aug 17, 2003 4:17 pm

Registered Newsbin User since: 05/15/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Sun Aug 01, 2021 8:58 pm

I'm able to post to Tweaknews but others have report not being able to post to Easynews and in my case Usenetserver seems to have disabled posting. At least I couldn't figure out how to turn it on.

So I'm aware of an issue but I'm not clear if it's Newsbin or the server. You could look for errors in the logging tab.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby kirm » Mon Aug 02, 2021 4:47 pm

I'm aware of an issue but I'm not clear if it's Newsbin or the server. You could look for errors in the logging tab


Use Newshosting. As mentioned rarely use this so not positive when this stopped working. The Reply to Group window opens up, fill in any pertinent info, then hit Post.
Window closes, check for updates, nothing showing up for new items. Nothing displayed or added in logging tab as per error. Checked a few times figuring a delay with servers, still nothing.

Sent a message off to Newshosting, to see if there is any reason from their end. Will keep you updated.

Thanks.
kirm
Seasoned User
Seasoned User
 
Posts: 249
Joined: Sun Aug 17, 2003 4:17 pm

Registered Newsbin User since: 05/15/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Emaubin » Tue Aug 03, 2021 6:47 am

Hello Quade,

There seems to be something broken concerning header management. This may be a long read, but it's because I want to explain the issue as clear and detailed as I can.

For the past 10 years I have been using Newsbin on a nearly daily basis where I quickly view what's new and don't want to keep any history:

- I click "download the latest group updates" from the menu bar (or "download latest headers" after a right click on a specific group or set of groups).
- I right click on a group (or set of groups), then click on "show all posts"
- after browsing, I right click on a post and click "delete all posts" (or right click group - "post storage" - "delete stored posts")

I find back each group as a folder in spool_v6 where range.db3 and storage.db3 reside, growing and shrinking and being 12K and 44K when posts are deleted.

However, now with this RC1 (and previous beta) these folders get deleted from spool_v6 after the "delete all posts". As a result, a "download latest headers" always retrieves the same mass of headers from months and years back, even with the download age set to 1. It looks as if there is a clean up of spool_v6 now, but without keeping track of the pointers in range.db3 in another place.

Then I think that maybe I need to change my ways and should not delete all headers every session. So I tried alternatives, but they do not work:
- "post storage" - "purge to display age" (display age=2), I get the error "ERROR Thread Died:" and I have to close the "Purging to Display Age" window manually. Nothing has happened.
- "post storage" - "use download age" (download age=1), nothing happens after clicking ok on the warning, also not after exiting Newsbin and restarting.

After a reinstall of v6.82 (over the RC1), everything works fine again as before. Now, I don't mind having to change my habit as long as there is a way to dismiss any history and prevent the newsbin folder from growing out of proportion, especially as more and more posts are not intended to be handled by a usenet browser.

I hope you are able to provide a solution for this but in any case, thank you for reading,

Kind regards,
Emau
Emaubin
n00b
n00b
 
Posts: 1
Joined: Fri Jun 15, 2012 6:42 am

Registered Newsbin User since: 02/14/11

Re: Newsbin Beta 6.90RC1 - build 5424

Postby jcthorne » Sat Aug 07, 2021 9:00 am

This version is still not handling many NZB files correctly. Instead of saving files in the file names listed in the NZB and shown in the download list, its saving files with names of abc.xyz.#############.ext where ############ seems to be a random string of characters. The ext is correct for the file. The result is a list of rar files that do not appear as sets and do not unrar. For NZBs that contain a single file to download the files can be renamed using the ext given and unrar manually but with nzbs that contain multiple files like season sets, the data cannot be reassembled even manually.

This has been going on for many versions. I have an example nzb or two I can send you if you like. I get them every day.

Just FYI, most of these are fed to newsbin by Sonarr but I can manually download the nzb file from an indexer and feed it to newsbin and the behavior is the same.
jcthorne
Active Participant
Active Participant
 
Posts: 68
Joined: Sat Nov 01, 2003 12:17 pm

Registered Newsbin User since: 11/01/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Sat Aug 07, 2021 10:46 am

I find back each group as a folder in spool_v6 where range.db3 and storage.db3 reside, growing and shrinking and being 12K and 44K when posts are deleted.


"Delete all" should be wiping the whole spool_v6 folder for the group so I'll check that. This should trigger it to use "download age" again. I'd suggest not doing this but instead set a short "Storage Age" so Newsbin automatically deletes posts older than "Storage Age".

Simply deleting posts doesn't shrink the database. The space is freed internally and re-used when new headers are downloaded.

I'll check the rest thanks.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Sat Aug 07, 2021 10:49 am

This version is still not handling many NZB files correctly. Instead of saving files in the file names listed in the NZB and shown in the download list, its saving files with names of abc.xyz.#############.ext where ############ seems to be a random string of characters. The ext is correct for the file. The result is a list of rar files that do not appear as sets and do not unrar. For NZBs that contain a single file to download the files can be renamed using the ext given and unrar manually but with nzbs that contain multiple files like season sets, the data cannot be reassembled even manually.


Autopar will rename them to a regular set of filenames which Autopar can then unrar.

If you want to email some of these nzb's to ts@newsbin.com I can try them. I'm going to hazard a guess that you disabled autopar or something like that.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby jcthorne » Sat Aug 07, 2021 10:56 am

Sent you an nzb file example of the problem. AutoPAR is enabled on my newsbin.
jcthorne
Active Participant
Active Participant
 
Posts: 68
Joined: Sat Nov 01, 2003 12:17 pm

Registered Newsbin User since: 11/01/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Sat Aug 07, 2021 12:11 pm

Thanks for the NZB.

No PAR files. That explains why there's no rename.

This is a different kind of obscured where a valid filename appears in the NZB Subject but not in the actual download filename. With no PAR files there's currently no rename. I can probably add this format. I just have to make sure it doesn't break anything else. A more typical obscured NZB, the actual filenames come from the PAR files.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby jcthorne » Sat Aug 07, 2021 12:16 pm

Thanks. Would be very helpful if you could add handling of nZbs in this format as I get a number of them every day.
jcthorne
Active Participant
Active Participant
 
Posts: 68
Joined: Sat Nov 01, 2003 12:17 pm

Registered Newsbin User since: 11/01/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Moondawgie » Tue Aug 10, 2021 11:44 pm

Quade wrote:I'm able to post to Tweaknews but others have report not being able to post to Easynews and in my case Usenetserver seems to have disabled posting. At least I couldn't figure out how to turn it on.

So I'm aware of an issue but I'm not clear if it's Newsbin or the server. You could look for errors in the logging tab.


Does the following information apply to the problem of some not being able to post to usenet via Easynews?

Unable to Post Via Easynews - Why can't I post through Easynews?
Last edited by Moondawgie on Wed Aug 11, 2021 11:14 am, edited 1 time in total.
MS Windows 10 Pro 64-Bit I- ASUS P8Z77-V LE; Intel i7-3770K 3.5 GHz; 16.0GB DDR3; 500 Mbps cable internet.
User avatar
Moondawgie
Seasoned User
Seasoned User
 
Posts: 418
Joined: Mon Aug 13, 2001 5:17 pm

Registered Newsbin User since: 05/17/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Wed Aug 11, 2021 8:11 am

I was wondering about that too.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby mjpierce » Wed Aug 11, 2021 11:22 am

I cannot delete "shift-del" posts from a Watch. The post will be removed from the display but closing the watch window and opening it again shows the post still there.
This is true for one post or many. This is keeping compacting from working.
mjpierce
Seasoned User
Seasoned User
 
Posts: 116
Joined: Tue Jun 05, 2007 5:29 pm

Registered Newsbin User since: 10/29/06

Re: Newsbin Beta 6.90RC1 - build 5424

Postby mjpierce » Wed Aug 11, 2021 1:08 pm

Also with watch topics, concerning "Look in Topic". If I have a topic, say "Adult", that is alphabetically before "All Groups" selecting "All Groups" will revert to "Adult" the next time I open the watch topic properties.
mjpierce
Seasoned User
Seasoned User
 
Posts: 116
Joined: Tue Jun 05, 2007 5:29 pm

Registered Newsbin User since: 10/29/06

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Thu Aug 12, 2021 12:38 am

Thanks. I'll check them both out.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby ccarlin » Thu Aug 12, 2021 4:48 pm

FYI - With RC1 I'm able to post (and then download) text and binaries via newsdemon w/o issues. Like folks have said of other news providers - one needs to be granted posting permissions ....

Thanks for the great work. Chris.
ccarlin
Occasional Contributor
Occasional Contributor
 
Posts: 25
Joined: Tue Aug 19, 2003 11:21 pm

Registered Newsbin User since: 04/03/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby JayPea » Sat Aug 14, 2021 5:15 am

ccarlin wrote:FYI - With RC1 I'm able to post (and then download) text <snip>


Weird. I am not able to post using the Reply to Group function using Newshosting. I've checked my account and posting is flagged as active but when I try to reply NOTHING happens. When will this feature be working again please, Quade? It's a pretty fundamental feature to NNTP clients, well to those of us that aren't just leeching, of course.

I like to be curtious and thank people for their time, participate in discussions, etc and currently that's impossible. Dropping back to an earlier version is no longer an option due to modern nzb compatibility.

Many thanks for your time & support.

Cheers.
JayPea
Active Participant
Active Participant
 
Posts: 92
Joined: Tue Dec 12, 2006 2:48 pm

Registered Newsbin User since: 12/11/06

Re: Newsbin Beta 6.90RC1 - build 5424

Postby kirm » Sat Aug 14, 2021 1:52 pm

Weird. I am not able to post using the Reply to Group function using Newshosting. I've checked my account and posting is flagged as active but when I try to reply NOTHING happens. When will this feature be working again please, Quade?


Exact same issue here with Newshosting. It also shows flagged as active on the account page. Tried sending them an email, never get past the captcha to contact support. Sent message through their Facebook page, no response.

As previously mentioned, rarely ever post a response, possibly couple years back. It always worked previously, no idea what version of Newsbin was using at that time. Simply a no-go now with Newsbin not displaying any error.

Why this works for some and not others, way beyond me.

Regards,
kirm
Seasoned User
Seasoned User
 
Posts: 249
Joined: Sun Aug 17, 2003 4:17 pm

Registered Newsbin User since: 05/15/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby ccarlin » Sun Aug 15, 2021 1:47 am

OK. I did some posting tests to 3 different news providers today, and all three worked fine. So, I highly doubt Newsbin RC1 is having a posting problem. Here's some data:

1. I tested/posted to blocknews.net, astraweb.com and newsdemon.com.
2. I used both "post to group" and "reply to group" and both worked/posted as expected
3. I tested with both text and binary posts

And then some observations;
4. I had [serious] problems posting to some "offbeat" groups like 0.test, but saw immediate results with more common groups like a.b.test2.
5. Blocknews and Astraweb allow posting, w/o any special requests/permissions to do so. Newsdemon requires permissions.
6. One can buy a 5G block from Blocknews for $1.99 if you want to test or post. Newsbin has a easy/simple mechanism for selecting which news provider to use when posting. So, one can use one provider for downloads, and a 2nd (like Blocknews) for uploads.
7. I've found Astraweb to be excellent (both performance and fill %'s, even for the most oddball requests), with Blocknews.net a very close 2nd (it's a reasonably priced block/fill provider). I can't recommend Newsdemon any more - as they changed their backbone Q4/2020 and too much content is missing.
8. While all the news providers state what their retention rate is, and what newsgroups they track, I don't believe any of them can meaningfully qualify to an end-user what % of postings they really retain. The less an individual post is requested, the higher the probability that it has not been captured/saved for later consumption. So, vendors like Astraweb and Blocknews will be able to fill most any request (within their date retention claim), while the 2nd tier providers won't be able to fulfill a surprising number of requests, even for "recent" content. For what it's worth I've found Astraweb is slightly better than Blocknews, though Blocknews would be just fine as one's only provider (it just has to retry on more of their servers and use more .par files when grabbing binaries.

Regards, Chris.
ccarlin
Occasional Contributor
Occasional Contributor
 
Posts: 25
Joined: Tue Aug 19, 2003 11:21 pm

Registered Newsbin User since: 04/03/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby gkar » Sun Aug 15, 2021 4:25 am

am seeing this error

[08/15 07:44:29] ERROR Downloader: Unknown Download Exception

NBPRO stalls at this point, I need to remove the file in list, restart NB then it carries on.

Happens with all my servers, i can download the files using another downloader. I've tried a full uninstall and reinstall and turning off security software. Any ideas?

Running windows 10 19041.1165
User avatar
gkar
Seasoned User
Seasoned User
 
Posts: 306
Joined: Tue Jun 26, 2012 1:40 pm

Registered Newsbin User since: 06/27/12

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Sun Aug 15, 2021 10:41 am

How are you downloading? Search, Headers or NZB? I'm not seeing anything like that but it could be download specific.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Stan » Sun Aug 15, 2021 2:34 pm

ccarlin wrote:OK. I did some posting tests to 3 different news providers today, and all three worked fine. So, I highly doubt Newsbin RC1 is having a posting problem. Here's some data:

1. I tested/posted to blocknews.net, astraweb.com and newsdemon.com.
2. I used both "post to group" and "reply to group" and both worked/posted as expected
3. I tested with both text and binary posts

And then some observations;
4. I had [serious] problems posting to some "offbeat" groups like 0.test, but saw immediate results with more common groups like a.b.test2.
5. Blocknews and Astraweb allow posting, w/o any special requests/permissions to do so. Newsdemon requires permissions.


it looks like everyone has different results which will make it so it may take awhile to get fixed. I know for a fact, for me, it's something to do with the latest newsbin releases. I know cause I go back to 6.82 and can post to whatever group I want and use any of the 3 news services I have that I want. So for me the ONLY fix is if/when I want to post is to go back to 6.82 then go back to RC1 if I want to download nzb's. Maybe it is certain groups then but like I said 6.82 works with any group I try. Also maybe it's certain newsservers although again 6.82 I can use any of the 3 I choose. In all my testing the only difference is the newsbin version. 6.82 does post and RC1 did not also beta 11 or 12 did not post either for me, nor did any earlier betas.
Stan
Occasional Contributor
Occasional Contributor
 
Posts: 38
Joined: Wed Jun 21, 2017 12:18 am

Registered Newsbin User since: 03/10/17

Re: Newsbin Beta 6.90RC1 - build 5424

Postby ccarlin » Sun Aug 15, 2021 4:23 pm

OK - yet more testing appears to confirm a problem. I'm able to successfully post ONLY if there is an attachment to the post (all my tests yesterday had attachments - binary and then text). I've confirmed this with 2 news vendors across 2 different news groups using both "post to" and "reply to". So, unless there is something weird with my systems/net connection, the bottom line appears to be if there is no attachment to the post then it doesn't work. Re: the 2 different news groups - one was a binary group (a.b.test2) and the other wasn't (alt.alt.test).

If the lack of an attachment turns out to be a problem there is a free/open server that allows text posting only that can be used by anyone to determine if there are 2nd order posting issues. See the site www.eternal-september.org for info.

Chris.
ccarlin
Occasional Contributor
Occasional Contributor
 
Posts: 25
Joined: Tue Aug 19, 2003 11:21 pm

Registered Newsbin User since: 04/03/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Sun Aug 15, 2021 4:58 pm

That's a good data point. I'll try it here.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby gkar » Mon Aug 16, 2021 3:03 am

Quade wrote:How are you downloading? Search, Headers or NZB? I'm not seeing anything like that but it could be download specific.


Headers

to be clear its only certain files.
User avatar
gkar
Seasoned User
Seasoned User
 
Posts: 306
Joined: Tue Jun 26, 2012 1:40 pm

Registered Newsbin User since: 06/27/12

Re: Newsbin Beta 6.90RC1 - build 5424

Postby stavros » Mon Aug 16, 2021 6:42 am

Hi,
Getting lots of:
[08/16 10:12:05] ERROR ReadFullDatabase: No filename or the NZB Data is missing
in the log.
Initially, no real clue as to which import file it is referring to, these appear in the middle of the import processing
of about 500ish import files.
On further investigation, some of these happen between 2 references to a group, so I assume that that is one of the groups affected.
There are lots more.
What does this message mean? what causes it and do I have to worry about data loss and what is happening to my spool?
regards
Stavros

*EDIT*
I've found an import file that generates 30 of these messages if that is of any use to you? it is about 245 KB as a text file.
I can also extract data from my storage.db3 and storagedata.db3 files for the timeframe surrounding the import
process - not sure how big that'll be, but not too big I hope (might need a little help to get the right date & times).
stavros
Seasoned User
Seasoned User
 
Posts: 118
Joined: Wed Dec 24, 2003 2:07 am

Registered Newsbin User since: 12/24/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Mon Aug 16, 2021 12:05 pm

"StorageData" isn't used anymore. What I suspect you're seeing is that Newsbin thinks there's download data but it's not in "Storage" and RC1 doesn't reference "StorageData" to look for this data.

RC2 now references StorageData if it can't find the information in Storage. I didn't realize so many people still had the old-school "StorageData" files. I think 6.82 is where I deprecated "StorageData".
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Mon Aug 16, 2021 12:07 pm

to be clear its only certain files.


If you want to PM me the subjects of some of these files and the group. I can try it here.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby stavros » Mon Aug 16, 2021 1:00 pm

Quade wrote: I think 6.82 is where I deprecated "StorageData".


I was running 6.82 without problems before jumping to trying 6.90RC1.

Shouldn't 6.82 have migrated the data from StorageData to Storage at some point -
or was that in an earlier version of newsbin?

Is it worth me trying the version of newsbin that did the auto migrate (if there was one),
before trying the manual sqlite3 SQL insert option to transfer data to tidy things up?

If I just copy/insert the data from StorageData.FileData_V4 to Storage.FileData_V4, what about the other tables
in Storage? Will newsbin take care of any other relationship processing that may be needed to maintain
integrity?
stavros
Seasoned User
Seasoned User
 
Posts: 118
Joined: Wed Dec 24, 2003 2:07 am

Registered Newsbin User since: 12/24/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Mon Aug 16, 2021 3:49 pm

Shouldn't 6.82 have migrated the data from StorageData to Storage at some point -
or was that in an earlier version of newsbin?


Migration would be a slow process. Instead doing what I did for RC2 is better. Just making use of the old files while not updating them anymore. All the new data follows the new format.

No version migrated. What changed with 6.90 is I mistakenly thought most people were using the new format already.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby stavros » Mon Aug 16, 2021 4:11 pm

Quade wrote:
No version migrated. What changed with 6.90 is I mistakenly thought most people were using the new format already.


Sorry, but I've got headers going back to 2003 in newsbin, and a lot older than that in other, now defunct, readers.

Thanks for supporting the old stuff :-)
stavros
Seasoned User
Seasoned User
 
Posts: 118
Joined: Wed Dec 24, 2003 2:07 am

Registered Newsbin User since: 12/24/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby stavros » Tue Aug 17, 2021 6:54 pm

Hi Quade,

I think there's a problem with import/header filters.
Since I upgraded, my filters, that worked in 6.82 to stop stuff getting in to the DB, are now letting stuff through - and I haven't changed anything.
I'm not sure if it is some/all of the Reject If filters or the minimum size filter - but stuff is getting through that, I believe, should not.

The time taken to import has gone from less than 3 mins to over 35 mins - and that is for just 6 hours elapsed between header d/ls.

In most cases, I apply a header filter at the GOG level, and let it cascade down to all of the groups within the GOG.
There are a handful of cases where groups may also have a header filter applied as well as a GOG filter.

What info do you need to help in checking things out?

*edit*
looking at the beta change history - beta 7 suffered from: "Default filter file isn't getting installed "
could this be involved?

regards
Stavros.
stavros
Seasoned User
Seasoned User
 
Posts: 118
Joined: Wed Dec 24, 2003 2:07 am

Registered Newsbin User since: 12/24/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Wed Aug 18, 2021 11:19 am

looking at the beta change history - beta 7 suffered from: "Default filter file isn't getting installed "


This was only an issue with new installs.

I might load up one of these slow groups, clear the other filters, then try applying the header download filter and see of it still rejects. My first guess is they changed posting so your existing filters don't work anymore.

I'll verify that that it's all working. GOG level filter assignment should work.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby stavros » Wed Aug 18, 2021 2:36 pm

Hi Quade,

thanks for looking into this.

After my previous post, I re-installed 6.82, then d/l headers.
I d/l 229 .txt header files, totaling 4.28 GB of data.
6.82 imported this data in about 2 mins.

I have looked at several groups that suffered from these rubbish posts, and checked what was loaded before I upgraded to 6.90RC1,
while I was running 6.90RC1 and after I downgraded to 6.82 again.

I saw that during the use of 6.90RC1 I was getting more than 10 times the number of headers being allowed through, compared to after I downgraded. e.g.
RC1 - 1 day - 53,505 6.82 - 20hrs - 4,021
RC1 - 1 day - 90,959 6.82 - 20hrs - 4,511

My filters have Subject Reject If and Poster Reject If clauses and also a Minimum File Size applied.

Using 6.82, I removed the size filter and reprocessed the last 20hrs of headers in one group - 653 MB of data in 22 files,
this added 13, 290 headers to the group and took about 9 mins to process.
I re-enabled the size filter, deleted the affected, extra headers from the group.

I re-installed RC1 and re-imported the same 20hrs headers in 22 files.
I verified that the minimum file size filter appeared in the filter in RC1.
this added about 5,800 new headers to the group and took about 5 mins to process.

This is a strange number - I expected a similar number to before.

In addition, I saw headers of a distinctive pattern being allowed in here that were excluded before by
a Reject If subject filter e.g.
[PRiVATE] \459790d650\::30339fbc4a89e5.4155fb5a4438d342c982033b0e4452.640630f1::/876db156a7ec/ [newzNZB] [2/12] - yEnc 1792917561
and a Poster filter for the same header e.g.
nEwZ[NZB] <pr3d@NET.world>

It seems to me that my filters work as I expected in 6.82 but that RC1 has problems with them.

This is not an exhaustive test but, unless you can see something wrong with what I've done,
I think that there is something here to be investigated?

I am falling back to 6.82 as, on my system at least, RC1 is polluting my spool.

Please let me know if you need more info. If you have a build that has enhanced debug info, like
record counts being processed, rejected, accepted per header import file, and any other useful stuff,
I'd be happy to try it out.

regards
Stavros
stavros
Seasoned User
Seasoned User
 
Posts: 118
Joined: Wed Dec 24, 2003 2:07 am

Registered Newsbin User since: 12/24/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Thu Aug 19, 2021 10:22 am

Ok I tracked it down. The fix will be in RC2. Thanks. I'm trying to get it out this week. I keep finding cosmetic stuff to fix.

I did add colors for each download file so you can see how much data is coming from which server.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby stavros » Thu Aug 19, 2021 11:53 am

Thanks Quade, looking forward to the next one :-)
stavros
Seasoned User
Seasoned User
 
Posts: 118
Joined: Wed Dec 24, 2003 2:07 am

Registered Newsbin User since: 12/24/03

Re: Newsbin Beta 6.90RC1 - build 5424

Postby mkrubsack » Thu Aug 19, 2021 5:46 pm

Quade wrote:I did add colors for each download file so you can see how much data is coming from which server.


I like that!
mkrubsack
Active Participant
Active Participant
 
Posts: 61
Joined: Sat Aug 25, 2012 5:39 pm
Location: California

Registered Newsbin User since: 08/25/12

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Thu Aug 19, 2021 9:45 pm

Yeah I wasn't going back to individual chunks colors but you guys convinced me that you need to know when it was banging on your pay server or erroring.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Bismark » Sat Aug 21, 2021 4:01 am

I think in older versions, items that were rejected had a red dot on the left of the subject. I found that helpful.
Bismark
Seasoned User
Seasoned User
 
Posts: 172
Joined: Fri Feb 26, 2010 4:03 am

Registered Newsbin User since: 02/27/10

Re: Newsbin Beta 6.90RC1 - build 5424

Postby weezee » Mon Aug 23, 2021 9:05 am

Problem with obfuscated filenames.
I've just tried this build. Downloaded all NZB's of a TV series from nzbgeek. They had clear titles.
After downloading with NewsBin I got all the files except the first with scrambled filenames.
This has been happening for a long time before, but I could manually look for those scrambled strings inside the NZBs and rename accordingly.
This time none of the filenames actually appears inside NZBs, so I can't know what is what.
Any way to fix this?

Code: Select all
23/08/2021  08:41    <DIR>          .
23/08/2021  08:41    <DIR>          ..
23/08/2021  08:41    <DIR>          000
05/07/2020  13:26       880.324.745 05a3fe76518a401d972faf004e8c09ea-(0001).mkv
05/07/2020  13:49       891.705.304 071b87200024445e9c321d6a82ecf7a4-(0001).mkv
05/07/2020  13:39       887.663.973 07cd04eea0bf4406a7abacbf31f34587-(0001).mkv
05/07/2020  13:15       877.742.861 0c33224e5b31403aa503d0f5804370e6-(0001).mkv
05/07/2020  15:12       882.760.452 0f98693a24bb4dc9ad7ed715c7c83de6-(0001).mkv
05/07/2020  14:01       878.268.545 11d32b63c3f74dd89ecfd8b08fc012d9-(0001).mkv
05/07/2020  13:35       877.944.819 1998c2a6422145318e47a0c06406f27e-(0001).mkv
05/07/2020  13:46       880.024.311 2360f02c99084e7ebb1fbddd000038f5-(0001).mkv
05/07/2020  13:16       888.448.731 2827bad322334205896eca1fc5516e6b-(0001).mkv
05/07/2020  13:55       890.300.627 40edaa54b9484eabbbba9878b9ea88e2-(0001).mkv
05/07/2020  13:24       882.601.828 424ece02ff6e4d23b5cff7a1a2b3ee3a-(0001).mkv
05/07/2020  13:23       726.160.054 4831b93189054466a00fa64a53f85534-(0001).mkv
05/07/2020  13:48       886.114.443 4c94eb1578c24c74a55247d3f454e9cc-(0001).mkv
05/07/2020  13:37       875.836.965 5f777d370bff4b179430899b4d90a0dc-(0001).mkv
05/07/2020  13:47       865.206.259 6a6b614be17649fc91960bbb8b2a711c-(0001).mkv
05/07/2020  13:39       830.583.830 6d859f60a80e453198eb2d4f9c64ea7c-(0001).mkv
05/07/2020  13:33       889.004.262 7a5887f080d04f8a8accd165e8396496-(0001).mkv
05/07/2020  15:13       877.536.543 8421a9ff5f2148d58d5ce86a4a450db6-(0001).mkv
05/07/2020  13:34       886.959.069 846ccdcd8dba4580bb3f1666c7163124-(0001).mkv
05/07/2020  13:22       891.476.492 9edcc285650f488381e42dd4b70257b8-(0001).mkv
05/07/2020  13:32       893.284.492 a1865c20910f401891f874bcfbd0e38f-(0001).mkv
05/07/2020  13:31       884.733.468 a66ed5ea57d4419fa40850c87766664a-(0001).mkv
05/07/2020  13:25       879.694.214 a816579247254b948f43994e14556f22-(0001).mkv
05/07/2020  13:26       867.342.506 b2d784c079f544a2a8e02f502d70a0fd-(0001).mkv
05/07/2020  13:50       880.090.236 b2f8ffea73f24c87b055012db1dfa7a6-(0001).mkv
05/07/2020  13:17       865.303.093 b8143e8d42d14c199031ec15752c8d7d-(0001).mkv
05/07/2020  13:45       891.481.575 be6de1e312bd414cb34fe4a7f1dea2cc-(0001).mkv
05/07/2020  13:44       856.170.136 e02efd2a93ec4844b1998e932e9a4b8f-(0001).mkv
05/07/2020  13:21       891.438.237 ea34650a38f4439f9cc359a3ade6228e-(0001).mkv
05/07/2020  16:13     2.816.908.709 f2701ff52f754b9ca2be2006a2e47ebc-(0001).mkv
05/07/2020  13:13       887.862.872 [Title.of.Series].S02E01.720p.AMZN.WEB-DL.DDP2.0.H.264-NTb.mkv
23/08/2021  08:41                 0 list.txt

I can't post any NZB as example because they exceed characters limit.
User avatar
weezee
Seasoned User
Seasoned User
 
Posts: 177
Joined: Tue Jul 26, 2005 11:48 am

Registered Newsbin User since: 07/26/05

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Mon Aug 23, 2021 11:36 am

You can email it to ts@newsbin.com

I'll check it out.

I have an action item for "all in one obscured NZB's" the issue is the de-obscurer uses the NZB Filename but for all in one NZB's (where a season is posted in a single NZB) the NZB Filename isn't sufficient. I didn't have a good sample NZB to test against.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Mon Aug 23, 2021 12:46 pm

Got the email.Thanks.

I drag and dropped the zip file onto the download list and every file downloaded, unrared and was renamed. Did you send them directly to the download list when you downloaded them? That's the only way obscured rename will work.

"Load NZB" from the files menu lets you look at the NZB's but I wouldn't download anything obscured that way. It's also possible this is something I fixed for RC2 which might go up today.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby weezee » Mon Aug 23, 2021 1:54 pm

I usually save plain unzipped NZBs in a folder then load them together from menu.
I will try this drag and drop thing, never done this way.

EDIT Yes, it seems to be working, the first four files have been renamed, it's doing fine.
Thanks Quade
User avatar
weezee
Seasoned User
Seasoned User
 
Posts: 177
Joined: Tue Jul 26, 2005 11:48 am

Registered Newsbin User since: 07/26/05

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Quade » Mon Aug 23, 2021 2:13 pm

There's other ways to do it too. Double-clicking the NZB in explorer or setting up a watch folder in Newsbin. If you save zips or NZB's to this folder, Newsbin will pick them up and download them.

Sending NZB's directly to the download list is now the default when double-clicking or "Opening" from the web browser.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin Beta 6.90RC1 - build 5424

Postby weezee » Mon Aug 23, 2021 2:43 pm

Yes, but as I mentioned in a previous build, I don't like the fact that doing so the download starts automatically. I prefer checking stuff and/or choosing different download folders before downloading.
User avatar
weezee
Seasoned User
Seasoned User
 
Posts: 177
Joined: Tue Jul 26, 2005 11:48 am

Registered Newsbin User since: 07/26/05

Re: Newsbin Beta 6.90RC1 - build 5424

Postby Stuart264 » Mon Aug 23, 2021 5:38 pm

Quade, not sure if this has been reported already but I found an issue with the speed limiter after I got an cable speed upgrade to 250mb. I have the slider set to 100mbs however when I use it in Newsbin it limits to around 50mbs which from memory was the old maximum limit.
User avatar
Stuart264
Seasoned User
Seasoned User
 
Posts: 227
Joined: Sun Jun 13, 2004 8:25 am
Location: United Kingdom

Registered Newsbin User since: 04/10/04

Next

Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: Bing [Bot] and 2 guests