Duplicate detection no longer working

This is the place to help test and discuss Version 6 Beta releases.

Duplicate detection no longer working

Postby dash » Fri Jun 15, 2018 3:07 pm

Hi guys,

i have a prob with dup detection. It just stopped working a few versions back for me.
Now I'm on 6.81B2 and the problem still persist.
Can't tell for sure what was the last version where it worked properly. I think it was before the all the RC-releases of 6.80.

"Use Duplicate Detector" option is enabled, I've also tried deleting all .DB3 files in Data folder and starting with completely new config, but no avail.
I can download the same file over and over again, without newsbin failing the download due to dup detection like it was before.

Any Idea what else i can try?
dash
Occasional Contributor
Occasional Contributor
 
Posts: 14
Joined: Sun Sep 23, 2012 8:40 am

Registered Newsbin User since: 04/21/12

Re: Duplicate detection no longer working

Postby Quade » Fri Jun 15, 2018 10:29 pm

If the file is already in the folder Newsbin just looks at it, decides all is good and pretends it actually downloaded.

Are you deleting the downloaded file between attempts?

If I add a single file to the download, list, watch it download, then delete it and add it again, it gets sent to the failed list second time through.

Bla [04/70] - "bla.part02.rar" yEnc,X:\Down-Load\,52.2 MB,[1 files, 0 pars] D:0 DL:0 Retry:0 N:0,Duplicate Detection: file has already been downloaded ,2d:03h:10m
bla [05/70] - "bla.part03.rar" yEnc,X:\Down-Load\,52.2 MB,[1 files, 0 pars] D:0 DL:0 Retry:0 N:0,Duplicate Detection: file has already been downloaded ,2d:03h:10m
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 43133
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Duplicate detection no longer working

Postby dash » Sat Jun 16, 2018 3:54 am

>If I add a single file to the download, list, watch it download, then delete it and add it again, it gets sent to the failed list second time through.
I've tried it with a single file and indeed does work like you describe and this is what i expect.

But it doesn't work with with "file groups" (bunch of grouped .rar and .par2 files, that you typically get when you open some .nzb file).
In older versions it did work though. Newsbin failed the download for any already downloaded file group, doesn't matter when you downloaded it.
The whole group was then moved to "Failed Files" tab with Duplicate Detection reason/error.

In newer versions i can open the same .nzb file over and over again and newsbin will download and extract the file group creating copies of extracted files:
xubuntu-18.04-desktop-amd64.iso
xubuntu-18.04-desktop-amd64-(0001).iso
xubuntu-18.04-desktop-amd64-(0002).iso
dash
Occasional Contributor
Occasional Contributor
 
Posts: 14
Joined: Sun Sep 23, 2012 8:40 am

Registered Newsbin User since: 04/21/12

Re: Duplicate detection no longer working

Postby dash » Fri Jun 22, 2018 5:09 am

So any ideas?
dash
Occasional Contributor
Occasional Contributor
 
Posts: 14
Joined: Sun Sep 23, 2012 8:40 am

Registered Newsbin User since: 04/21/12

Re: Duplicate detection no longer working

Postby Quade » Fri Jun 22, 2018 1:20 pm

I'm seeing it prevent re-downloads of stuff I add from headers. Are you using NZB's? NZB's bypass the dup checker I believe.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 43133
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Duplicate detection no longer working

Postby dash » Fri Jun 22, 2018 4:36 pm

> Are you using NZB's? NZB's bypass the dup checker I believe.
Yes i do. I rarely index headers myself. I mostly download nzbs from index sites and Dup checker did work with NZB files as well.
dash
Occasional Contributor
Occasional Contributor
 
Posts: 14
Joined: Sun Sep 23, 2012 8:40 am

Registered Newsbin User since: 04/21/12

Re: Duplicate detection no longer working

Postby Moerges » Sun Jul 15, 2018 7:24 pm

Hello Quade, me again. I've noticed this new dupe handling behaviour as well (well, "new" compared to my 5.59 experience with Newsbin) and was on my way to open a thread about it when I found this one.

I almost exclusively download via .nzb files, and found the dupe checker in 5.59 rather useful to avoid unnecessary redownloads. Now that .nzbs simply bypass the dupe checker, it unfortunately no longer does the job it used to do for my usage scenario.

If it is not too much of a bother, would it be possible to reimplement the dupe checking for downloads added via .nzb files—perhaps as an option so the user can choose which behaviour he prefers?

---

Also, while testing the dupe checking behaviour of 6.80 build 5058, I came across an oddity that I suspect might be linked to the duplicate detection:

My Newsbin is set to download to G:\Newsbin temp\$(NZBFILE)\ and to unrar to D:\Newsbin\$(NZBFILE)\.

When I add a download via an .nzb file for the first time, it works as one would expect: the .rars are downloaded to G:\Newsbin temp\$(NZBFILE)\ and are then extracted to D:\Newsbin\$(NZBFILE)\.

However, when I add the same download via the same .nzb file a second time, instead of downloading the .rars to G:\Newsbin temp\$(NZBFILE)\, Newsbin downloads them to D:\Newsbin\$(NZBFILE)\ (the unrar folder) and afterwards extracts them there, too.

This happens no matter if I have deleted the first file or not, both from the "Files List" and from my disk. It also does not matter whether Use Duplicate Detector in the Advanced tab of the settings is checked or not. Still, I could imagine this is linked to the dupe checker in some way, or possibly some other form of download history.

I hope you can reproduce the issue on your end, or tell me how I can fix it on mine in case I am doing something wrong. (I am not running any filters or scripts, by the way. I'm simply adding the .nzbs—either via a watched folder or via double click on an .nzb file—without altering any settings or download paths in between.)

Again, thank you in advance for any insights you can offer.

Moerges
Moerges
n00b
n00b
 
Posts: 4
Joined: Sun Jul 15, 2018 3:25 am

Registered Newsbin User since: 01/29/10


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 3 guests

cron