Files don't get unrared if final filepath is very long

This is the place to help test and discuss Version 6 Beta releases.

Files don't get unrared if final filepath is very long

Postby billynews » Mon Nov 25, 2024 9:12 am

Hi,

I've noticed that files don't get unrared if the resulting (downloadpath + rar internal path + filename) filepath exceeds 259 characters.
They are just missing in the UnRAR Folder.

I've also noticed that Newsbin shows a size of 18446744 TB for these files on the "Files List"-Tab.

I will send you an example NZB and what to do to verify this.

Best regards,

billy
billynews
Seasoned User
Seasoned User
 
Posts: 146
Joined: Sat Apr 12, 2008 2:57 pm

Registered Newsbin User since: 07/15/07

Re: Files don't get unrared if final filepath is very long

Postby Quade » Mon Nov 25, 2024 10:03 pm

Yeah this is always problematic. Even though Newsbin uses UNC paths that are supposedly allowed to exceed 255 bytes total length.

I recommend keeping the download and unrar paths as short as possible.

X:\DL\

X:\Decode\

Don't use $(GROUP). $(NZBFILE) in the decode path might blow things over but most people use it.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45009
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Files don't get unrared if final filepath is very long

Postby billynews » Tue Nov 26, 2024 12:50 pm

Quade wrote:Don't use $(GROUP). $(NZBFILE) in the decode path might blow things over but most people use it.


Yeah, I'm guilty - I used $(NZBFILE) but changed it after running into this problem.

It's unfortunate that there is no error message or something.

Btw. if I manually unpack with either WinRAR 6.23 or RAR 6.23 resp. this doesn't happen. So perhaps you can upgrade the RAR-version used by Newsbin :wink:
billynews
Seasoned User
Seasoned User
 
Posts: 146
Joined: Sat Apr 12, 2008 2:57 pm

Registered Newsbin User since: 07/15/07

Re: Files don't get unrared if final filepath is very long

Postby billynews » Wed Nov 27, 2024 12:42 pm

Is it correct that the subject you pass to a post-unrar script is $(NZBFILE) if I load a NZB from a NZB Watch Folder? It looks like it.

If yes, I could move the Unrar-Folder e.g. D:\DL to D:\$(NZBFILE) in a post-unrar script which would basically solve my problem.

I also noticed that the input files are deleted after the post-unrar script has finished which lets me check if all files in Output.txt
exists and if not move the Download Folder so that Newsbin can't delete it and I get a chance to manually unrar.

P.S.: Would it be possible to pass an additional parameter, the name of the NZB Watch Folder, to the post-unrar script?
If not, no problem. I could change my Download Folder from currently E:\newsbin\$(NZBFILE)\ for all NZB Watch Folders to E:\newsbin\pictures\$(NZBFILE), E:\newsbin\music\$(NZBFILE) etc.
and read the Input.txt and get the "category" from there.
billynews
Seasoned User
Seasoned User
 
Posts: 146
Joined: Sat Apr 12, 2008 2:57 pm

Registered Newsbin User since: 07/15/07

Re: Files don't get unrared if final filepath is very long

Postby Quade » Wed Nov 27, 2024 2:41 pm

The subject you see in the download list is doctored on the fly to include the NZB Filename. It's probably not sent during the post-unrar script. Just looking, it looks like you get the raw subject from one of the files. The NZB Filename is sent to the script.

I'm in the middle of making a test for really long file paths using the rared up NZB you sent. I suspect the issue is downloading and unraring to the same path where Newbin will actually unrar to a deeper temporary folder then move the files after the unrar. The issue is that you can have files inside the RAR set that overwrite the actual source rars so, you can't reliably download and unrar to the same path.

I'd say downloading to a unique path with $(NZBFILE). Then decoding to some other folder is the most reliable way to download.

X:\DL\$(NZBFILE)\

X:\Decode\$(NZBFILE)\

For instance.



The file paths in the rar you sent aren't THAT long.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45009
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Files don't get unrared if final filepath is very long

Postby Moondawgie » Thu Nov 28, 2024 3:08 am

This is a reported problem with Windows. At least with Windows 10 Pro 64 bit.

Feedback Hub: No Longer Able to Create or use file names longer than 247 characters.
MS Windows 10 Pro 64-Bit I- ASUS P8Z77-V LE; Intel i7-3770K 3.5 GHz; 16.0GB DDR3; 600/20 Mbps cable internet.
User avatar
Moondawgie
Seasoned User
Seasoned User
 
Posts: 426
Joined: Mon Aug 13, 2001 5:17 pm

Registered Newsbin User since: 05/17/03

Re: Files don't get unrared if final filepath is very long

Postby billynews » Thu Nov 28, 2024 9:07 am

Moondawgie wrote:This is a reported problem with Windows. At least with Windows 10 Pro 64 bit.


This seems to be something different.

I can manually unrar the download with no problem. It's only a problem if Newsbin does the unrar.
billynews
Seasoned User
Seasoned User
 
Posts: 146
Joined: Sat Apr 12, 2008 2:57 pm

Registered Newsbin User since: 07/15/07

Re: Files don't get unrared if final filepath is very long

Postby billynews » Fri Nov 29, 2024 2:26 am

Hi Quade,

I've written a small post-unrar script with Autohotkey which validates if all files from Output.txt are present in the Unrar-Folder.
If not, it displays a modal message so it doesn't exit, Newsbin doesn't delete the source files and I get the chance to manually unrar the download.

Code: Select all
Loop, Read, C:\Users\USERNAME\AppData\Local\Newsbin\Scripts\Output.txt
{

  if not FileExist(A_LoopReadLine)
  {
    MsgBox, %A_LoopReadLine%
    break
  }

}


It works well but then I noticed that FileExist() returns false for these files with long filenames even if they do exist!!!
I looked into it and in the end I found https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

After setting

Code: Select all
Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001


as described in the article FileExist() now works as expected.

Maybe you want to look into this article.

Best regards,

billy
billynews
Seasoned User
Seasoned User
 
Posts: 146
Joined: Sat Apr 12, 2008 2:57 pm

Registered Newsbin User since: 07/15/07

Re: Files don't get unrared if final filepath is very long

Postby Moondawgie » Fri Nov 29, 2024 3:35 am

To my undereducated mind, that article seems to explain why WinRAR can write files with long "filenames" to Windows folders that can't then be renamed, deleted or moved using Windows File Explorer.
MS Windows 10 Pro 64-Bit I- ASUS P8Z77-V LE; Intel i7-3770K 3.5 GHz; 16.0GB DDR3; 600/20 Mbps cable internet.
User avatar
Moondawgie
Seasoned User
Seasoned User
 
Posts: 426
Joined: Mon Aug 13, 2001 5:17 pm

Registered Newsbin User since: 05/17/03

Re: Files don't get unrared if final filepath is very long

Postby Quade » Sat Nov 30, 2024 12:23 am

You have to use UNC pathing for long filename support to work.

"\\?\X:\Decode\"

So I did some testing and when I decoded to a different path than the RAR, all the files showed out out beyond 370 bytes. When I decoded to the same path as the rars, it used the temp folder, All the files that were longer than 260 bytes were lost. I believe they were decoded but the move to the decode folder lost the really long ones. It's something I'm fixing.

What I probably need to do is post a rar with huge path length then download so the entire decode chain is tested.

My version of Win10 can delete files longer than 260 bytes. It wasn't able to before. I assume MS finally updated Explorer to handle really long filenames. The recycle bin can't handle long file paths still. When you delete really long paths is reports it can't recycle them.

Edit: While Windows can support long file paths, it doesn't support them without jumping through hoops.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45009
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Files don't get unrared if final filepath is very long

Postby Quade » Mon Dec 02, 2024 1:22 pm

Looking at this more. When using RAR to decode, I could get it to work when either the rar was in the same path as decode and when not. On the other hand 7zip failed to decode them to the long paths.

I'm looking at that next.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45009
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: Google [Bot] and 6 guests