newsbin.com

Hi,

I upgraded from 6.80 RC3 to RC4 and have noticed a huge increase in the data footprint in the IMPORT directory as a result of the saving of the downloaded headers in text format.

My IMPORT directory is on an SSD (the spool is on a hard drive) and I am concerned that the increase is going to affect the life span of the SSD adversly.
With RC3 I was seeing a daily footprint of about 2.5GB of data in the IMPORT directory, with RC4 I am seeing close to 50GB per day.

I don't know enough about the way NB handles the downloading and storage of headers to decide if this is a problem or not for the SSD - if the RC3 way was still writing the 50GB, but I was only seeing the 'final' .gz state of the downloaded headers in the IMPORT directory, or if the amount of data actually written to the SSD was closer to the 2.5GB.

Is it wise to use RC4 wih an SSD or should I move the NB IMPORT directory to a hard drive?

regards
Stavros.

If you turn on compression for the import folder, the total amount of data between the GZ and Text file probably isn't that different. I've been using SSD's for awhile now and have never had one "age out". I don't even worry about wear anymore with my Samsung SSD's. They run far longer than their rated limits.

Thanks Quade.

In RC3, when headers are read 'off the wire' are they landed to a file on the disc, then compressed to a .gz? (even if xfeatures is turned on?)
or are they compressed in memory and then only the compressed data in a .gz file is actually landed to the disc?

I guess that RC4 is just going to read the data off the wire (then uncompress it if xfeatures is on), then just write it to the disc as a text file?

I've been doing some reading (a dangerous thing ) and there are some old posts/advice about not using NTFS compression, SSDs and large files, especially with a Sandforce controller (it already has compression built in).

The SSD is a Samsung SSD 850 EVO 500G SCSI Disk Device, so it may be ok to try compression and see how it goes.

Headers are read off the wire. If compressed with xfeatures they're decompressed. Then written to the GZ or Txt file.

In our testing the pure TXT import files import faster than the GZ Compressed files.

Yeah, compressing with a sandforce drive is probably pointless. I don't believe any Samsung drives use sandforce. NTFS compression is what I'm suggesting.

OK, thanks. I'll give it a go and keep an eye on the SSD drive state.