data file compression / cache / bz2?

[John Caron]

Hi Rob:

The files should be in content/thredds/cache/ and YDS allows up to 1 Gbyte before deleting oldest files. It is possible to change the place, but not yet the maximum size of the cache. Another possible cache stategy is to delete files after a certain age (which makes more sense for a rolling archive like motherlode). 

Im going to make some/all of this user settable. what would you like to do?

If file ends with ".Z", ".zip", ".gzip", ".gz", or ".bz2", it will uncompress/unzip and write to new file without the suffix.


Rob Cermak wrote:
> Hi,
> 
> Is there a way to have THREDDS systematically clean up uncompressed
> data files?   We noticed that THREDDS can support compressed (gzip)
> NetCDF files, but the uncompressed files linger.
> 
> A periodic program can be run to scan through and remove them, but
> this would assume that the periodic cleaning would keep up with the
> rate of uncompression as data is accessed.
> 
> Maybe something similar to OPeNDAP Server 3 (dods_cache)?  Even this
> has its limits.  If the cache size is not high enough, with enough
> traffic, requests will get stomped.   Or just a table in memory
> tracking uncompressed files.  If the server/system ever crashed,
> then this would leave stuff about.
> 
> We were wondering if the same compression support can be extended to
> include bzip2 compression (bz2)?
> 
> Thanks!
> Rob

==============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================