Hi All:
We just converted one or our larger datasets (larger in terms of the number of
files that are aggregated) into compressed netCDF4. There is a substantial
savings in storage, but we wanted to do a series of tests to see what hit in
access time we would take, if any, wsince many of our users will make requests
involving a lot of time periods.
In order to design these tests properly, we need to get a better understanding
of how the TDS handles netcdf4 datasets that have compression. Are the
decompressed data cached, or more accurately cached any differently from data
read from an uncompressed series of netcdf3 files, or since the decompression
is handled automatically on the read, is everything handled the same after that?
We would also be interested other peoples experience with compressed netcdf4
files in TDS, in particular when the extracts are not synoptic, but cover a lot
of time periods in a region, or make a lot of very small calls to a large
number of time periods - such as we need to do for tagging data.
Thanks for any info,
-Roy
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097
e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
