Re: [netcdfgroup] NetCDF-4 filesize question.

• To: Wei Huang <huangwei@xxxxxxxx>, NetCDF Mailing List <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] NetCDF-4 filesize question.
• From: Kent Yang <myang6@xxxxxxxxxxxx>
• Date: Tue, 15 Apr 2014 01:30:39 +0000

Hi Wei,

I don't think your programs cause this. For unlimited dimensions, the HDF5 
chunking storage is used. There are space overheads for chunks, especially when 
the  chunk size is small. One way to mitigate this issue  is to increase the 
chunk size and then to use compression.

Kent

From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx 
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Wei Huang
Sent: Monday, April 14, 2014 4:38 PM
To: NetCDF Mailing List
Subject: [netcdfgroup] NetCDF-4 filesize question.

Hello group,

I am doing some NetCDF-4 test, and find when I use fixed, and unlimited 
dimensions cause file size
changed dramatically.

Below is a list of files (with dimension 5 x 10 x 73 x 144, with few group 
names):

1. has one unlimited dimension (the most left one)
2. has two unlimited dimensions (the left two)
3. has fixed dimensions.

-rw-r--r--   1 huangwei  CIT\Domain Users   6312828 Apr 14 15:25 
NCLcreatedNC4.nc.1unlimited
-rw-r--r--   1 huangwei  CIT\Domain Users  10508612 Apr 14 15:24 
NCLcreatedNC4.nc.2unlimited
-rw-r--r--   1 huangwei  CIT\Domain Users   2112758 Apr 14 15:26 
NCLcreatedNC4.nc.fixed


My question is: does our program (has issues which) cause the file size 
difference,
or, is it NetCDF4 which needs the extra space (for unlimited dimension)?

Thanks,


Wei Huang

• Follow-Ups:
  • Re: [netcdfgroup] NetCDF-4 filesize question.
    • From: Chris Barker - NOAA Federal

• References:
  • [netcdfgroup] NF90_CHAR vs NF90_STRING for attribute values
    • From: Mark Hadfield
  • Re: [netcdfgroup] NF90_CHAR vs NF90_STRING for attribute values
    • From: Mark Hadfield
  • [netcdfgroup] NetCDF-4 filesize question.
    • From: Wei Huang