[netcdfgroup] questions about netCDf file size.

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] questions about netCDf file size.
• From: LB <bravo.loic@xxxxxxxxx>
• Date: Fri, 27 Aug 2010 15:30:20 +0200

Hi,

I'm very new to netCDF and I was surprised by the size of files generated by
netCDF.
Here is the header of the netCDF file :

ncdump -h cycle.nc
netcdf cycle {
dimensions:
        Regime = 3 ;

group: Regimes {
  variables:
        float T41(Regime) ;
        float TTR(Regime) ;
        float N1(Regime) ;

  } // group Regimes

group: Cycles {
  dimensions:
        Cycle = 5 ;
        SousCycle = 11 ;
  variables:
        byte Type(Cycle) ;
        short Nb_lignes(Cycle) ;
        short Indice(SousCycle) ;
        short Repeat(SousCycle) ;
        float Duree(SousCycle) ;

  } // group Cycles

group: Simulation {
  dimensions:
        Temps = UNLIMITED ; // (19168 currently)
  variables:
        float Temps(Temps) ;
        byte Regime(Temps) ;
        float T41(Temps) ;
        float e_p1(Temps) ;
        float Temps_Touche ;
  } // group Simulation
}

The generated file is 3.1 Mo and so 50% heavier than I had expected. Here is
how I calculated the expected file size :

     name  Dimension cell size Nb cell variable size    group: Regimes {
           float T41 Regime 32 3 96      float TTR Regime 32 3 96      float
N1 Regime 32 3 96    group: Cycles {



     byte Type Cycle 8 5 40      short Nb_lignes Cycle 16 5 80      short
Indice SousCycle 16 11 176      short Repeat SousCycle 16 11 176      float
Duree SousCycle 32 11 352    group: Simulation {



     float Temps Temps 32 19168 613376      byte Regime Temps 8 19168 153344
     float T41 Temps 32 19168 613376      float e_p1 Temps 32 19168 613376
  float Temps_Touche 32 1 32







  total size 1994616 Bits            soit 1.99 Mo

Is there anything I forgot ?

Besides, I would like to know how to if there is any efficient way to reduce
this size:

 - I've tried to use deflate_level=1 to 9 in nf90_def_var for the variables
in the simulation group but it didn't change the file size.
Is there anything else to do to to let netcdf compress data ? gzip seems to
compress the resulting file quite well.

 - Is there a way to store 16bits float to compress data (with loss) in
order to reduce the size of data where we have a limited resolution ?

Best regards,

-- 
Loic

• Follow-Ups:
  • Re: [netcdfgroup] questions about netCDf file size.
    • From: John Caron
  • Re: [netcdfgroup] questions about netCDf file size.
    • From: Cedric Roux