[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #AEB-315486]: nf90_float too accurate...

George,

> Thanks very much for your help.
> Ok, maybe I was just getting a little confused here. If I do an ncdump
> and look at my (typical) variable dd12_3d this is what I see (below).
> The numbers have 6 decimal places and are denoted as float.  Is this
> what is stored in the file (ie, a 6 decimal place number) - ie, is the
> ncdump command showing the actual number (as opposed to a truncated
> version) ?  If so then I think I'm fine - if I really need to cut down
> the filesize I can use the integer data packing method you describe -
> that is clear.

No what is actually stored in the file is a binary 32-bit floating-point
IEEE-754 representation of the value, which typically has 24 bits of 
precision in the mantissa.  The 7 significant digits that ncdump shows
are usually enough for all the precision, but if you really want every last
bit of precision represented in the ncdump output for all possible 
floating-point values, you will have to specify "-p  9" to ncdump, 
according to Theorem 15 of the paper:

  What Every Computer Scientist should Know About Floating-Point 
  Arithmetic, D.   Goldberg, ACM Computing Surveys, Vol. 23, No. 1, 
  March 1991, pp. 5-48.

which says

  When a binary IEEE single-precision number is converted to the
  closest eight digit decimal number, it is not always possible to
  recover the binary number uniquely from the decimal one. If nine
  decimal digits are used, however, then converting the decimal number
  to the closest binary number will recover the original
  floating-point number.

Sorry to be pedantic, but binary-to-decimal conversion has those sorts
of extreme corner cases!  

--Russ

> >>>>>>>>>>>>>>>>>>>>>>>>>>
> 
> 
> netcdf wsa_enlil.latest.suball {
> dimensions:
> x = 512 ;
> y = 60 ;
> z = 180 ;
> t = 169 ;
> variables:
> float x_coord(x) ;
> float y_coord(y) ;
> float z_coord(z) ;
> float time(t) ;
> float dd12_3d(t, y, x) ;
> 
> ......
> 
> data:
> 
> dd12_3d =
> 4.689313e-19, 4.519964e-19, 4.336969e-19, 4.148909e-19, 3.958285e-19,
> 3.771553e-19, 3.594542e-19, 3.427877e-19, 3.272004e-19, 3.126775e-19,
> 2.991249e-19, 2.864187e-19, 2.744344e-19, 2.631388e-19, 2.524795e-19,
> 2.42441e-19, 2.330259e-19, 2.242003e-19, 2.159138e-19, 2.081488e-19,
> 2.008492e-19, 1.939533e-19, 1.874012e-19, 1.811512e-19, 1.751806e-19,
> 1.694775e-19, 1.64039e-19, 1.588641e-19, 1.539523e-19, 1.492887e-19,
> 
> 
> 
> 
> 
> On 3/15/2011 3:56 PM, Unidata netCDF Support wrote:
> > Hi George,
> >
> >> Hi there,
> >>
> >> I was wondering if you could help me.  I have searched the mailing lists
> >> but can't find anything suitable.
> >>
> >> I am using netcdf as the format for output from an atmospheric model.
> >> The data in the netcdf is used solely for color contour plotting using
> >> IDL. The problem I have is that the nf90_float data is much larger than
> >> it needs to be.  For instance a typical number written out from my idl
> >> code is:
> >>
> >> 2.43866705894470210000e+000
> >>
> >> There is way more accuracy in this number than needed for color
> >> applications - a simple:
> >>
> >> 2.439e+00 would be perfectly adequate.
> >>
> >> Why this is a problem is that the datafiles are very large - because
> >> they are large 2D arrays so the nf90_float accuracy is a problem.
> >>
> >> What I would ideally like is something like an NF90_smallfloat or
> >> something.  Or is there another way of doing this ?
> >>
> >> I am using netcdf 3.5 right now btw.
> >
> > The nf90_float type is only a 32-bit float, with about 5 significant
> > digits of precision, as documented here:
> >
> >    
> > http://www.unidata.ucar.edu/netcdf/old_docs/really_old/f90-html-docs/guide7.html
> >
> > If you're seeing more precision than that, it may be the result of
> > printing the values as double-precision.
> >
> > Arrays of nf90_float values require 4 bytes of storage for each value
> > in a binary netCDF file.  If you want to use less storage than that,
> > you could pack the values into 2-byte nf90_short values, as described
> > here:
> >
> >    
> > http://www.unidata.ucar.edu/netcdf/docs/BestPractices.html#Packed%20Data%20Values
> >
> > using the add_offset and scale_factor attributes for the packed variable.
> >
> > If you want to see the values with less precision when you print them out
> > with the ncdump utility, you can use the "-p float_digits[,double_digits]"
> > option to ncdump, as documented here:
> >
> >    http://www.unidata.ucar.edu/netcdf/docs/ncdump-man-1.html
> >
> > If we've misunderstood your question, please let us know.
> >
> > --Russ
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: AEB-315486
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> 
> --
> ----------------------------------------------------------------
> Dr. George Millward
> Cooperative Institute for Research in Environmental Sciences (CIRES)
> University of Colorado, Boulder
> 
> Visiting scientist at
> NOAA Space Weather Prediction Center
> 325 Broadway, Boulder, CO 80305
> Tel: 303-497-6775
> 
> address@hidden
> ----------------------------------------------------------------
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: AEB-315486
Department: Support netCDF
Priority: Normal
Status: Closed