[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #LHC-295018]: ncgen: formatting small floats



One other detail I think worth mentioning, if you're using ncdump/ncgen to
"conserve data integrity as much as possible during conversion to netCDF",
is the "-p" option of ncdump to specify precision of the string representation
for floating-point values.  Check out the ncdump man page documentation
for this option, which says the default for floating-point and double
precisions are 7 and 15 respectively, but to preserve every bit of the binary
representations, you need 9 and 17 digits in the string representation for
some ranges of values, as described in this paper:

  What Every Computer Scientist should Know About Floating-Point Arithmetic
  http://www.validlab.com/goldberg/paper.pdf

 --Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: LHC-295018
Department: Support netCDF
Priority: Normal
Status: Closed