[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #LHC-295018]: ncgen: formatting small floats

One other detail I think worth mentioning, if you're using ncdump/ncgen to
"conserve data integrity as much as possible during conversion to netCDF",
is the "-p" option of ncdump to specify precision of the string representation
for floating-point values.  Check out the ncdump man page documentation
for this option, which says the default for floating-point and double
precisions are 7 and 15 respectively, but to preserve every bit of the binary
representations, you need 9 and 17 digits in the string representation for
some ranges of values, as described in this paper:

  What Every Computer Scientist should Know About Floating-Point Arithmetic


Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu

Ticket Details
Ticket ID: LHC-295018
Department: Support netCDF
Priority: Normal
Status: Closed

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.