Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [netcdfgroup] Precision an dSignificant digits in netcdf

On Fri, Jul 8, 2016 at 1:26 PM, Ismail SEZEN <sezenismail@xxxxxxxxx> wrote:

> The answer from ESRL PSD data about precision and least_significant_digit:
>
> Hi
> Least significant digit is in the original grib files.


OK, so despite the docs these mean two different things -- Least
significant digit seems to represent the actual precision of the original
values (maybe -- it could have a specific meaning in GRIB...)


> Precision is calculated here and is related to the packing. The precision
> may be a little sloppy with regards to our packing.
>

and precision is related to the packing.

Hopefully, they have used a packing scheme that handles the full precision
of the original data, so you could jsut ignore that and use least
significant digit to guide your rounding.


> We hope to be transitioning to netCDF4 at some point soon, which is what
> we use for other datasets. It does not need scale/offset packing as it
> packes internally.
>

that would be a nice step.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@xxxxxxxx
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: