[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #TKC-172853]: Missing values and TDS

NCL seems to handle NaNs:

http://www.ncl.ucar.edu/Document/Functions/Built-in/isnan_ieee.shtml

have you tried using that?

> Ethan
> 
> My basica problem is that NCL does not interpret NaN correctly. So, before
> doing any plotting or analysis, for all reads of CFSR data, I need to have
> something like
> 
> replace_ieeenan (hgt, value, 0)
> hgt@_FillValue=1.e20
> hgt@missing_value=1.e20
> 
> which is extra time and lines of code. This could be interpreted as a NCL
> issue of course (it probably is). But, I see in the Unidata doc (from Google
> search)
> 
> Using an IEEE infinity or NaN for a fill value is not recommended, because
> the resulting file may not be interpreted correctly on platforms that don't
> use the IEEE 754 Standard for floating-point. If you use non-normal numbers
> for fill values, a fix is to apply this
> patch<http://www.unidata.ucar.edu/software/netcdf/patches/vardata-patch>
> to
> the file ncdump/vardata.c and rebuild *ncdump*. If you don't have the IEEE
> finite() function used in this patch in libm, you can try compiling
> ncdump/vardata.c with a -DNO_FINITE flag and with compiler flags that
> support IEEE comparisons (e.g. NaN != NaN).
> 
> That isn't the issue for NCL but it does indicate that Unidata seems it's a
> bad idea to use NaN.
> 
> I haven't checked too much but I know it does work in GrADS and IDV and our
> dncdump. I would be very curious about Fortran but don't have compiled code
> for OPEnDAP.
> 
> Cathy
> 
> 
> 
> 
> address@hidden> wrote:
> 
> > > Ethan
> > >
> > > I am using opendap access to the NOMADS server. I open the files in NCL
> > > and then read them. I also tested open the files using dncdump
> > >
> > > dncdump -h
> > >
> > http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/197901/ocnh06.gdas.197901.grb2
> > >
> > > and see NaN as the missing value there. e.g.
> > >
> > > ......
> > > float Vertical_velocity_geometric(time, depth_below_sea, lat, lon) ;
> > > Vertical_velocity_geometric:long_name =
> > > "Vertical_velocity_geometric @ depth_below_sea" ;
> > > Vertical_velocity_geometric:units = "m s-1" ;
> > > Vertical_velocity_geometric:missing_value = NaNf ;
> > > .......
> > >
> > > We downloaded the grib file directly and looked at it. It does not have
> > > the NaN for any of the values as there is a mask applied when the grib
> > > is converted to netcdf (in this case, a land mask).
> > >
> > > So, it is how the grib files are being converted to netCDF, as you
> > suggest.
> > >
> > > Let me know if I can answer any other questions.
> > >
> > > Cathy
> >
> > Hi Cathy:
> >
> > The rib file does not store the missing values, so each library chooses
> > that representation. The netcdf-java library uses NaNs to indicate missing
> > values. This currently cannot be changed. What problem are you having with
> > portability of NaNs?
> >
> > Ethan
> >
> 
> NCL does not read NaN correctly. So, for all reads, I need to have in
> addition
> 
> replace_ieeenan (hgt, value, 0)
> hgt@_FillValue=1.e20
> hgt@missing_value=1.e20
> 
> 
> If you use an IEEE infinity as the value of the _FillValue attribute for a
> variable, the *ncdump* program will output _ for ordinary values of the
> variable, as if they had not been written.
> 
> Using an IEEE infinity or NaN for a fill value is not recommended, because
> the resulting file may not be interpreted correctly on platforms that don't
> use the IEEE 754 Standard for floating-point. If you use non-normal numbers
> for fill values, a fix is to apply this
> patch<http://www.unidata.ucar.edu/software/netcdf/patches/vardata-patch>
> to
> the file ncdump/vardata.c and rebuild *ncdump*. If you don't have the IEEE
> finite() function used in this patch in libm, you can try compiling
> ncdump/vardata.c with a -DNO_FINITE flag and with compiler flags that
> support IEEE comparisons (e.g. NaN != NaN).
> 
> 
> >
> > >
> > > On 9/28/11 10:31 AM, Unidata THREDDS Support wrote:
> > > > Hi Cathy,
> > > >
> > > > How are you getting the data? Are you using the NetCDF Subset Service
> > (NCSS) access method to request the dataset?
> > > >
> > > > There is no way to specify that you would like non-NaN missing values
> > when you request a netCDF file from the TDS. I suspect it isn't a simple
> > matter in this case as it has to do with how the GRIB files are being
> > converted into netCDF.
> > > >
> > > > However, I'm going to ask John Caron to give a more definitive answer.
> > Unfortunately he won't be back from vacation for about two weeks.
> > > >
> > > > Ethan
> > > >
> > > > Cathy Smith wrote:
> > > >> I am reading CFSR data from NOMADS and missing values are being
> > returned
> > > >> as NaN (for example, for  ocean variables over land) from the TDS.
> > This
> > > >> causes problems for me in NCL and likely would in other languages.
> >  Can
> > > >> missing values be returned as something more portable such as
> > 9.9692e+36
> > > >> as is the default in C?
> > > >> Thanks,
> > > >> Cathy Smith
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: TKC-172853
> > > > Department: Support THREDDS
> > > > Priority: High
> > > > Status: Open
> > > >
> > >
> > > --
> > > ----------------------------------------------
> > > NOAA/ESRL PSD and CIRES CDC
> > > 303-497-6263
> > > http://www.esrl.noaa.gov/psd/people/cathy.smith/
> > >
> > > Emails about data/webpages may get quicker responses from emailing
> > > address@hidden
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: TKC-172853
> > Department: Support THREDDS
> > Priority: Critical
> > Status: Open
> >
> >
> 
> 


Ticket Details
===================
Ticket ID: TKC-172853
Department: Support THREDDS
Priority: Critical
Status: Open