[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #TKC-172853]: Missing values and TDS



NCL seems to handle NaNs:

http://www.ncl.ucar.edu/Document/Functions/Built-in/isnan_ieee.shtml

have you tried using that?

> Ethan
> 
> My basica problem is that NCL does not interpret NaN correctly. So, before
> doing any plotting or analysis, for all reads of CFSR data, I need to have
> something like
> 
> replace_ieeenan (hgt, value, 0)
> address@hidden
> address@hidden
> 
> which is extra time and lines of code. This could be interpreted as a NCL
> issue of course (it probably is). But, I see in the Unidata doc (from Google
> search)
> 
> Using an IEEE infinity or NaN for a fill value is not recommended, because
> the resulting file may not be interpreted correctly on platforms that don't
> use the IEEE 754 Standard for floating-point. If you use non-normal numbers
> for fill values, a fix is to apply this
> patch<http://www.unidata.ucar.edu/software/netcdf/patches/vardata-patch>
> to
> the file ncdump/vardata.c and rebuild *ncdump*. If you don't have the IEEE
> finite() function used in this patch in libm, you can try compiling
> ncdump/vardata.c with a -DNO_FINITE flag and with compiler flags that
> support IEEE comparisons (e.g. NaN != NaN).
> 
> That isn't the issue for NCL but it does indicate that Unidata seems it's a
> bad idea to use NaN.
> 
> I haven't checked too much but I know it does work in GrADS and IDV and our
> dncdump. I would be very curious about Fortran but don't have compiled code
> for OPEnDAP.
> 
> Cathy
> 
> 
> 
> 
> address@hidden> wrote:
> 
> > > Ethan
> > >
> > > I am using opendap access to the NOMADS server. I open the files in NCL
> > > and then read them. I also tested open the files using dncdump
> > >
> > > dncdump -h
> > >
> > http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/197901/ocnh06.gdas.197901.grb2
> > >
> > > and see NaN as the missing value there. e.g.
> > >
> > > ......
> > > float Vertical_velocity_geometric(time, depth_below_sea, lat, lon) ;
> > > Vertical_velocity_geometric:long_name =
> > > "Vertical_velocity_geometric @ depth_below_sea" ;
> > > Vertical_velocity_geometric:units = "m s-1" ;
> > > Vertical_velocity_geometric:missing_value = NaNf ;
> > > .......
> > >
> > > We downloaded the grib file directly and looked at it. It does not have
> > > the NaN for any of the values as there is a mask applied when the grib
> > > is converted to netcdf (in this case, a land mask).
> > >
> > > So, it is how the grib files are being converted to netCDF, as you
> > suggest.
> > >
> > > Let me know if I can answer any other questions.
> > >
> > > Cathy
> >
> > Hi Cathy:
> >
> > The rib file does not store the missing values, so each library chooses
> > that representation. The netcdf-java library uses NaNs to indicate missing
> > values. This currently cannot be changed. What problem are you having with
> > portability of NaNs?
> >
> > Ethan
> >
> 
> NCL does not read NaN correctly. So, for all reads, I need to have in
> addition
> 
> replace_ieeenan (hgt, value, 0)
> address@hidden
> address@hidden
> 
> 
> If you use an IEEE infinity as the value of the _FillValue attribute for a
> variable, the *ncdump* program will output _ for ordinary values of the
> variable, as if they had not been written.
> 
> Using an IEEE infinity or NaN for a fill value is not recommended, because
> the resulting file may not be interpreted correctly on platforms that don't
> use the IEEE 754 Standard for floating-point. If you use non-normal numbers
> for fill values, a fix is to apply this
> patch<http://www.unidata.ucar.edu/software/netcdf/patches/vardata-patch>
> to
> the file ncdump/vardata.c and rebuild *ncdump*. If you don't have the IEEE
> finite() function used in this patch in libm, you can try compiling
> ncdump/vardata.c with a -DNO_FINITE flag and with compiler flags that
> support IEEE comparisons (e.g. NaN != NaN).
> 
> 
> >
> > >
> > > On 9/28/11 10:31 AM, Unidata THREDDS Support wrote:
> > > > Hi Cathy,
> > > >
> > > > How are you getting the data? Are you using the NetCDF Subset Service
> > (NCSS) access method to request the dataset?
> > > >
> > > > There is no way to specify that you would like non-NaN missing values
> > when you request a netCDF file from the TDS. I suspect it isn't a simple
> > matter in this case as it has to do with how the GRIB files are being
> > converted into netCDF.
> > > >
> > > > However, I'm going to ask John Caron to give a more definitive answer.
> > Unfortunately he won't be back from vacation for about two weeks.
> > > >
> > > > Ethan
> > > >
> > > > Cathy Smith wrote:
> > > >> I am reading CFSR data from NOMADS and missing values are being
> > returned
> > > >> as NaN (for example, for  ocean variables over land) from the TDS.
> > This
> > > >> causes problems for me in NCL and likely would in other languages.
> >  Can
> > > >> missing values be returned as something more portable such as
> > 9.9692e+36
> > > >> as is the default in C?
> > > >> Thanks,
> > > >> Cathy Smith
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: TKC-172853
> > > > Department: Support THREDDS
> > > > Priority: High
> > > > Status: Open
> > > >
> > >
> > > --
> > > ----------------------------------------------
> > > NOAA/ESRL PSD and CIRES CDC
> > > 303-497-6263
> > > http://www.esrl.noaa.gov/psd/people/cathy.smith/
> > >
> > > Emails about data/webpages may get quicker responses from emailing
> > > address@hidden
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: TKC-172853
> > Department: Support THREDDS
> > Priority: Critical
> > Status: Open
> >
> >
> 
> 


Ticket Details
===================
Ticket ID: TKC-172853
Department: Support THREDDS
Priority: Critical
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.