[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: FMRC and NaN



On 4/9/2010 9:34 AM, Kevin O'Brien wrote:

Hi John -

John Caron wrote: 4/7/2010 3:05 PM, Kevin O'Brien wrote:
Hi John -

I've been working lately with Rich Signell  to add a bunch of IOOS Forecast Model Run Collections to the UAF THREDDS server at
   http://ferret.pmel.noaa.gov/geoide/geoIDECleanCatalog.html

One thing I've noticed about all of the FMRC that I've run across is that the "best time series" data seems to use NaN as the missing value.  This seems fine, except that the missing value attribute is not defined for the particular variables.  I know this causes a bit of a problem when Ferret looks at the data, and I can imagine it might have ramifications for other clients as well.  There are several ways that I can work around this, and one way I've done it is to wrap the FMRC with some NcML defining the _FillValue attribute as NaN for the variables.    This works, but also means a loss of performance because now the data has to take two hops, from our TDS and the TDS it originally is served from.

Anyway, should it be that by default, the missing_value or _FillValue attribute is defined as NaN in the "best time series" data?

Thanks
Kevin



I think it will use the value in missing_value or _FillValue if defined. If not defined, NaN is apparently the default choice. I suppose the right thing to do is to add that attribute if not defined, for consistency. do you agree?

Hi John -

Thanks for getting back to me.

Actually, in looking a little deeper into the issue after I emailed you,  I think there might be something else going on that I don't understand.

If you look at the best time series at:

    http://geoport.whoi.edu:8081/thredds/dodsC/coawst/fmrc/coawst_2_best.ncd.html

The variable ZETA has these attributes (as seen through the HTML interface):
long_name: "free-surface"
units: "meter"
time: "ocean_time"
field: "free-surface, scalar, series"
_FillValue: 0.0
coordinates: "lon_rho lat_rho ocean_time "

which is consistent with the individual forecast files. 

However, when I actually look at the data:

    http://geoport.whoi.edu:8081/thredds/dodsC/coawst/fmrc/coawst_2_best.ncd.ascii?zeta[0:1:10][0:1:10][0:1:10]

I see the below:
Dataset {
    Float32 zeta[ocean_time = 11][eta_rho = 11][xi_rho = 11];
} coawst/fmrc/coawst_2_best.ncd;
---------------------------------------------
zeta[11][11][11]
[0][0], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][1], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][2], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][3], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][4], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][5], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][6], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][7], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][8], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][9], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[0][10], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[1][0], NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN
[1][1], NaN, -4100.3965, -4096.8623, -4090.378, -4082.1177, -4072.744, -4062.7856, -4052.7595, -4043.087, -4034.1382, -4026.2202
[1][2], NaN, -4112.4053, -4108.7847, -4102.329, -4093.9312, -4084.07, -4073.1624, -4061.7964, -4050.805, -4041.122, -4033.1853
[1][3], NaN, -4129.4663, -4125.637, -4119.091, -4110.316, -4099.464, -4086.7424, -4072.8909, -4059.5364, -4048.5781, -4040.5977
[1][4], NaN, -4147.614, -4143.8057, -4137.275, -4128.0645, -4115.893, -4100.7466, -4083.5476, -4066.868, -4053.7632, -4044.9514
[1][5], NaN, -4165.1304, -4161.61, -4155.5195, -4146.2856, -4133.0703, -4115.5186, -4094.48


  
So, it appears that there are both missing/_FillValues of 0, as well as NaNs scattered through the dataset....Should the NaNs be the same as the _FillValue in this case?

yes, probably. ill have a look at that.