Re: NDBC buoy data aggregation

• To: Joe McLean <joe.mclean@xxxxxxxx>
• Subject: Re: NDBC buoy data aggregation
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Wed, 24 Sep 2003 17:54:32 -0600

Joe McLean wrote:

Hi John,

Attached is an NcML aggregation file which is a good example of  NDBC time
series data which has missing values.
The netcdf elements with missing data are below.

  <!-- 2002-10 has 0 hrs = 0 days -->
  <netcdf
uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2002_10_stdmet.nc'/>
MISSING 744 of 744 hrs
I could create a dummy file using all missing_value values and plug it in place
of this uri.

  <!-- 2003-04 has 184 hrs = 7.66666666666667 days -->
  <netcdf
uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2003_04_stdmet.nc'/>
MISSING 536 of 720 hrs: 'Apr-08 16:00' thru 'Apr-30 23:00'

  <!-- 2003-05 has 414 hrs = 17.25 days -->
  <netcdf
uri='dods://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/burl1_2003_05_stdmet.nc'/>
MISSING 330 of 744 hrs: 'May-01 00:00` thru 'May-14 17:00'

The last two are missing data on either end of the expected time axis.  In the
previous example (email 22SEP 16:50), the missing data was in the middle of the
expected time axis.  I'll leave this to you to see how one could use NCML to
aggregate such files, but using agg type 'joinNew' and adding a range attribute
to the netcdf element (then, when the missing data is in the middle of the time
axis, the uri could be listed twice) might be an idea.  Then there is the whole
issue of irregularly stepped time axes - I realize with NCML aggregations, one
can hard code the coordinate values, but these aggregations are 17,000+ time
steps - what issues would be involved then?.

Joe

get a new version of the AS:

ftp://ftp.unidata.ucar.edu/pub/thredds/dodsCupdate.war

and try this in your NcML, instead of the <values> element:

 <variable name='time' type='double' shape='time'>
   <attribute name='long_name' type='string' value='time coordinate'/>
   <attribute name='units' type='string' value='hours since 2003-08-19'/>
   <units convertTo="hours since 2003-08-19" />
 </variable>

where you change your units (in both places) to whatever you want.

let me know how that works.