[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Fwd: NDBC NetCDF-4 Files



Hi Rich, Walt:

On 2/16/2011 6:56 AM, Rich Signell wrote:
John,
I'm down here at NOAA's National Data Buoy Center (NDBC) and could
really use some advice from you about how to handle their NetCDF4 data
in groups.  They really like the groups, and they are working
internationally with the OceanSites and Ifremer folks, who also like
this group structure.

1) Could they keep their data in groups, but then write an IOSP so
that it would appear to TDS as a group of datasets, which each could
be CDM compliant?
Or would you recommend splitting them up and writing valid CDM netcdf
files, and keeping the information about the group some other way.
you wouldnt have to write an IOSP, just a CoordSysBuilder.
2) Is it even a good idea to store using groups?  It seems there could
be advantages to storing at a more atomic level, to enable more
database or metadata searching capabilities.

this is not CF compliant, and we recommend CF compliance for long-term interoperability.

OTOH, its fairly easy to get data into the CDM/IDV, as long as the coordinate information is present. A quick look at this doesnt raise any alarm bells so far.

If you are creating your own conventions, you do need to document it, send us a link to the document, and give it a unique name in the Convention attribute.

So it really depends on who you want to use the data, and what they need.

Do you have time to talk today at all?  Pick a time.

sorry i missed your window I guess

Thanks,
-Rich

On Thu, Feb 10, 2011 at 1:29 PM, John Caron<address@hidden>  wrote:
Hi Rich, Walt:

These actually have groups, not structures.

Im not against using netcdf-4 data model, but its new territory for CF.
Currently, these files are not readable by the CDM poinmt featire layer (of
course,CDM canread the underlying data just fine, it just doesnt understand
what should be done with it).

At minimum, if you are going your own way with conventions, we want you to
document them and give them a name and put that name in the "Conventions"
global attribute.

John

On 2/9/2011 7:45 PM, Rich Signell wrote:
Walt,

I'm not sure this style of NetCDF4 (with structures) is a good choice
for archiving NDBC buoy data.   Although there is nothing wrong with
using NetCDF4, there are not very many applications that can deal with
NetCDF4 structures.  Unidata and the CF community recommend storing
time series data using the "Point" featureType, described here:

https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventionsJMG

If you put the attached file you sent on the latest THREDDS

http://geoport.whoi.edu/thredds/catalog/usgs/data1/rsignell/data/ndbc/catalog.html?dataset=usgs/data1/rsignell/data/ndbc/NDBC_42040_201101_D1116_v00.nc

The OpenDAP access form appears:

http://geoport.whoi.edu/thredds/dodsC/usgs/data1/rsignell/data/ndbc/NDBC_42040_201101_D1116_v00.nc.html

but when you try to load the OPeNDAP URL in ToolsUI, you get:
"Variable name (wind speed) must be unique within group"

We can talk more about this when I visit NDBC next week (or more
tomorrow, if you wish).

I could take a stab at creating a valid Point featureType NetCDF 4
file from this data if it would be helpful.

-Rich

---------- Forwarded message ----------
From: Walt McCall<address@hidden>
Date: Wed, Feb 9, 2011 at 5:05 PM
Subject: NDBC NetCDF-4 Files
To: Rich Signell<address@hidden>


Rich,

Attached is our NetCDF-4 file that we are about to start using for
archiving all buoy data. Would you mind taking a look at it and seeing
if there is anything that seems non-standard? Also, do you know if the
new THREDDS 4.2 server will handle NetCDF-4? We tried THREDDS 4.1, but
found that it didn't handle our files well; so we are installing 4.2
and plan on giving that a try tomorrow.

If there is a problem with THREDDS, maybe we can work on it Tuesday in
addition to the enhancements.


Thanks,
Walt