[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Point data CDF format

Hi Bruce:

For station data, use the "contiguous list" option if you can. Its quite
efficient and allows variable number of obs per station. If you only
have one station, you can just call if a "point observation collection"
and you dont need any linked list at all.

Bruce Flynn wrote:

Hi John,

I did look at the Unidata Obs Convention and also what was available
for the CF convention.  Looking at the 2 different conventions was
confusing me, but now I think I have a better understanding of what I
was reading.

It looks like the Unidata standard says that you have to use a linked
list of some sort.  Because my main goal is to make the data easy to
use for scientists.  That being said, it would be really nice to be
able to use the data in IDV, but I would rather not add the extra
confusion of the linked list.  Is there a way to make the data useful
in IDV without using a linked list data structure?  The Unidata
standard seems to suggest that there isn't.

Thanks
Bruce


On Dec 6, 2007, at 1:59 PM, John Caron wrote:


Hi Bruce:

CF "point obs" conventions are incomplete, and we dont have an
adapter for them. as Don probably mentionaed, we are working on an
improved version that we will submit to CF soon. in the meantime, if
you need to get working with the IDV now, you should use the
conventions at



http://www.unidata.ucar.edu/software/netcdf-java/formats/UnidataObsConvention.html




Don Murray wrote:

Hi Bruce-

I'm going to pass this off to John Caron who knows the convention
better than I do. I don't think you want CF 1.0 as the convention
and I don't see that you have the  relation between the station
and the record.

John?

Don


Bruce Flynn wrote:

Don,

I now have a file that my CF validator says is valid and has the
coordinate variable time.  I tried to open it in IDV with the
datatype of netCDF Point Data and it does not seem to work.  If I
leave the file chooser at "I'm feeling lucky" and select "netCDF
Point Data" it fails silently.  If I select "netCDF Point Data"
from the start I get the following exception:

ucar.unidata.data.BadDataException: Unable to make a
PointObsDataset from
/Users/brucef/dev/cvs/TOOLS/dev/rig/python/db2nc/test.nc
Error = **Failed to find Datatype Factory for=
/Users/brucef/dev/cvs/TOOLS/dev/rig/python/db2nc/test.nc datatype=
Station

        at


ucar.unidata.data.point.NetcdfPointDataSource.doMakeDataset(NetcdfPointDataSource.java:247)


        at


ucar.unidata.data.point.NetcdfPointDataSource.getDataset(NetcdfPointDataSource.java:212)


        at


ucar.unidata.data.point.NetcdfPointDataSource.initAfterCreation(NetcdfPointDataSource.java:199)


        at
ucar.unidata.data.DataManager.createDataSource(DataManager.java:1222)
        at


ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1829)


        at


ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1748)


        at


ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1682)


        at


ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1663)


        at
ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:830)

        at


ucar.unidata.idv.chooser.FileChooser.selectFilesInner(FileChooser.java:631)


        at
ucar.unidata.idv.chooser.FileChooser.selectFiles(FileChooser.java:500)
        at
ucar.unidata.idv.chooser.FileChooser.doLoadInThread(FileChooser.java:522)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at


sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)


        at


sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)


        at java.lang.reflect.Method.invoke(Method.java:585)
        at ucar.unidata.util.Misc$2.run(Misc.java:983)
        at ucar.unidata.util.Misc$3.run(Misc.java:1011)

I can't really tell from the exception if it's a problem with my
data format or if it's a bug in the code.

Here's the latest NetCDF Header for my file, and I've also placed
the file at ftp://ftp.ssec.wisc.edu/pub/incoming/test.nc if you
want to take a look at it.

netcdf test {
dimensions:
        time = UNLIMITED ; // (14851 currently)
        station = 1 ;
variables:
        double time(time) ;
                time:long_name = "Record epoch time offset" ;
                time:units = "seconds since 1970-1-1 0:00:00 0:00"
;tation as calculated." ;
                ACCURAIN:missing_value = -9999. ;
        double temperature(time) ;
                temperature:resolution = 5 ;
                temperature:units = "degC" ;
                temperature:standard_name = "air_temperature" ;
                temperature:description = "Outside temperature
reading." ;
                temperature:missing_value = -9999. ;
        double rh(time) ;
                rh:resolution = 5 ;
                rh:units = "%" ;
                rh:standard_name = "relative_humidity" ;
                rh:description = "Outside relative humidity
reading." ;
                rh:missing_value = -9999. ;
        double wspd(time) ;
                wspd:resolution = 5 ;
                wspd:units = "m/s" ;
                wspd:standard_name = "wind_speed" ;
                wspd:description = "RM Young Wind Monitor speed
reading." ;
                wspd:missing_value = -9999. ;
        double wdir(time) ;
                wdir:resolution = 5 ;
                wdir:units = "degrees" ;
                wdir:standard_name = "wind_from_direction" ;
                wdir:description = "RM Young Wind Monitor direction
reading." ;
                wdir:missing_value = -9999. ;
        double elevation(station) ;
                elevation:units = "meters" ;
                elevation:description = "Height of the base of the
RIG Tower" ;
        double latitude(station) ;
                latitude:units = "degrees_east" ;
                latitude:description = "Latitiude of the RIG Tower" ;
        double longitude(station) ;
                longitude:units = "degrees_north" ;
                longitude:description = "Longitude of the RIG Tower" ;

// global attributes:
                :convention = "CF-1.0" ;
                :description = "Meteorological Station Observations
from the tower on the top of\n",
                        "the UW SSEC/AOS building, commonly referd
to as the Rooftop Instrument Group (RIG)." ;
                :contact = "University of Wisconsin\n",
                        "Space Science and Engineering Center\n",
                        "1225 W. Dayton St.\n",
                        "Madison, WI 53706\n",
                        "http://www.ssec.wisc.edu"; ;
                :url = "http://rig.ssec.wisc.edu"; ;
}
On Dec 4, 2007, at 10:45 AM, Don Murray wrote:


Hi Bruce-

I guess the docs have been updated as we get ready to move to
Netcdf 4.  However, the CDL at:

http://www.unidata.ucar.edu/software/netcdf-java/formats/metar.cdl.txt


doesn't seem to use structures but adheres to the convention.  I'm
cc'ing John Caron who is responsible for this convention for his
input.

Don

Bruce Flynn wrote:

Don,
Most of the record formats listed involve NetCDF Structures.  I
think this is obviously the best way to go for efficiency, but it
seems there are not a lot of APIs that support reading
Structures.  I was planing on using the "pseudo-Structure" model
so the scientists can use what ever API they want.  It seems to
want a station dimension though.  Is this really necessary if you
only have a single station?
Bruce
On Dec 3, 2007, at 3:44 PM, Don Murray wrote:







--
*************************************************************
Don Murray                               UCAR Unidata Program
address@hidden                        P.O. Box 3000
(303) 497-8628                              Boulder, CO 80307
http://www.unidata.ucar.edu/staff/donm
*************************************************************