Re: NetCDF conventions for mobile stations



Janine Goldstein wrote:
John,

Thanks for the link. I wasn't aware you had added conventions for observation data. I don't see this page linked off the main "NetCD conventions" page. I am glad you are working on them!

I *am* planning to put all of these variations into a single file. We create "composite" datasets, and advertise them to the users as having all the date in the same format in a single file.

In that case, you'll be forging new ground.

You can put multiple coordinate systems in the same file, so I would approach it that way; a file that contains multiple sets of data, each with its own coordinate system.
Do your observations have multiple parameters (temperature, pressure, etc) at 
the same time and point? If so, you want to group these together.

For station data:

 id(station)
 lat(station)
 lon(station)
 alt(station)
 firstChild(station) // index into stationObs
 numChildred(station)

 time(stationObs)
 param1(stationObs)
 param2(stationObs)
 ...

For trajectory data:

 id(trajectory)
 firstChild(trajectory) // index into trajectoryObs
 numChildred(trajectory)

 lat(trajectoryObs)
 lon(trajectoryObs)
 alt(trajectoryObs)
 time(trajectoryObs)
 param3(trajectoryObs)
 param4(trajectoryObs)

For station data that occasionally moves, perhaps you should just break it up into 
seperate "stations" whenever it moves. You can add another variable that stays 
constant across the move:

 id(station)
 name(station); // may be the same for different "stations" indicating that the 
station moved.



I read the new conventions and I see two options.

1) Use the multiple trajectory format, for true trajectories make lat(time), and for fixed location stations make lat a scalar variable so that the scalar value will be applied to all observations.

Would that meet the convention? Do you foresee any problems doing it this way?


2) Use the station observation format, and break trajectories into a series of individual stations (i.e. for a 100 observation long trajectory I would define 100 stations each with a unique lat/lon and numChildren=1 (i.e. one report per station).

This option is less optimal because you loose the information that those 100 stations are actually a trajectory. But would it work otherwise? Do you foresee any problems?

Which do you think is the better way to go, if either?

Thanks!

Janine


John Caron wrote:
Hi Janine:

Im unclear if you plan on putting all of these variations in the same file? My first choice might be to put them in multiple files. The formats that you seem to have:

1) station data
2) track data
3) station data that occasionally moves.

Our format for this kind of data is here:

http://www.unidata.ucar.edu/software/netcdf-java/formats/UnidataObsConvention.html

which covers 1 and 2. we havent dealt with 3 before, but we can come up with some variant. these are not part of the CF spec, but we plan to submit them.

perhaps you should look at the above spec, then we can talk some more.

Janine Goldstein wrote:
Hi John,

I am over at EOL and am wondering if you are the right person to help me. I am developing a netCDF file to contain data from irregularly spaced surface stations. Most of these are fixed with respect to time (i.e. ASOS stations, etc) but some move continuously with time (ships) or move intermittently, maybe even just once (as in stations that are relocated but keep the same station id, or truck mounted stations used during field deployments which are stationary for a few hours to a few days, and then move and are stationary again.

I am wondering how best to implement this within a CF compliant netCDF file. For fixed stations, I currently have:

parameter(time, station)
time(time)
lat(station)
lon(station)

I can change this to:

lat(time, station)
lon(time, station)

but less that 1% of the stations are mobile, so this seems like a huge waste of space. For example, 2 months worth of 5 minute data = 17280 repetitions of the lat/lon for each fixed station, compared to one instance of lat/lon in my current model, and I have hundreds of stations per file.

Do you have any suggestions as to how to implement this? Can you point me at any resources that try to tackle this problem?

I just wanted to check before I reinvent the wheel (-:

Thanks so much,

Janine Goldstein
NCAR/EOL