[thredds] oceanotron, in-situ profiles through OPeNDAP interface

Dear Roy, dear all,

We've just published our in-situ data server at Ifremer. It is fully compliant with what has been done with DAPPER (and Dchart can request our interface). It disseminates vertical profiles datasets and soon it will disseminate pointSeries (e.g. mooring datasets) and trajectories (e.g. drifting buoys or ferry box).
The URL  is : http://www.ifremer.fr/oceanotron/OPENDAP/

The request-able criteria are of course, x,y,z,t coordinates and id, but also feature's metadata (platform code, institution) and phenomenons values (e.g. temperatures). A session/pagination system is also implemented (with OPeNDAP indices) to get big data subsets results with several requests each of them not exceeding a quota.

The software is open-source and can be extended with modules to read datasets (storage units), to transform datasets (transform unit) and disseminate the datasets (front-desk). The communication between these modules, especially observations features, is standardized with Value Objects (their description is available at : http://www.ifremer.fr/isi/oceanotron/umlVOReport/oceanotron_uml.html)

You can get the software documentation and source from our wiki : https://gforge.ifremer.fr/wiki/doku.php?id=oceanotron:start

If you have any question or comment, please do not hesitate to send it.

Best regards,

Thomas

On 12/06/2010 05:53 PM, Roy Mendelssohn wrote:
Agreed.  I said similar things at the meeting. But I was starting from the 
discussions at the meeting, which was the CDM as implemented in THREDDS.  So I 
want to get clear in my mind what TDS will be able to do for observational data 
first before looking for alternatives.

-Roy

On Dec 2, 2010, at 2:42 PM, Upendra Dadi wrote:

Roy,
With regard WODB, the users want to be able to search not just using physical 
coordinates like lat-lon bounding box, but also using other attributes like 
institution, project, platform etc which are usually not represented using 
coordinate variables. So grouping the data based on physical coordinates might 
cause performance issues when searching using other criteria. Would it be 
possible to have some additional structures, something like a database index, 
but without a full-fledge DBMS? Does TDS allow such structures?

Upendra


Roy Mendelssohn wrote:
Hi John:

Since the UAF meeting in Seattle I have been giving some thought about how to 
serve some large, important datasets, such as the raw ICOADS observations or 
the WODB observations. While reading over the PointObservation Conventions 
proposal on the CF site, while the proposal makes it clear how I might put data 
into a netcdf file, it doesn't make clear what the interplay might be with a 
service in TDS, and how a possible service might be affected by a very large 
dataset without further structure.

So it seems pretty clear that the ICOADS would be points.  From the example:

dimensions:
  obs = 1234 ;

variables:
  double time(obs) ;
    time:long_name = "time of measurement" ;
    time:units = "days since 1970-01-01 00:00:00" ;
  float lon(obs) ;
    lon:long_name = "longitude of the observation";
    lon:units = "degrees_east";
  float lat(obs) ;
    lat:long_name = "latitude of the observation" ;
    lat:units = "degrees_north" ;
  float alt(obs) ;
    alt:long_name = "vertical distance above the surface" ;
    alt:standard_name = "height" ;
    alt:units = "m";
    alt:positive = "up";
    alt:axis = "Z";

  float humidity(obs) ;
    humidity:long_name = "specific humidity" ;
    humidity:coordinates = "time lat lon alt" ;
  float temp(obs) ;
    temp:long_name = "temperature" ;
    temp:units = "Celsius" ;
    temp:coordinates = "time lat lon alt" ;

attributes:
  :CF\:featureType = "point";
Now I am assuming that in a TDS implementation of a service, I will be able to 
select on the coordinate variables, is that correct?  Even so, for something 
like ICOADS, obs is quite large and that extract could be quite slow unless 
either there is additional structure or the  TDS pre-fetches the coordinate 
variables much as the present Dapper server does.

Other options would be to say have a file for each 10-degree block, and then 
have TDS aggregate over the files - would this be possible.  Then the search 
would a lot faster when people want time series in a region as opposed to more 
synoptic extractions.  Would the TDS service be supporting such an option?  Or, 
as netcdf-4 supports groups, to have 10-degree groups with 2-degree subgroups, 
which would work as far as netcdf-4 is concerned, but that is not the same as 
TDS knowing what to do with the hierarchy or to take advantage of the structure.

My questions for Profiles  (that is for the WODB) are pretty much the same.  I 
assume that the TDS service will be able to search on the coordinate variables, 
is that correct?  And I have the issue with the fact that the profile dimension 
variable will get quite large and without further structure the search could be 
very slow.  Adding the same types of structures mentioned above would provide 
possible solutions, but only if TDS, as opposed to netcdf4, supported them.

As you may have guessed, these are not theoretical questions - I would really 
like to see ICOADS and WODB served as part of the year 2 UAF effort.  So now is 
a good time to start thinking about how to do it correctly and what the service 
will be able to do.

Thoughts?

Thanks,

-Roy







**********************
"The contents of this message do not reflect any position of the U.S. Government or 
NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail:Roy.Mendelssohn@xxxxxxxx  <mailto:Roy.Mendelssohn@xxxxxxxx>  (Note new 
e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www:http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
------------------------------------------------------------------------

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  
visit:http://www.unidata.ucar.edu/mailing_lists/
**********************
"The contents of this message do not reflect any position of the U.S. Government or 
NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail:Roy.Mendelssohn@xxxxxxxx  (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www:http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  
visit:http://www.unidata.ucar.edu/mailing_lists/

--


-------------------------------------------------------------
Thomas LOUBRIEU
IFREMER IDM/ISI
BP70
29280 Plouzane
FRANCE

email:Thomas.Loubrieu@xxxxxxxxxx
WWW  :http://www.coriolis.eu.org/cdc
Tel.:  (+33) (0)2 98 22 48 53
Fax:   (+33) (0)2 98 22 46 44

-------------------------------------------------------------




  • 2010 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: