[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #TOG-744630]: Grids from an OPeNDAP server



Hi Valentijn,

> >Yes, NcML can be used this way. I will try to put an example together for 
> >the Pathfinder URL you gave >above and send that to you soon.
> 
> Don made some fixes to the IDV and for the Pathfinder it now seems to work. 
> But can you try to give an example for TOMS instead?
> http://reason.gsfc.nasa.gov/opendap-bin/nph-dods/FTP_DATA/Giovanni/OPS/TOMS/EP/

There are some deeper reasons our code isn't able to open the TOMS data. It has 
to do with how HDF-EOS names variables and/or how the OPeNDAP HDF server 
encodes things. We just don't deal with those things well at the moment. We 
will have to look into this issue some more before we can estimate how much 
time it will take or when we will start working on it.

The Pathfinder data can be read in by the IDV. However, there are some things 
that could be improved by wrapping that data with some NcML. I'm attaching an 
NcML file that points to one of the OPeNDAP Pathfinder datasets. It adds 
"units" and "long_name" attributes as well as changing the sst "add_off" 
attribute to "add_offset". (I played around with a "missing_value" attribute 
but I don't know enough about the data to decide if that is accurate or not.)

As you can see, the NcML contains a pointer to the actual dataset. You will 
need to change the NcML for each dataset you want to look at. NcML can be 
placed in datasetScan elements. However, since you guys are scanning remote 
datasets and accessing them from the remote server (rather than serving the 
data yourselves), the NcML wrapping won't work.

> >> Q2: Can you send me an example catalog that uses the datasetScan
> >> element for the Pathfinder SST OpenDAP server? I'd like to aggregate
> >> over multiple files but i'm not sure if the "JoinNew" / "Join Existing"
> >> element creates one virtual dataset e.g. all 8-day gridded SST's as a
> >> timeseries?
> 
> >I'll have to take a look at that. Are the different datasets over the same 
> >spatial extent?
> 
> Yes, in the case of TOMS and Pathfinder it is over the same spatial extent.

TDS aggregation can scan a local directory to determine all the datasets to 
aggregate. It doesn't know how to scan remote servers (and the underlying code 
doesn't yet use CrawlableDataset, so your work in that realm won't help at the 
moment). However, you could generate the NcML aggregation in some other way 
(maybe even by hand if you weren't aggregating too many datasets). If you did 
this, you could add some snippets of the NcML I attached and fix some of the 
attributes.

Let me know how you want to proceed with this.

Ethan


Ticket Details
===================
Ticket ID: TOG-744630
Department: Support THREDDS
Priority: Normal
Status: Open

Attachment: pathfinderTest.ncml
Description: Binary data