THREDDS API Question

[Ethan Davis]

Hi Nathan,

Can you explain the context for these questions. This is on the server 
side (in Hyrax)?

Nathan Potter wrote:
> Greetings,
>
> So I am using the THREDDS API in an attempt to get the <property> 
> elements for a dataset. I've run into a couple of (possibly related) 
> problems.

Just to clarify our terminology. When you say "THREDDS API" you mean 
both the thredds.catalog and thredds.servlet packages? I generally split 
those apart and call the thredds.catalog package the "THREDDS Catalog 
API" and call the thredds.servlet package the "THREDDS Servlet 
Framework" (TSF).

 [Note: the TSF is probably only useful for those writing servers.]

> ** 1) I can't get the dataset information without searching.
>
> In the HttpServletRequest I have the URL for the dataset, say:
>
> http://localhost:8080/opendap/wcs/MODIS/Grid/test.hdf.html

Is this URL for an OPeNDAP HTML response?

Are you trying to get the property from the THREDDS catalog so you can 
use it in the OPeNDAP response?

> In order for me to get THREDDS to divulge the <property> elements for 
> the dataset I have to:
>
> - take the dataset name "wcs/MODIS/Grid/test.hdf.html" and back track 
> to the
>   collection name, "wcs/MODIS/Grid/".
> - ask the DataRootHandler for the InvCatalog for "wcs/MODIS/Grid/"
> - Ask the InvCatalog for the InvDataset for "wcs/MODIS/Grid/"
> - Search the child datasets of the "wcs/MODIS/Grid/" InvDataset for the
>   one whose name (lexically) matches "wcs/MODIS/Grid/test.hdf.set"
> - Read the properties of that InvDataset
>
> That seems awfully complex. (Of course there may a more straight 
> forward way that I am not aware of.)

That is about as simple as it gets. Though I would suggest you make sure 
the THREDDS configuration (TSF) knows about this dataset first by 
getting the CrawlableDataset that matches the dataset URL:
       DataRootHandler.getCrawlableDataset("wcs/MODIS/Grid/test.hdf")
       // I dropped of the trailing ".html" assuming it was the OPeNDAP 
dataset URL extension

Are you using InvDataset.findDatasetByName( String name) to find the 
child dataset?

Also, depending on how you setup your dataset IDs, you could ask the 
catalog to find the dataset by ID, like

       cat.findDatasetByID( "wcs/MODIS/Grid/test.hdf")


> ** 2) When I ask for a catalog I have to know the name of the XML file 
> in which it resides.
>
> In the above example, when I ask the DataRootHandler for the 
> InvCatalog I ask for: " wcs/MODIS/Grid/catalog.xml" Which is all well 
> and good if all of the catalogs are stored in files called 
> catalog.xml. Essentially this means that anyone configuring a THREDDS 
> catalog has to create a hierarchy of directories that mimics the 
> organizatiopn of the collections, and all of the THREDDS information 
> must be stored in files called "catalog.xml".

Why do you need to create this hierarchy of directories mimicking the 
data collection hierarchy? The TSF should keep track of your config 
catalogs and the automatically generated catalogs.

> THREDDS does not actually require this - I can make a complex 
> hierarchy of collections by using either a single (complex) top level 
> catalog.xml file, or a collection of XML files in a single directory 
> that employ <catalogRef> elements to create their organizations.
> However the API breaks down in both cases.
>
> If the catalog is composed of a collection of XML files in a single 
> directory that employ <catalogRef> elements to create their 
> organizations, then in order to retrieve catalog information I would 
> have to KNOW how the information was organized (file names, directory 
> hierarchy , etc.) But I don't know - since the catalog may be created 
> by a user after compile time (although THREDDS does know this since it 
> parsed all of the catalog information at start up) - and I shouldn't 
> have to know. For me to know would require that I parse the top level 
> catalog.xml file and build the XML doc tree myself. At which point it 
> I can get the elusive <property> elements from the XML doc in memory.
>
> If the catalog is composed of a single (complex) top level catalog.xml 
> file then I would have to know that and just ask for the top level 
> catalog.
>
> (Searching the entire catalog from the top down for my dataset doesn't 
> seem to work either...)

I'm sorry, I'm having a hard time following here. What are you trying to 
do and why?

Is the problem that you may not know if the dataset is contained in a 
catalog generated because of a datasetScan element or contained directly 
in one of the THREDDS config catalogs?

> All of these methods of writing and organizing catalogs are legitimate 
> in THREDDS, and users writing THREDDS catalogs would likely employ one 
> or more of these methods when writing their catalogs.
>
>
> I propose that the THREDDS API be extended so that one can simply ask 
> the DataRootHandler for an InvDataset or an InvCatalog. Like:
>
>     InvDataset id = drh.getDataSet("wcs/MODIS/foo.nc");
>     InvCatalog id = drh.getCatalog("wcs/MODIS/");
>
> or possible the InvDataset that represents a collection:
>
>     InvDataset id = drh.getDataSet("wcs/MODIS/");
>
>
> If the DataRootHandler doesn't have it, return null.
>
>
> Is that unreasonable?

I'll have to take a closer look at this.

Ethan

>
> Nathan
>
>
> = 
> Nathan Potter                        ndp at opendap.org
> OPeNDAP, Inc.                        541.752.1852
>
>
>
>
> =============================================================================== 
>
> To unsubscribe thredds, visit:
> http://www.unidata.ucar.edu/mailing-list-delete-form.html
> =============================================================================== 
>

-- 
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis at ucar.edu
P.O. Box 3000
Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
---------------------------------------------------------------------------


==============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================