[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Followup and questions from Boulder

Hi Roy:

Roy Mendelssohn wrote:
Hi Guys:

I am slowly catching up on things from all my travels (boy do I love traveling?!?!?). I just want to reiterate several points I raised, and also have several questions that have arisen since returning and talking with my group.

Points we discussed:

1. We are very interested in the WCS services provided by TDS, but presently almost all of the data that we serve using the TDS are aggregated. Aggregated data can not be served by the WCS in the present version of the TDS - having that capability would be really nice and would warm Ben's heart as we could add all of these data to the Galeon list of served data.

The latest version of TDS should now allow WCS serving of aggregation data. If you want to try our "release candidate", I would be grateful for any feedback on it. We wont do an official release until (probably) next week .

Its at ftp://ftp.unidata.ucar.edu/pub/thredds/3.9/thredds.war.

2. The datasetscan program for automatic generation of catalogs does not read the attributes in the netcdf files and spit it out as part of the catalog. This can make life very difficult if you have a lot of data. I would prefer if there is a question that it read in and spit out all attributes, including global (as we have the ADD metadata there in our files) as it is a lot easier to delete info in a file if not wanted than to enter it by hand.

Its difficult to figure out automatically where attributes should go in a catalog. We have considered functionality where you indicate in the catalog where and which file attributes should be placed into it. Is that what you have in mind? Eg:

<documentation type="rights">address@hidden</documentation>

would substitute the value of the global attribute "rights" in the file for the 
token address@hidden

If thats the kind of thing you are thinking about, it would help us a lot to 
work up a concrete example of what data is in your file, and where it should be 
placed in the catalog.

New questions:

1. Can the TDS aggregate over remotely served OPeNDAP data, or just on local data. If it can not do this, are there plans to incorporate that capability?

Yes, TDS can aggregate over remore data. The main limitation is that the NcML scan element can only scan local directories, so you have to specify the remote datasets yourself, for example:

(2)<aggregation dimName="time" type="joinNew">
   <variableAgg name="T"/>
   <netcdf location="http://server/dods/data/ncml/nc/time0.nc"; />
   <netcdf location="http://server/dods/data/ncml/nc/time1.nc"; />
   <netcdf location="http://server/dods/data/ncml/nc/time2.nc"; />

2. Can the THREDDS catalogs work in a distributed, hierarchical manner, so that we can have a master catalog and distributed local catalogs, and the master catalog can automatically in some fashion update itself with the information in the local catalogs? If this capability does not exist, are their plans for such a capability or has any one else developed this?

I wonder if the "catalog reference" functionality wouldnt do what you want? For example:

<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0"; xmlns:xlink="http://www.w3.org/1999/xlink"; name="Unit Test datasets" version="1.0.1">
<catalogRef xlink:href="enhancedCatalog.xml" xlink:title="Test Enhanced Catalog" name="" />
<catalogRef xlink:href="test/catalog.xml" xlink:title="Test Nested Catalog" name="" />
<catalogRef xlink:href="wcsExample/catalog.xml" xlink:title="Test WCS Catalog" name="" />


The catalogRefs point to other catalogs (reletive or absolute URLS).

Thanks for the information. It was good seeing you guys in Boulder.

good to see you, hope to see you again soon.

-Roy M.

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.