[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: wrestling with new server



I have quoted the full message below for context. Basically, it's a pretty basic requirement for use that I can access resources not on the local disk. This is because I have a number of data resources which do not exist as files, or even as single entities within a database. Rather, they are the results of database queries, where that database is an object-oriented database whose native storage formats are not supported by OpenDAP. Furthermore, we have other sources of data which come in by unusual means.

To address the problem without writing code which is tightly coupled with the current THREDDS implementation, I have written a number of cgi script interpreters which take an identifying string (path + record number) and perform a request-and-conversion, bringing the results back as NetCDF, and feeding them in to OpenDAP.

The previous version of THREDDS worked fine using this method, and in a limited way, the new one does too.

Using the the service named below, let's look at two simple cases:

<service name="ncmars" serviceType="OpenDAP" base="/thredds/dodsC/">
<datasetRoot path="ncmars" dirLocation="http://linux-dev.bom.gov.au:8040/mars/"/>
 </service>

This works:

<dataset name="Working NC" ID="WNC" serviceName="ncmars" urlPath="ncmars/start_request/300/end_request/300.nc"/>

While the following does not :

<dataset name="3D Ocean Analysis" path="ncmars">
   <metadata inherited="true">
     <serviceName>ncmars</serviceName>
   </metadata>

<dataset name="31-08-2005" id="3DOCEAN_0" urlPath="ncmars/start_request/300/end_request/300.nc"/> <dataset name="31-09-2005" id="3DOCEAN_1" urlPath="ncmars/start_request/301/end_request/301.nc"/>
</dataset>

We have a situation where, for whatever reason, the nested datasets aren't working with an HTTP resourced service.

Java's URL class should be able to interpret file:///whatever just as easily as http://whatever, so I see no hard-and-fast reason why http should be a problem.

I would be happy enough to pursue another method of linking new data sources into THREDDS, but my current approach of doing the data conversion outside of thredds means that (a) those scripts can be re-used for applications other than OpenDAP, and (b) I'm decoupled from code maintainance as new versions of THREDDS mature and change...

Cheers,
-T


Ethan Davis wrote:

Hi Tennessee,

Tennessee Leeuwenburg wrote:

Hi John,

v3.2 installed. Apart from the problems with the debug interface, all going reasonably well.

I was able to publish a specific file, both from disk and accessed via HTTP. I'm still having trouble creating a dataset of these individual files however, even with your suggestions. If you look at the catalog below, everything works except for ids 3DOCEAN_0 and 3DOCEAN_1.
There must be something small I'm missing...

Cheers,
-T

<?xml version="1.0" encoding="UTF-8"?>
<catalog name="THREDDS Server Default Catalog : You must change this to fit your server!" xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";
       xmlns:xlink="http://www.w3.org/1999/xlink";>

 <service name="thisDODS" serviceType="OpenDAP" base="/thredds/dodsC/">
   <datasetRoot path="test" dirLocation="content/dodsC/data/"/>
 </service>

 <service name="footoo" serviceType="OpenDAP" base="/thredds/dodsC/">
   <datasetRoot path="data" dirLocation="/data/pymars/"/>
 </service>

 <service name="ncmars" serviceType="OpenDAP" base="/thredds/dodsC/">
<datasetRoot path="ncmars" dirLocation="http://linux-dev.bom.gov.au:8040/mars/"/>
 </service>


I don't believe this will work. The datasetScan and datasetRoot does not yet support remote data file access.


<dataset name="AVHRR Sample 0" ID="AVHRR0" serviceName="footoo" urlPath="data/bufr_conversion_samples/AVHRR_SST_v0.nc"/> <dataset name="AVHRR Sample 1" ID="AVHRR1" serviceName="footoo" urlPath="data/bufr_conversion_samples/AVHRR_SST_v1.nc"/>


<dataset name="3D Ocean Analysis" ID="3DOCEAN" serviceName="ncmars" path="ncmars"> <dataset name="31-08-2005" id="3DOCEAN_0" urlPath="ncmars/start_request/300/end_request/300.nc"/> <dataset name="31-08-2005" id="3DOCEAN_1" urlPath="start_request/300/end_request/300.nc"/>
 </dataset>


Try instead of the above four lines:

<dataset name="3D Ocean Analysis" ID="3DOCEAN">
  <metadata inherited="true">
     <serviceName>ncmars</serviceName>
  </metadata>
<dataset name="31-08-2005" id="3DOCEAN_0" urlPath="ncmars/start_request/300/end_request/300.nc"/> <dataset name="31-08-2005" id="3DOCEAN_1" urlPath="ncmars/start_request/300/end_request/300.nc"/>
</dataset>

Are these supposed to be two different files? The urlPaths are the same except the second one was missing the "ncmars/" that the first one starts with. (I added it in my XML above but now they refer to the same file.)

Though, again, I don't think this will work because the datasetRoot doesn't point to a local disk.

Ethan




 <datasetScan name="Data" ID="rootData"
              path="data" dirLocation="/data/pymars/" filter=".*\.nc$">

   <metadata inherited="true">
     <serviceName>thisDODS</serviceName>
   </metadata>

 </datasetScan>
</catalog>