[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: DODS AS

Joe McLean wrote:


Hi John,
I have tried to use an NcML file in the latest Agg Server configuration I downloaded last Friday. I repeatedly receive a java error from the server at http://stout.pmel.noaa.gov:8180/dodsC/ 

This is the pertinent snippet of the error:

java.lang.NullPointerException
at dods.servers.agg.CatalogServlet.addAccess(CatalogServlet.java:611) at dods.servers.agg.CatalogServlet.readCatalog(CatalogServlet.java:257) 
        at dods.servers.agg.CatalogServlet.init(CatalogServlet.java:178)
        at javax.servlet.GenericServlet.init(GenericServlet.java:258)
are you still getting this error? i cant reproduce it. I am seeing this catalog: 

<catalog name="Test NCML Aggregation" version="0.6">
 <dataset name="Top-Level Dataset" dataType="Grid" serviceName="this">
   <service name="this" serviceType="DODS" base=""/>
<service name="stdmet" serviceType="NetCDF" base="file://WEB-INF/dodsAS/"/> 

   <dataset name="NDBC stdmet buoy data">
      <property name="internalService" value="stdmet"/>
      <dataset name="stdmetNcML Agg test" urlPath="TestNcML.xml"/>
   </dataset>
 </dataset>
</catalog>


with this external URL:  http://stout.pmel.noaa.gov:8180/dodsC/TestNcML.xml

and this internal URL: file://WEB-INF/dodsAS/TestNcML.xml
this is a relative URL to the current path, so the absolute path is apparently: 

file:/usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/TestNcML.xml

and i get the error message:

message = "CatalogServlet ERROR opening NcDataset TestNcML.xml
access = file:/usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/TestNcML.xml
Is this the correct path? is not, try using the absolute path in the "stdmet" service element. absolute paths i think are system dependent; on linux try using 3 slashes or 1 slash: 

   <service name="stdmet" serviceType="NetCDF" 
base="file:///path/something/WEB-INF/dodsAS/"/>
   <service name="stdmet" serviceType="NetCDF" 
base="file:/path/something/WEB-INF/dodsAS/"/>
I used your Catalog and NcML aggregation as templates (following this message) and ctreated the following Both are in $tomcat_root/dodsC/WEB-INF/dodsAS/ the web.xml file points to the correct Catalog.xml file. Any ideas? Is it that the NcML netcdf element can't handle a DODS URL? AmI confused about where the aggregation should take place? 

--------------------------------------
AggServTestCatalog.xml
---------------------------------------
<!DOCTYPE catalog SYSTEM
"http://www.unidata.ucar.edu/projects/THREDDS/xml/AggServerCatalog.dtd";>
<catalog name="Test NCML Aggregation" version="0.6"
xmlns="http://www.unidata.ucar.edu/thredds";
xmlns:xlink="http://www.w3.org/1999/xlink";>
        <service name="this" serviceType="DODS" base=""/>
        <service name="stdmet" serviceType="NetCDF"
           base="file://WEB-INF/dodsAS/"/ <file://WEB-INF/dodsAS/>>
        <dataset name="NDBC stdmet buoy data">
            <property name="internalService" value="stdmet"/>
<dataset name="stdmet NcML Agg test" urlPath="TestNcML.xml" /> 
        </dataset>
</catalog>


-
<catalog name="Test NCML Aggregation" version="0.6">
-
<dataset name="Top-Level Dataset" dataType="Grid" serviceName="this">
<service name="this" serviceType="DODS" base=""/>
<service name="stdmet" serviceType="NetCDF" base="file://WEB-INF/dodsAS/"/>
-
<dataset name="NDBC stdmet buoy data">
<property name="internalService" value="stdmet"/>
<dataset name="stdmetNcML Agg test" urlPath="TestNcML.xml"/>
</dataset>
</dataset>
</catalog>



Also tried full path for service:
<service name="stdmet" serviceType="DODS"
base="file:///usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/"/ <file:///usr/local/www/html/TOMCAT_DEMO/las/las_servlet/jakarta/webapps/dodsC/WEB-INF/dodsAS/>> 

Also tried serviceType="NetCDF"
<service name="stdmet" serviceType="NetCDF"

---------------------------------------
TestNcML.xml
---------------------------------------
<?xml version='1.0' encoding='UTF-8'?>
<netcdf xmlns='http://www.ucar.edu/schemas/netcdf'
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation='http://www.ucar.edu/schemas/netcdf
http://www.unidata.ucar.edu/schemas/netcdfDataset2.xsd' >

  <variable name='recnum' type='int' shape='recnum' >
    <attribute name='long_name' type='string' value='time coordinate' />
<attribute name='units' type='string' value='hours since 2001-09-31 00:00:00 UTC' /> 
    <values start='0' increment='1' />
  </variable>

  <aggregation dimName='recnum' type='joinExisting' />
<netcdf uri='http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_09_stdmet.nc'/ <http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_09_stdmet.nc%27/>> <netcdf uri='http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_10_stdmet.nc' /> <netcdf uri='http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_11_stdmet.nc'/ <http://dods.ndbc.noaa.gov/cgi-bin/nph-nc/dods/b42001_2001_11_stdmet.nc%27/>> </netcdf> 
---------------------------------------

Joe
------------------------------------------------
John Caron wrote:


ok, you need to download a new dodsC.war file, follow instructions from
this page:
http://www.unidata.ucar.edu/projects/THREDDS/tech/aggServer/AggServerInstall.html 

here is an example NcML  aggregation that i tested on 3 of your files;
it assumes a uniform 10 minute increment to add a coordinate variable.
it does a join on the recnum dimension on the listed 3 files. it should
work for you, except that obviously you replace with your files, in time
order, and get the starting time right.

<?xml version='1.0' encoding='UTF-8'?>
<netcdf xmlns='http://www.ucar.edu/schemas/netcdf'
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xsi:schemaLocation='http://www.ucar.edu/schemas/netcdf
http://www.unidata.ucar.edu/schemas/netcdfDataset2.xsd' >

  <variable name='recnum' type='int' shape='recnum' >
    <attribute name='long_name' type='string' value='time coordinate' />
    <attribute name='units' type='string' value='minutes since 2001-8-31
23:00:00 UTC' />
    <values start='0' increment='10' />
  </variable>

  <aggregation dimName='recnum' type='joinExisting' />

  <netcdf uri='C:/data/buoy/cwind1.nc'/>
  <netcdf uri='C:/data/buoy/cwind2.nc' />
  <netcdf uri='C:/data/buoy/cwind3.nc'/>

</netcdf>

the agg server configuration looks like this:

<!DOCTYPE catalog SYSTEM
"http://www.unidata.ucar.edu/projects/THREDDS/xml/AggServerCatalog.dtd";>
<catalog name="Test NCML Aggregation" version="0.6"
xmlns="http://www.unidata.ucar.edu/thredds";
xmlns:xlink="http://www.w3.org/1999/xlink";>
        <service name="cwind" serviceType="NetCDF"
base="file:///c:/data/buoy/"/ <file:///c:/data/buoy/>>
        <dataset name="cwind buoy data">
            <property name="internalService" value="cwind"/>
<dataset name="cwind NcML Agg test" urlPath="cwindAgg.xml" /> 
        </dataset>
</catalog>
where the NcML file (as above) lives at file:///c:/data/buoy/cwindAgg.xml 

hopefully this will get you started, let me know how it goes.

BTW, this wont scale too huge numbers of netcdf files, but a few dozen
should be ok. i will need to add some file caching to speed it up, but
the above shoul be ok for small sets of files.