Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [thredds] grib2 feature collection on TDS 4.5 dev

Hi Guan:
<collection spec="/var/lib/tomcat7054/content/thredds/public/testdata/glw.grl.t*z.grib2$" 

is a regular expression, not a glob

it probably should be

/var/lib/tomcat7054/content/thredds/public/testdata/glw.grl.t.*z.grib2$

John

On 6/9/2014 8:47 AM, Guan Wang wrote:


Hi,
I have trouble to make TDS 4.5 dev work with grib2 feature collection. The WaveWatch III grib2 file can be served individually in TDS 4.5 but the feature collection doesn't work. I guess I must miss something in the xml. Could you please shed some light on this? 

Here is the xml:

<?xml version="1.0" encoding="UTF-8"?>

<catalog name="WWIII GL"

xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0";

xmlns:xlink="http://www.w3.org/1999/xlink";>

<service name="latest" serviceType="Resolver" base="" />

  <service name="both" serviceType="Compound" base="">

    <service name="ncdods" serviceType="OPENDAP" base="/thredds/dodsC/" />
<service name="HTTPServer" serviceType="HTTPServer" base="/thredds/fileServer/" /> 

</service>

<dataset name="Great Lakes WWIII">

<metadata inherited="true">

<serviceName>both</serviceName>

<authority>edu.ucar.unidata</authority>

<dataFormat>GRIB-2</dataFormat>

      <documentation type="rights">Freely available</documentation>

      <documentation xlink:href=" " xlink:title=" "></documentation>

      <creator>

        <name vocabulary="DIF"> </name>

        <contact url=" " email=" " />

      </creator>

      <publisher>

        <name vocabulary="DIF"> </name>
<contact url="http://www.unidata.ucar.edu/"; email="support@xxxxxxxxxxxxxxxx" /> 

      </publisher>

    </metadata>
<featureCollection name="WWIII-GL" featureType="GRIB2" harvest="true" path="glww3/"> 

   <metadata inherited="true">

<!--<dataFormat>GRIB2</dataFormat>-->

       <documentation type="summary">

        WWIII GL Test

       </documentation>

     </metadata>
<collection spec="/var/lib/tomcat7054/content/thredds/public/testdata/glw.grl.t*z.grib2$" timePartition="directory" olderThan="5 min" dateFormatMark="#ww3_#yyyy-MM-ddTHH:mm:ssZ"/> 

   <update startup="true" trigger="allow"/>

   <tdm rescan="0 0/15 * * * ? *" trigger="allow"/>

   <gribConfig datasetTypes="TwoD Best LatestFile Files" />

</featureCollection>

</dataset>

</catalog>

Here is the traceback stack:
2014-06-06T20:10:33.535 +0000 INFO - FeatureCollection added = name ='WWIII_GL' type='GRIB2' 

spec='/var/lib/tomcat7054/content/thredds/public/testdata/glw.grl.t??z.grib2$'

  olderThan =5 min

  timePartition =none
updateConfig =UpdateConfig{userDefined=false, recheckAfter='null', rescan='null', triggerOk=true, updateType=test}
tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='0 0/15 * * * ? *', triggerOk=true, updateType=test}
ProtoConfig{choice=Penultimate, change='null', param='null', outerNcml='null', cacheAll=true} 

  hasInnerNcml =false
GribConfig{datasets=[TwoD, Best, Latest], pdsHash={useGenType=false, intvMerge=true, useTableVersion=true}, filesSortIncreasing=true}
2014-06-06T20:10:33.613 +0000 INFO - Schedule startup scan force=test for 'WWIII_GL' at 2014-06-06T20:10:36.607+0000
2014-06-06T20:10:36.641 +0000 WARN - No records found in files. Check Grib1/Grib2 for collection WWIII_GL-testdata. If wrong, delete gbx9.
2014-06-06T20:10:36.642 +0000 ERROR - UpdateCollectionJob.execute startup failed collection=WWIII_GL
java.lang.IllegalStateException: No records found in dataset WWIII_GL-testdata
at ucar.nc2.grib.collection.Grib2CollectionBuilder.makeGroups(Grib2CollectionBuilder.java:157) ~[grib-4.5.0.jar:4.5.0]
at ucar.nc2.grib.collection.GribCollectionBuilder.createIndex(GribCollectionBuilder.java:117) ~[grib-4.5.0.jar:4.5.0]
at ucar.nc2.grib.collection.GribCdmIndex.updateGribCollection(GribCdmIndex.java:243) ~[grib-4.5.0.jar:4.5.0]
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollection(GribCdmIndex.java:461) ~[grib-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFcGrib.updateCollection(InvDatasetFcGrib.java:151) ~[grib-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.checkState(InvDatasetFeatureCollection.java:294) ~[cdm-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.update(InvDatasetFeatureCollection.java:313) ~[cdm-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.sendEvent(InvDatasetFeatureCollection.java:230) ~[cdm-4.5.0.jar:4.5.0]
at thredds.inventory.CollectionUpdater$UpdateCollectionJob.execute(CollectionUpdater.java:207) [cdm-4.5.0.jar:4.5.0]
at org.quartz.core.JobRunShell.run(JobRunShell.java:207) [quartz-2.2.0.jar:?]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:560) [quartz-2.2.0.jar:?]
2014-06-06T20:13:20.577 +0000 INFO - FeatureCollection added = name ='WWIII_GL' type='GRIB2' 

spec='/var/lib/tomcat7054/content/thredds/public/testdata/glw.grl.t??z.grib2$'

  olderThan =5 min

  timePartition =none
updateConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='null', triggerOk=true, updateType=never}
tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='0 0/15 * * * ? *', triggerOk=true, updateType=test}
ProtoConfig{choice=Penultimate, change='null', param='null', outerNcml='null', cacheAll=true} 

  hasInnerNcml =false
GribConfig{datasets=[TwoD, Best, Files, Latest], pdsHash={useGenType=false, intvMerge=true, useTableVersion=true}, filesSortIncreasing=true}
2014-06-06T20:13:20.659 +0000 INFO - Schedule startup scan force=never for 'WWIII_GL' at 2014-06-06T20:13:23.653+0000
2014-06-06T20:13:23.667 +0000 INFO - updateGribCollection WWIII_GL changed false took 3 msecs
2014-06-06T20:13:23.670 +0000 WARN - GribCdmIndex.openCdmIndex failed on /var/lib/tomcat7054/content/thredds/public/testdata/WWIII_GL.ncx2
java.io.FileNotFoundException: /var/lib/tomcat7054/content/thredds/public/testdata/WWIII_GL.ncx2 (No such file or directory) 

        at java.io.RandomAccessFile.open(Native Method) ~[?:1.7.0_60]
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241) ~[?:1.7.0_60]
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:122) ~[?:1.7.0_60]
at ucar.unidata.io.RandomAccessFile.<init>(RandomAccessFile.java:279) ~[cdm-4.5.0.jar:4.5.0]
at ucar.unidata.io.RandomAccessFile.<init>(RandomAccessFile.java:261) ~[cdm-4.5.0.jar:4.5.0]
at ucar.nc2.grib.collection.GribCdmIndex.openCdmIndex(GribCdmIndex.java:109) [grib-4.5.0.jar:4.5.0]
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollection(GribCdmIndex.java:464) [grib-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFcGrib.updateCollection(InvDatasetFcGrib.java:151) [grib-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.checkState(InvDatasetFeatureCollection.java:294) [cdm-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.update(InvDatasetFeatureCollection.java:313) [cdm-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.sendEvent(InvDatasetFeatureCollection.java:230) [cdm-4.5.0.jar:4.5.0]
at thredds.inventory.CollectionUpdater$UpdateCollectionJob.execute(CollectionUpdater.java:207) [cdm-4.5.0.jar:4.5.0]
at org.quartz.core.JobRunShell.run(JobRunShell.java:207) [quartz-2.2.0.jar:?]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:560) [quartz-2.2.0.jar:?]
2014-06-06T20:13:23.673 +0000 ERROR - InvDatasetFcGrib.updateCollection failed name ='WWIII_GL' type='GRIB2' 

spec='/var/lib/tomcat7054/content/thredds/public/testdata/glw.grl.t??z.grib2$'

  olderThan =5 min

  timePartition =none
updateConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='null', triggerOk=true, updateType=never}
tdmConfig =UpdateConfig{userDefined=true, recheckAfter='null', rescan='0 0/15 * * * ? *', triggerOk=true, updateType=test}
ProtoConfig{choice=Penultimate, change='null', param='null', outerNcml='null', cacheAll=true} 

  hasInnerNcml =false
GribConfig{datasets=[TwoD, Best, Files, Latest], pdsHash={useGenType=false, intvMerge=true, useTableVersion=true}, filesSortIncreasing=true}
2014-06-06T20:13:23.674 +0000 ERROR - UpdateCollectionJob.execute startup failed collection=WWIII_GL 

java.lang.NullPointerException
at thredds.catalog.InvDatasetFcGrib.makeDatasetFromCollection(InvDatasetFcGrib.java:183) ~[grib-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFcGrib.makeDatasetTop(InvDatasetFcGrib.java:516) ~[grib-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.checkState(InvDatasetFeatureCollection.java:295) ~[cdm-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.update(InvDatasetFeatureCollection.java:313) ~[cdm-4.5.0.jar:4.5.0]
at thredds.catalog.InvDatasetFeatureCollection.sendEvent(InvDatasetFeatureCollection.java:230) ~[cdm-4.5.0.jar:4.5.0]
at thredds.inventory.CollectionUpdater$UpdateCollectionJob.execute(CollectionUpdater.java:207) [cdm-4.5.0.jar:4.5.0]
at org.quartz.core.JobRunShell.run(JobRunShell.java:207) [quartz-2.2.0.jar:?]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:560) [quartz-2.2.0.jar:?]
It appears that the index file can not be built for feature collection. But, I bet it's not the root cause. 

Thanks a lot!

Guan



_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/
  • 2014 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: