[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
Re: New Client Reply - [THREDDS !GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's
- Subject: Re: New Client Reply - [THREDDS !GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's
- Date: Wed, 27 Mar 2013 11:36:52 -0600
ok, i have a new war file for you
ftp://ftp.unidata.ucar.edu/pub/thredds/4.3/4.3.17/thredds.war
1. you should delete all the ncx files and let them get recreated.
2. the featureCollectionScan.log should give you errors on the problems, but
tolerate the errors.
3. once you fix the files, you need to delete the ncx files of the partition
that had problems - so MAKE A COPY OF THE problems from the
featureCollectionScan.log, as it gets deleted on each restart
4. You may notice that the time partition ncx files now have the collection
name in them.
5. i would name each collection explicitly:
<collection name="makeupaname"
like:
<featureCollection name="GFSANL feature collection test"
path="gribFC/gfsanlfc"
harvest="true"
featureType="GRIB">
<metadata inherited="true">
<serviceName>ALL</serviceName>
<documentation type="summary">NCEP GFS Model : GRID 003 : Instantaneous
analysis only collection (000.grb)
</documentation>
<dataFormat>GRIB-1</dataFormat>
</metadata>
<collection name="makeupaname"
spec="Q:/support/2013-02_NCDC_GHR-456072/data/**/gfsanl_3_.*\.grb$"
timePartition="1 day"
dateFormatMark="#gfsanl_3_#yyyyMMdd_HHmm"
/>
<update startup="true" trigger="allow"/>
<gribConfig datasetTypes="Best Latest Files"/>
</featureCollection>
otherwise it will use the <featureCollection name="GFSANL feature collection
test"
and better that you can change that later if you want.
6. set logging back to this level:
<!-- featureCollection debugging -->
<logger name="thredds.inventory.MFileCollectionManager" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="thredds.inventory.CollectionUpdater" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="thredds.server.root.CollectionController" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="ucar.nc2.grib" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="ucar.nc2.grib.grib1.Grib1CollectionBuilder" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="ucar.nc2.grib.grib2.Grib2CollectionBuilder" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="thredds.catalog.InvDatasetFeatureCollection" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="thredds.catalog.InvDatasetFcGrib" additivity="false">
<level value="INFO"/>
<appender-ref ref="fcScan"/>
</logger>
<logger name="org.quartz" additivity="false">
<level value="WARN"/>
<appender-ref ref="fcScan"/>
</logger>
7. all partitions will still be one day - working on that.
thanks!
On 3/27/2013 9:43 AM, Dan Swank - NOAA Affiliate wrote:
> Here : wget http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/catalog.xml
> "1 month" seemed fine when accessing via. openDap.
> However, it still showed this on the browser catalog list:
>
> Dataset Size Last Modified
> Folder GFSANL feature collection test --
> Best Timeseries --
> Folder 20040302/ --
> Folder 20040303/ --
> Folder 20040304/ --
> Folder 20040305/ --
> Folder 20040306/ --
> Folder 20040307/
> --- more---
>
>
> Which doesn't seem right. ~ Believe this is what your speaking of.
> Seems to do this no matter what you put in catalog.xml for timePartition.
>
> -Dan
>
>
>
> On Wed, Mar 27, 2013 at 11:30 AM, John Caron <address@hidden> wrote:
>> i saw yesterday that arbitrary time partitions are not working, but ity
>> always uses "1 day". im going to get that fixed.
>>
>> send me your catalog.xml again, thanks
>>
>>
>> On 3/27/2013 9:26 AM, Dan Swank - NOAA Affiliate wrote:
>>>
>>> Still getting the "null" error upon access.
>>>
>>> This seems to be the related exception.
>>>>
>>>> From what I can tell, seems something is wrong
>>>
>>> with the timePartition~ which clearing the .ncx is not correcting
>>> I have in catalog.xml:
>>> timePartition="year"
>>> Is this correct.. should it be "1 year" ?
>>> Going to see if altering this makes a difference.
>>> If it doesn't, i'll have another log package for you later.
>>>
>>>
>>> =========================
>>>
>>> 2013-03-27T11:18:31.048 -0400 [ 1168229][ 12] ERROR -
>>> thredds.server.opendap.OpendapServlet - path=
>>> /gribFC/gfsanlfc/best.ascii
>>> java.lang.NullPointerException
>>> at
>>> ucar.nc2.grib.GribCollection$VariableIndex.readRecords(GribCollection.java:722)
>>> at
>>> ucar.nc2.grib.TimePartition$VariableIndexPartitioned.getVindex(TimePartition.java:204)
>>> at
>>> ucar.nc2.grib.grib1.Grib1Iosp.readDataFromPartition(Grib1Iosp.java:507)
>>> at ucar.nc2.grib.grib1.Grib1Iosp.readData(Grib1Iosp.java:477)
>>> at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:1930)
>>> at ucar.nc2.Variable.reallyRead(Variable.java:894)
>>> at ucar.nc2.Variable._read(Variable.java:880)
>>> at ucar.nc2.Variable.read(Variable.java:692)
>>> at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:553)
>>> at ucar.nc2.dataset.VariableDS._read(VariableDS.java:533)
>>> at ucar.nc2.Variable.read(Variable.java:692)
>>> at ucar.nc2.Variable.read(Variable.java:666)
>>> at thredds.server.opendap.NcSDArray.read(NcSDArray.java:115)
>>> at thredds.server.opendap.NcSDGrid.read(NcSDGrid.java:71)
>>> at opendap.servlet.AsciiWriter.writeAsc(AsciiWriter.java:95)
>>> at opendap.servlet.AsciiWriter.toASCII(AsciiWriter.java:56)
>>> at
>>> thredds.server.opendap.OpendapServlet.doGetASC(OpendapServlet.java:320)
>>> at
>>> thredds.server.opendap.OpendapServlet.doGet(OpendapServlet.java:226)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>> at
>>> thredds.servlet.filter.CookieFilter.doFilter(CookieFilter.java:54)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>> at
>>> thredds.servlet.filter.RequestQueryFilter.doFilter(RequestQueryFilter.java:120)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>> at
>>> thredds.servlet.filter.RequestPathFilter.doFilter(RequestPathFilter.java:102)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>> at
>>> thredds.server.RequestBracketingLogMessageFilter.doFilter(RequestBracketingLogMessageFilter.java:48)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>> at
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
>>> at
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
>>> at
>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
>>> at
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
>>> at
>>> com.googlecode.psiprobe.Tomcat70AgentValve.invoke(Tomcat70AgentValve.java:38)
>>> at
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
>>> at
>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
>>> at
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>>> at
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
>>> at
>>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:999)
>>> at
>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565)
>>> at
>>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>> at java.lang.Thread.run(Thread.java:722)
>>> 2013-03-27T11:18:31.050 -0400 [ 1168231][ 12] INFO -
>>> threddsServlet - Request Completed - 500 - -1 - 110
>>>
>>>
>>>
>>>
>>> On Wed, Mar 27, 2013 at 10:42 AM, Dan Swank - NOAA Affiliate
>>> <address@hidden> wrote:
>>>>
>>>> John,
>>>>
>>>> That is what I have in the catalog.xm now (and yesterday)l
>>>> Clearing up the file system this morning. didn't realize
>>>> there were zero-byte files in there, over 20000 - vast majority
>>>> of them in ./inventory which is not important ~ but a few
>>>> .grb & .grb2's are being patched from archive.
>>>>
>>>> We'll see if that improves things.
>>>> Some level of tolerance for these types of file system
>>>> errors.. an ignore-blacklist of sorts .. would be
>>>> very helpful for stability in huge datasets.
>>>>
>>>> -Dan
>>>>
>>>>
>>>>
>>>> On Tue, Mar 26, 2013 at 5:53 PM, John Caron <address@hidden>
>>>> wrote:
>>>>>
>>>>> So indexing is barfing on these 0 length files:
>>>>>
>>>>> http://nomads.ncdc.noaa.gov/data/gfsanl/201110/20111025/
>>>>>
>>>>> ill try to make it be able to recover....
>>>>>
>>>>> or maybe a "pre scan" report or something
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 3/26/2013 3:32 PM, Dan Swank - NOAA Affiliate wrote:
>>>>>>
>>>>>>
>>>>>> Yeah, thats included in the tar.bz2
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 26, 2013 at 5:26 PM, John Caron <address@hidden>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> can i see the new version of the catalog.xml?
>>>>>>>
>>>>>>>
>>>>>>> On 3/26/2013 3:02 PM, Dan Swank - NOAA Affiliate wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> John,
>>>>>>>>
>>>>>>>> After the changes and restarting the server this morning I could not
>>>>>>>> get anything but :
>>>>>>>>
>>>>>>>>
>>>>>>>> Error {
>>>>>>>> code = 500;
>>>>>>>> message = "null";
>>>>>>>> };
>>>>>>>>
>>>>>>>>
>>>>>>>> So then
>>>>>>>> find cache/GribIndex -name "*.ncx" | xargs rm
>>>>>>>> rm logs/*
>>>>>>>> Restarted -- waited for startup scan to finish --
>>>>>>>> Access problem did not go away.
>>>>>>>>
>>>>>>>> Putting up another set of logs here:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/tdsLogsAndConfig-0326.tar.bz2
>>>>>>>>
>>>>>>>> This has me confused again.
>>>>>>>>
>>>>>>>> No longer seeing any NPE's in featureCollectionScan.log
>>>>>>>> I included log4j.xml for good measure.
>>>>>>>>
>>>>>>>> Should I zap the entire GribIndex directory and start fresh?
>>>>>>>>
>>>>>>>> -Dan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> From openDap requests on the full FC on about 5 different unique
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> parameter requests.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -Dan
>>>>>>>>
>>>>>>>> On Tue, Mar 26, 2013 at 9:59 AM, Dan Swank - NOAA Affiliate
>>>>>>>> <address@hidden> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Rats! that is a 3 minute rescan?
>>>>>>>>> I was intending 3 hours. Good catch,
>>>>>>>>> I Will change.
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> -Dan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Mar 25, 2013 at 6:32 PM, John Caron <address@hidden>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I think this is a problem (in catalog.xml):
>>>>>>>>>>
>>>>>>>>>> <update startup="true" rescan="3 * * * * ? *"
>>>>>>>>>> trigger="allow"
>>>>>>>>>> />
>>>>>>>>>>
>>>>>>>>>> its firing off a whole lot of scans.
>>>>>>>>>>
>>>>>>>>>> try just
>>>>>>>>>>
>>>>>>>>>> <update startup="true" trigger="allow" />
>>>>>>>>>>
>>>>>>>>>> same drill, stop, remove logs, start
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 3/25/2013 3:33 PM, Dan Swank - NOAA Affiliate wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> John,
>>>>>>>>>>>
>>>>>>>>>>> I've been getting a few inconsistant episodes.
>>>>>>>>>>>
>>>>>>>>>>> I've run [find ~cache/GribIndex -name "*.ncx" | xargs rm] several
>>>>>>>>>>> times in between restarts.
>>>>>>>>>>> Does this effectively remove the timePartition information?
>>>>>>>>>>>
>>>>>>>>>>> It seems to cause grief if the aggregation is accessed before the
>>>>>>>>>>> TDM
>>>>>>>>>>> is finished doing its thing in the background... But I suppose
>>>>>>>>>>> thats
>>>>>>>>>>> to be expected. Sometimes its hard to tell when its done and does
>>>>>>>>>>> not seem to function normally until I try accessing the FC once,
>>>>>>>>>>> letting it fail, then restarting the TDS server and/or tomcat.
>>>>>>>>>>> Then I can access the full PoR featurecollection and try some
>>>>>>>>>>> openDap
>>>>>>>>>>> requests.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Todays log sample:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/tdsLogsAndConfig-0325.tar.bz2
>>>>>>>>>>>
>>>>>>>>>>> In threddsServlet.log* , toward the bottom show some of the
>>>>>>>>>>> access-time problems I'm seeing.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> message = "Protocol message tag had invalid wire type.";
>>>>>>>>>>>
>>>>>>>>>>> message = "While parsing a protocol message, the input
>>>>>>>>>>> ended
>>>>>>>>>>> unexpectedly in the middle of a field. This could mean either
>>>>>>>>>>> than
>>>>>>>>>>> the input has been truncated or that an embedded message
>>>>>>>>>>> misreported
>>>>>>>>>>> its own length.";
>>>>>>>>>>>
>>>>>>>>>>> message = "Protocol message contained an invalid tag
>>>>>>>>>>> (zero).";
>>>>>>>>>>>
>>>>>>>>>>> These errors are varying depending on what [t1:t2] i'm using in
>>>>>>>>>>> the
>>>>>>>>>>> URL, which I find a bit strange.
>>>>>>>>>>> Fairly sure I have log4j debug turned on for everything as you
>>>>>>>>>>> asked.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -Dan
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Mar 25, 2013 at 1:16 PM, John Caron
>>>>>>>>>>> <address@hidden>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Dan:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) something happening with the memory cache; index files are not
>>>>>>>>>>>> getting
>>>>>>>>>>>> released. Lines in cache.log:
>>>>>>>>>>>>
>>>>>>>>>>>> [2013-03-22T11:37:19.587-0400] FileCache
>>>>>>>>>>>> GribCollectionPartitionCache
>>>>>>>>>>>> cleanup couldnt remove enough to keep under the maximum= 100 due
>>>>>>>>>>>> to
>>>>>>>>>>>> locked
>>>>>>>>>>>> files; currently at = 771
>>>>>>>>>>>>
>>>>>>>>>>>> this seems to just keeps growing.
>>>>>>>>>>>>
>>>>>>>>>>>> lets add debug messages to FileCache, so in WEB-INF/log4j.xml:
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="cacheLogger" additivity="false">
>>>>>>>>>>>> <level value="DEBUG"/>
>>>>>>>>>>>> <appender-ref ref="cacheLog"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2) not getting all the debug messages i need. check log4j.xml,
>>>>>>>>>>>> make
>>>>>>>>>>>> sure
>>>>>>>>>>>> all these are set to DEBUG:
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="thredds.inventory.MFileCollectionManager"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="thredds.inventory.CollectionUpdater"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="thredds.server.root.CollectionController"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="ucar.nc2.grib" additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="ucar.nc2.grib.grib1.Grib1CollectionBuilder"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="ucar.nc2.grib.grib2.Grib2CollectionBuilder"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="thredds.catalog.InvDatasetFeatureCollection"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="thredds.catalog.InvDatasetFcGrib"
>>>>>>>>>>>> additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> 3) turn this one back to INFO, its mostly noise:
>>>>>>>>>>>>
>>>>>>>>>>>> <logger name="org.quartz" additivity="false">
>>>>>>>>>>>> <level value="INFO"/>
>>>>>>>>>>>> <appender-ref ref="fcScan"/>
>>>>>>>>>>>> </logger>
>>>>>>>>>>>>
>>>>>>>>>>>> 4) theres a damaged ncx file somewhere in one of the time
>>>>>>>>>>>> partitions
>>>>>>>>>>>> in :
>>>>>>>>>>>>
>>>>>>>>>>>> 2013-03-22T11:37:19.557 -0400 [ 3730116][ 12] ERROR -
>>>>>>>>>>>> thredds.server.opendap.OpendapServlet - path=
>>>>>>>>>>>> /gribFC/gfs4anlfc/best.ascii
>>>>>>>>>>>> com.google.protobuf.InvalidProtocolBufferException: While parsing
>>>>>>>>>>>> a
>>>>>>>>>>>> protocol message, the input ended unexpectedly in the middle of a
>>>>>>>>>>>> field.
>>>>>>>>>>>> This could mean either than the input has been truncated or that
>>>>>>>>>>>> an
>>>>>>>>>>>> embedded
>>>>>>>>>>>> message misreported its own length.
>>>>>>>>>>>>
>>>>>>>>>>>> maybe delete all the time partition and let them regenerate. dont
>>>>>>>>>>>> hit
>>>>>>>>>>>> server until that is done.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 5) when you do the restart, be sure to stop tomcat, delete all
>>>>>>>>>>>> logs,
>>>>>>>>>>>> then
>>>>>>>>>>>> restart tomcat.
>>>>>>>>>>>>
>>>>>>>>>>>> thanks!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 3/22/2013 12:02 PM, Dan Swank - NOAA Affiliate wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The location is
>>>>>>>>>>>>> http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/
>>>>>>>>>>>>> For various .ncx
>>>>>>>>>>>>>
>>>>>>>>>>>>> and specifically
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/tdsLogsAndConfig-0322.tar.bz2
>>>>>>>>>>>>> for the latest log run with the access-time errors on gfsanl 4.
>>>>>>>>>>>>> The catalog.xml is in there and has a the gfsanl 4
>>>>>>>>>>>>> addition.
>>>>>>>>>>>>> I've only been enabling one fc a time.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) I was not running two servers, but thought for
>>>>>>>>>>>>> a moment that I had accessed an aggregation from
>>>>>>>>>>>>> a browser before the indexer was finished and that
>>>>>>>>>>>>> might have caused duplicate logging we were seeing.
>>>>>>>>>>>>> Suppose not. Not sure what the dup.s were all about.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) NCDC-NOMADS directory convention allows multiple
>>>>>>>>>>>>> grids to be placed in the same dataset/yyyymm/yyyymmdd tree.
>>>>>>>>>>>>> (in the gfsanl case ~ global grid 3 & 4)
>>>>>>>>>>>>> Unfortunately the CFSR does this as well.
>>>>>>>>>>>>> So I'm assuming the need to split them up
>>>>>>>>>>>>> into multiple discrete featureCollections
>>>>>>>>>>>>> using the <collection spec> to distinguish them.
>>>>>>>>>>>>> Given that grid 3 is GRIBv1 and grid 4 is GRIBv2,
>>>>>>>>>>>>> two fc's are probably required.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was having the trouble with gfsanl 3 before I
>>>>>>>>>>>>> added the gfsanl 4 to the picture. At first I had
>>>>>>>>>>>>> an FC working, with access-errors, just like the gfsanl 4
>>>>>>>>>>>>> is now. Then I updated to the latest thredds.war and
>>>>>>>>>>>>> that seemed to break the gfsanl 3 completely.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Dan
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Mar 22, 2013 at 1:31 PM, John Caron
>>>>>>>>>>>>> <address@hidden>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Dan:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) Only one indexer can work at a time. Since your files are
>>>>>>>>>>>>>> static,
>>>>>>>>>>>>>> once
>>>>>>>>>>>>>> they get indexed, should be able to turn that off
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) are the feature collections always working on disjoint sets
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>> files?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) Your point about a conflict with 2 feature collections
>>>>>>>>>>>>>> working
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> files
>>>>>>>>>>>>>> in the same directory seems possible, even if files are
>>>>>>>>>>>>>> disjoint.
>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>> explain your troubles that ibe never seen. Let me try to
>>>>>>>>>>>>>> reproce
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>> look
>>>>>>>>>>>>>> for a workaround.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 3/22/2013 10:04 AM, Dan Swank - NOAA Affiliate wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> tdsLogsAndConfig-0322.tar.bz2
>>>>>>>>>>>>>>> and several .ncx files supposedly created successfully
>>>>>>>>>>>>>>> have been placed in the usual location.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Examples of errors from my previous message can
>>>>>>>>>>>>>>> be found near 11:38AM in tdsLogsAndConfig-0322.tar.bz2
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Back to your points.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think the tripling in the logs was dur to me accessing the
>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>> before the initial fire-up scan was finished. Sorry.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This may explain the 'cannot delete log' too, as one process
>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> swept it out from under another process.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Its unlikely to be permissions
>>>>>>>>>>>>>>> -rw-rw-r-- 1 tomcat tomcat 6690 Mar 21 16:17
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070115.ncx
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Side question:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -rw-rw-r-- 1 tomcat tomcat 16372 Mar 15 19:48
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ./global/nomads/nexus/gfsanl/201003/20100304/gfsanl_3_20100304_1800_000.grb.gbx9
>>>>>>>>>>>>>>> -rw-rw-r-- 1 tomcat tomcat 6428 Mar 20 15:24
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ./global/nomads/nexus/gfsanl/201003/20100304/gfsanl_3_20100304_1800_000.grb.ncx
>>>>>>>>>>>>>>> -rw-rw-r-- 1 tomcat tomcat 32631 Mar 21 18:40
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ./global/nomads/nexus/gfsanl/201003/20100304/gfsanl_4_20100304_1800_000.grb2.gbx9
>>>>>>>>>>>>>>> -rw-rw-r-- 1 tomcat tomcat 7384 Mar 21 18:40
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ./global/nomads/nexus/gfsanl/201003/20100304/gfsanl_4_20100304_1800_000.grb2.ncx
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2 different fc aggregations (gfsanl 3 & gfsanl 4) seem to
>>>>>>>>>>>>>>> share
>>>>>>>>>>>>>>> this,
>>>>>>>>>>>>>>> will this be OK:
>>>>>>>>>>>>>>> -rw-rw-r-- 1 tomcat tomcat 15705 Mar 21 18:40
>>>>>>>>>>>>>>> ./global/nomads/nexus/gfsanl/20100304.ncx
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Mar 22, 2013 at 11:16 AM, John Caron
>>>>>>>>>>>>>>> <address@hidden>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So looking at the first problem in the logs:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2013-03-21T13:12:03.001-0400] DEBUG
>>>>>>>>>>>>>>>> thredds.inventory.CollectionUpdater:
>>>>>>>>>>>>>>>> Update for GFSANL feature collection test trigger =
>>>>>>>>>>>>>>>> rescan.GFSANL
>>>>>>>>>>>>>>>> feature
>>>>>>>>>>>>>>>> collection test
>>>>>>>>>>>>>>>> [2013-03-21T13:13:29.906-0400] INFO
>>>>>>>>>>>>>>>> thredds.inventory.MFileCollectionManager: GFSANL feature
>>>>>>>>>>>>>>>> collection
>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>> was scanned MCollection{name='/global/nomads/nexus/gfsanl',
>>>>>>>>>>>>>>>> dirName='/global/nomads/nexus/gfsanl', wantSubdirs=true,
>>>>>>>>>>>>>>>> ff=WildcardMatchOnPath{wildcard=null
>>>>>>>>>>>>>>>> regexp=gfsanl_3_........_..00_000\.grb$}}
>>>>>>>>>>>>>>>> [2013-03-21T13:13:29.913-0400] INFO
>>>>>>>>>>>>>>>> thredds.inventory.MFileCollectionManager: GFSANL feature
>>>>>>>>>>>>>>>> collection
>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>> was scanned MCollection{name='/global/nomads/nexus/gfsanl',
>>>>>>>>>>>>>>>> dirName='/global/nomads/nexus/gfsanl', wantSubdirs=true,
>>>>>>>>>>>>>>>> ff=WildcardMatchOnPath{wildcard=null
>>>>>>>>>>>>>>>> regexp=gfsanl_3_........_..00_000\.grb$}}
>>>>>>>>>>>>>>>> [2013-03-21T13:13:29.913-0400] INFO
>>>>>>>>>>>>>>>> thredds.inventory.MFileCollectionManager: GFSANL feature
>>>>>>>>>>>>>>>> collection
>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>> was scanned MCollection{name='/global/nomads/nexus/gfsanl',
>>>>>>>>>>>>>>>> dirName='/global/nomads/nexus/gfsanl', wantSubdirs=true,
>>>>>>>>>>>>>>>> ff=WildcardMatchOnPath{wildcard=null
>>>>>>>>>>>>>>>> regexp=gfsanl_3_........_..00_000\.grb$}}
>>>>>>>>>>>>>>>> [2013-03-21T13:13:31.431-0400] INFO
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder:
>>>>>>>>>>>>>>>> TimePartitionBuilder
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/GFSANL_feature_collection_test.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:32.118-0400] INFO
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder:
>>>>>>>>>>>>>>>> TimePartitionBuilder
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/GFSANL_feature_collection_test.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:32.121-0400] INFO
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder:
>>>>>>>>>>>>>>>> TimePartitionBuilder
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/GFSANL_feature_collection_test.ncx
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (not sure why we are getting triplet logs above)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.628-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070101: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.628-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070101:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070101.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.720-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070102: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.720-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070102:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070102.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.779-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070103: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.779-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070103:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070103.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.820-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070104: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.820-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070104:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070104.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.872-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070105: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.872-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070105:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070105.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.920-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070106: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.920-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070106:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070106.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.976-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070107: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:34.977-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070107:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070107.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.025-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070108: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.025-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070108:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070108.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.076-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070109: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.076-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070109:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070109.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.101-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070110: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.101-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070110:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070110.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.122-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070111: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.122-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070111:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070111.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.141-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070112: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.141-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070112:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070112.ncx
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (these first 12 files appear to be successfully regernerated.
>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> look
>>>>>>>>>>>>>>>> and make sure they are reasonable length? if so, grab one and
>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> me)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.160-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070113: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.161-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070113:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070113.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.170-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070113:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070113.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.171-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070113:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070113.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070114: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070114: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070114: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070114:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070114.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070114:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070114.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] WARN
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> Failed
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070114.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.210-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070114:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070114.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070115: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070115: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070115: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] WARN
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> Failed
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070115.ncx
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (this is funny - why would it fail to delete? permissions?
>>>>>>>>>>>>>>>> 20070114
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> 20070115 )
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070115:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070115.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070115:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070115.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.234-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070115:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070115.ncx
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (back to triple message mode)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.261-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070116: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.261-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> GribCollection 20070116: invalid index
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.261-0400] WARN
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> Failed
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> delete
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070116.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.261-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070116:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070116.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.261-0400] DEBUG
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection:
>>>>>>>>>>>>>>>> 20070116:
>>>>>>>>>>>>>>>> createIndex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070116.ncx
>>>>>>>>>>>>>>>> [2013-03-21T13:13:35.262-0400] ERROR
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder: Failed to
>>>>>>>>>>>>>>>> open
>>>>>>>>>>>>>>>> partition
>>>>>>>>>>>>>>>> 20070116
>>>>>>>>>>>>>>>> java.io.FileNotFoundException:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/gfsanl/20070116.ncx
>>>>>>>>>>>>>>>> (No such file or directory)
>>>>>>>>>>>>>>>> at java.io.RandomAccessFile.open(Native Method)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> java.io.RandomAccessFile.<init>(RandomAccessFile.java:118)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.unidata.io.RandomAccessFile.<init>(RandomAccessFile.java:277)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.unidata.io.RandomAccessFile.<init>(RandomAccessFile.java:259)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1CollectionBuilder.readIndex(Grib1CollectionBuilder.java:183)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1CollectionBuilder.readOrCreateIndex(Grib1CollectionBuilder.java:154)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1CollectionBuilder.factory(Grib1CollectionBuilder.java:85)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> ucar.nc2.grib.GribCollection.factory(GribCollection.java:177)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.TimePartition$Partition.makeGribCollection(TimePartition.java:170)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder.createPartitionedIndex(Grib1TimePartitionBuilder.java:162)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder.readOrCreateIndex(Grib1TimePartitionBuilder.java:128)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ucar.nc2.grib.grib1.Grib1TimePartitionBuilder.factory(Grib1TimePartitionBuilder.java:73)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> ucar.nc2.grib.TimePartition.factory(TimePartition.java:96)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thredds.catalog.InvDatasetFcGrib.updateCollection(InvDatasetFcGrib.java:225)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thredds.catalog.InvDatasetFcGrib.update(InvDatasetFcGrib.java:164)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thredds.catalog.InvDatasetFeatureCollection.handleCollectionEvent(InvDatasetFeatureCollection.java:226)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (something different happens here - not sure why since the
>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>> invalid
>>>>>>>>>>>>>>>> index and " Failed to delete" that 20070114 and 20070115
>>>>>>>>>>>>>>>> have,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ill look deeper, but can you see what you see on those ncx
>>>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>> ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> =======================================
>>>>>>>>> Dan Swank
>>>>>>>>> NCDC-NOMADS Project
>>>>>>>>> Team ERT/STG
>>>>>>>>> (NCDC Government Contractor)
>>>>>>>>> Data Access & Applications Branch
>>>>>>>>> Veach-Baley Federal Building
>>>>>>>>> 151 Patton Avenue
>>>>>>>>> Asheville, NC 28801-5001
>>>>>>>>> Email: address@hidden
>>>>>>>>> Phone: 828-271-4007
>>>>>>>>> ============================================
>>>>>>>>> Any opinions expressed in this message are mine personally and
>>>>>>>>> do not represent any position from STG, ERT, NCDC, or NOAA
>>>>>>>>> ============================================
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> =======================================
>>>> Dan Swank
>>>> NCDC-NOMADS Project
>>>> Team ERT/STG
>>>> (NCDC Government Contractor)
>>>> Data Access & Applications Branch
>>>> Veach-Baley Federal Building
>>>> 151 Patton Avenue
>>>> Asheville, NC 28801-5001
>>>> Email: address@hidden
>>>> Phone: 828-271-4007
>>>> ============================================
>>>> Any opinions expressed in this message are mine personally and
>>>> do not represent any position from STG, ERT, NCDC, or NOAA
>>>> ============================================
>>>
>>>
>>>
>>>
>>
>
>
>
