[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's

1. grab new war file dated today:

ftp://ftp.unidata.ucar.edu/pub/thredds/4.3/4.3.16/thredds.war

2. unload, then reinstall and let new warfile get made. go into

webapps/thredds/WEB-INF/log4j.xml

and add (anywhere):

  <logger name="thredds.filesystem.ControllerOS" additivity="false">
    <level value="DEBUG"/>
    <appender-ref ref="fcScan"/>
  </logger>

3) start up again, let the thing rescan and then look at (and send to me) 

  content/thredds/logs/featureCollectionScan.log 

4) i added a protection against the null pointer exception , so i think you 
wont see it again. however, you should see the DEBUG log messages that shouold 
tell us whats goin on, like:

[2013-03-14T13:20:03.004-0600] DEBUG thredds.filesystem.ControllerOS: List 
Directory Q:\support\2013-02_NCDC_GHR-456072\data
[2013-03-14T13:20:03.008-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013
[2013-03-14T13:20:03.009-0600] DEBUG thredds.filesystem.ControllerOS: List 
Directory Q:\support\2013-02_NCDC_GHR-456072\data\2013
[2013-03-14T13:20:03.012-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101
[2013-03-14T13:20:03.014-0600] DEBUG thredds.filesystem.ControllerOS: List 
Directory Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101
[2013-03-14T13:20:03.017-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_000.grb.ncx
[2013-03-14T13:20:03.019-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_003.grb.gbx9
[2013-03-14T13:20:03.021-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_003.grb.ncx
[2013-03-14T13:20:03.023-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_006.grb.gbx9
[2013-03-14T13:20:03.025-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_006.grb.ncx
[2013-03-14T13:20:03.027-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_006.grb
[2013-03-14T13:20:03.028-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_000.grb.gbx9
[2013-03-14T13:20:03.030-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_000.grb
[2013-03-14T13:20:03.032-0600] DEBUG thredds.filesystem.ControllerOS:   File 
Q:\support\2013-02_NCDC_GHR-456072\data\2013\20130101\gfsanl_3_20130101_0000_003.grb



 

> Thanks, need something to tell me what exactly
> is tripping it up.   Hard to tell from the exceptions in the logs.
> 
> 
> 
> On Thu, Mar 14, 2013 at 1:26 PM, Unidata THREDDS Support
> <address@hidden> wrote:
> > yeah, something on your file system, that i havent seen here.
> > do you have any details on that file system?
> >
> > i will give you something to run to try and track it down.
> >
> > John
> >
> >
> >
> >> John, Something is odd b/c
> >>
> >> spec="/global/nomads/nexus/gfsanl/201001/**/gfsanl_3_........_..00_000\.grb$"
> >> works great, but as soon as I change it to
> >> spec="/global/nomads/nexus/gfsanl/**/gfsanl_3_........_..00_000\.grb$"
> >> The featureCollectionScan fails with NPE's in the logs.
> >>
> >> New Logs package:
> >> http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/tds-logs-0314.tar.bz2
> >>
> >> I am assuming there is something extraneous in the top level
> >> directory that may be causing this.
> >> http://nomads.ncdc.noaa.gov/data/gfsanl/
> >> But want your take on it.   And if so,
> >> is there a way to specify a pattern such as :
> >>
> >> /global/nomads/nexus/gfsanl/??????/**/gfsanl_3_........_..00_000\.grb$
> >>
> >> -Dan
> >>
> >>
> >>
> >> On Wed, Mar 13, 2013 at 7:32 PM, Unidata THREDDS Support
> >> <address@hidden> wrote:
> >> > I would expect this one to work:
> >> >
> >> >  spec="/global/nomads/nexus/gfsanl/**/gfsanl_3_........_..00_000\.grb$"
> >> >
> >> > try
> >> >
> >> >  spec="/global/nomads/nexus/gfsanl/**/gfsanl_3_.*00_000\.grb$"
> >> >
> >> > or similar.
> >> >
> >> >> Hopefully an easy one...
> >> >>
> >> >> I'm trying to expand the gfsanl to all months.  path pattern:
> >> >> /global/nomads/nexus/gfsanl/yyyymm/yyyymmdd/~files~
> >> >>
> >> >> Having some trouble figuring the correct collection spec... the path 
> >> >> part of it.
> >> >> I've attempted:
> >> >> spec="/global/nomads/nexus/gfsanl/**/**/gfsanl_3_........_..00_000\.grb$"
> >> >> spec="/global/nomads/nexus/gfsanl/**/gfsanl_3_........_..00_000\.grb$"
> >> >> Both fail to scan anything on TDS startup.
> >> >> Could you tell how to correctly set this?
> >> >>
> >> >> I've looked here.. but it doesn't seem to clarify how it deals with
> >> >> nested subdirectories.
> >> >> https://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.2/reference/collections/CollectionSpecification.html
> >> >>
> >> >>
> >> >> -Dan
> >> >>
> >> >>
> >> >> On Wed, Mar 13, 2013 at 10:50 AM, Dan Swank - NOAA Affiliate
> >> >> <address@hidden> wrote:
> >> >> > Morning,
> >> >> >
> >> >> > I've been removing the entire grib index directory for each test.
> >> >> > I know ultimately this won't be necessary, but for the monthly tests
> >> >> > its only taking about 3-5 mins to regenerate from scratch.
> >> >> >
> >> >> > #1)
> >> >> > <dataFormat>GRIB-2</dataFormat>
> >> >> > * That was the NAM problem, thanks.  Overlooked that too.
> >> >> >
> >> >> > The "1 day" time partition was only a test for the monthly sample.
> >> >> > Planned to increase once we get rolling on larger tests.
> >> >> >
> >> >> > For now, I've been using 000.grb for simplicity -- eventually I want
> >> >> > to "stripe across" 003.grb & 006.grb as well.
> >> >> > Unsure if this can be done in the same feature collection.. could it?
> >> >> >
> >> >> >
> >> >> > #2)  We have:
> >> >> > /san5102/netcdf4/libnetcdf.a
> >> >> > Which is intended to be used as static.
> >> >> > TDS seems to be looking for a shared object lib which isn't there.
> >> >> > (its a kludge installation intended for making a static wgrib2)
> >> >> > Can this somehow be rigged to go for the .a?
> >> >> >
> >> >> >
> >> >> > #3)  Done.
> >> >> >
> >> >> > misc)
> >> >> >
> >> >> > In catalogInit there is:
> >> >> >
> >> >> > readCatalog(): full
> >> >> > path=/home/tomcat/tomcat-main.4.3.15/content/thredds/catalog.xml;
> >> >> > path=catalog.xml
> >> >> >    valid catalog -- ----Catalog Validation
> >> >> > *** XML parser error (469:37)= s4s-elt-invalid-content.1: The content
> >> >> > of 'gribConfigType' is invalid.  Element 'element' is invalid,
> >> >> > misplaced, or occurs too often.
> >> >> >
> >> >> > Not sure what this is, but its not stopping things.
> >> >> >
> >> >> > Looks like no need for another log dump at this time.
> >> >> > I'll be working on expanding these featureCollections today and see
> >> >> > how they scale up.
> >> >> >
> >> >> >          Thanks! -Dan
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Tue, Mar 12, 2013 at 6:35 PM, Unidata THREDDS Support
> >> >> > <address@hidden> wrote:
> >> >> >> Another thing:
> >> >> >>
> >> >> >> You probably want to use
> >> >> >>
> >> >> >>         timePartition="1 month"
> >> >> >>
> >> >> >> instead of
> >> >> >>
> >> >> >>         timePartition="1 day"
> >> >> >>
> >> >> >> not sure how much youve thought about this?
> >> >> >>
> >> >> >> lets talk about it when you are ready to deal.
> >> >> >>
> >> >> >> if you want to change, delete the .ncx files and let them regenerate 
> >> >> >> (pretty quick). gbx9 are ok (they take a long time)
> >> >> >>
> >> >> >> Ticket Details
> >> >> >> ===================
> >> >> >> Ticket ID: GHR-456072
> >> >> >> Department: Support THREDDS
> >> >> >> Priority: Critical
> >> >> >> Status: Open
> >> >> >>
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > =======================================
> >> >> > Dan Swank
> >> >> > NCDC-NOMADS Project
> >> >> > Team ERT/STG
> >> >> > (NCDC Government Contractor)
> >> >> > Data Access & Applications Branch
> >> >> > Veach-Baley Federal Building
> >> >> > 151 Patton Avenue
> >> >> > Asheville, NC 28801-5001
> >> >> > Email: address@hidden
> >> >> > Phone: 828-271-4007
> >> >> > ============================================
> >> >> > Any opinions expressed in this message are mine personally and
> >> >> > do not represent any position from STG, ERT, NCDC, or NOAA
> >> >> > ============================================
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> =======================================
> >> >> Dan Swank
> >> >> NCDC-NOMADS Project
> >> >> Team ERT/STG
> >> >> (NCDC Government Contractor)
> >> >> Data Access & Applications Branch
> >> >> Veach-Baley Federal Building
> >> >> 151 Patton Avenue
> >> >> Asheville, NC 28801-5001
> >> >> Email: address@hidden
> >> >> Phone: 828-271-4007
> >> >> ============================================
> >> >> Any opinions expressed in this message are mine personally and
> >> >> do not represent any position from STG, ERT, NCDC, or NOAA
> >> >> ============================================
> >> >>
> >> >>
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: GHR-456072
> >> > Department: Support THREDDS
> >> > Priority: Critical
> >> > Status: Open
> >> >
> >>
> >>
> >>
> >> --
> >> =======================================
> >> Dan Swank
> >> NCDC-NOMADS Project
> >> Team ERT/STG
> >> (NCDC Government Contractor)
> >> Data Access & Applications Branch
> >> Veach-Baley Federal Building
> >> 151 Patton Avenue
> >> Asheville, NC 28801-5001
> >> Email: address@hidden
> >> Phone: 828-271-4007
> >> ============================================
> >> Any opinions expressed in this message are mine personally and
> >> do not represent any position from STG, ERT, NCDC, or NOAA
> >> ============================================
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: GHR-456072
> > Department: Support THREDDS
> > Priority: Critical
> > Status: Open
> >
> 
> 
> 
> --
> =======================================
> Dan Swank
> NCDC-NOMADS Project
> Team ERT/STG
> (NCDC Government Contractor)
> Data Access & Applications Branch
> Veach-Baley Federal Building
> 151 Patton Avenue
> Asheville, NC 28801-5001
> Email: address@hidden
> Phone: 828-271-4007
> ============================================
> Any opinions expressed in this message are mine personally and
> do not represent any position from STG, ERT, NCDC, or NOAA
> ============================================
> 
> 

Ticket Details
===================
Ticket ID: GHR-456072
Department: Support THREDDS
Priority: Critical
Status: Open