[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's



Hi Dan:

Can you zip up the entire content/thredds/logs files and send them to me?

thanks

John

> John,
> 
> I scraped this month of gfsanl data down to the skeleton
> 279 .grb files... nothing else in the directory tree.
> 
> Shut down tomcat.
> Scraped the cache/GribIndex directory clean.
> Scraped logs directory clean.
> restarted tomcat.
> Waited for grib indexing to finish
> (they were all written to the right place)
> Attempted to access.
> 
> Results were disappointing.
> 
> Upon server startup (and its associated index file creation pass)
> 9 java.lang.NullPointerException's
> Upon attempting to access the entry from a browser:
> (which fails with HTTP:404)
> 23 java.lang.NullPointerException
> 
> from command [ grep -i -n "nullPoint" ./* ]
> in logs directory : <tomcat>/content/thredds/logs
> 
> At this point, I'm finding it harder to believe it may be an obscurity
> in our system and/or file systems.
> 
> I attempted to look at an ncx file with ToolsUI but could not make
> heads or tails out of it.
> 
> ~~~~
> at 
> thredds.inventory.MFileCollectionManager$DateSorter.compare(MFileCollectionManager.java:457)
> at 
> thredds.inventory.MFileCollectionManager$DateSorter.compare(MFileCollectionManager.java:455)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
> 
> Are what most of the NPE's started with.   What are these
> methods trying to do?    ~ Might lead us in the right direction.
> 
> -Dan
> NCDC-NOMADS
> 
> 
> 
> 
> On Fri, Mar 8, 2013 at 12:24 PM, Unidata THREDDS Support
> <address@hidden> wrote:
> > Hi Dan:
> >
> > (sorry i missed this message from last week- was just thinking that i hadnt 
> > heard from you).
> >
> > for sure remove the spurious index files from the data directory. make sure 
> > that only the files you want are getting included.
> >
> > you can look at the index files (.ncx) in latest toolsui iosp/cdmIndex tab. 
> > far right info button shown what files are in it.
> >
> > John
> >
> >
> >> I have redone the lists and included the
> >> GribIndex directory 4.3.15 has created:
> >>
> >> http://nomads.ncdc.noaa.gov/data/gfsanl/gribindex.tar.gz
> >> http://nomads.ncdc.noaa.gov/data/gfsanl/findlist.txt.bz2
> >>
> >> The .grb2 are for GFS grid 4, which should not be included
> >> due to the collectionSpec.   I have changed this silghtly since
> >> sending you the configuration, due to Ethans suggestions...
> >> Homing into just one month makes no difference in the
> >> nullPointerException that gets tirggered.
> >>
> >> The index files resident in the file system itself was
> >> not created by this installation -- they ARE going into
> >> GribINdex.. but I'm just not sure if the presense of these
> >> old index files might be causing trouble in some way.  I've
> >> seen stranger things.
> >>
> >> We need to have index files in a separate directory, in some
> >> cases a read only file system is present due to security reasons
> >> and this is actually essential and I'm glad you have developed
> >> this option.
> >>
> >> -Dan
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Feb 25, 2013 at 3:54 PM, Unidata THREDDS Support
> >> <address@hidden> wrote:
> >> > Hi Dan:
> >> >
> >> > 1) can you redo this with "ls -l" to get time, size and ownership?
> >> >    can restrict to *grb*
> >> >
> >> > 2) I see some .grb2 in some directories, not sure what they are:
> >> >
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.grb
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.grb.gbx9
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.grb.ncx
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.idx
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.inv
> >> > ./200711/20071130/gfsanl_4_20071130_1200_000.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1200_003.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1200_006.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1800_000.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1800_003.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1800_006.grb2
> >> >
> >> > 3) the gbx9 and ncx files are the cdm index files. But you have
> >> >
> >> >         <GribIndex>
> >> >                 <alwaysUse>true</alwaysUse>
> >> >                 
> >> > <dir>/home/tomcat/tomcat-main.4.3.14/content/thredds/cache/GribIndex</dir>
> >> >                 <policy>nestedDirectory</policy>
> >> >                 <scour>48 hours</scour>
> >> >                 <maxAge>90 days</maxAge>
> >> >         </GribIndex>
> >> >
> >> > so these will go into 
> >> > /home/tomcat/tomcat-main.4.3.14/content/thredds/cache/GribIndex/
> >> >
> >> > if you are sure this is what you want to do, for sure delete the gbx9 
> >> > and ncx files in the data dirs.
> >> >
> >> > 4) i need a file listing on 
> >> > /home/tomcat/tomcat-main.4.3.14/content/thredds/cache/GribIndex/
> >> >
> >> > 5) we might want to get this working on just one month first.
> >> >
> >> >
> >> >
> >> >> John,
> >> >> Assuming you want the source GRIB files, you can
> >> >> get them here:
> >> >>
> >> >> wget -r http://nomads.ncdc.noaa.gov/data/gfsanl/201001/20100104/
> >> >>
> >> >> For a complete find file list from the top level (nexus/gfsanl):
> >> >>
> >> >> http://nomads.ncdc.noaa.gov/data/gfsanl/findlist.txt.bz2
> >> >>
> >> >> I immediately noticed : gfsanl_3_20040302_0000_000.grb.ncx
> >> >> and the like... I'm not sure what created these...  may have been an
> >> >> older version of TDS.
> >> >> Should I get rid of them?
> >> >>
> >> >> -rw-rw-r-- 1 nomads nomads 5884 Nov 29 12:11
> >> >> ./200403/20040302/gfsanl_3_20040302_0000_000.grb.ncx
> >> >>
> >> >>
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: GHR-456072
> >> > Department: Support THREDDS
> >> > Priority: High
> >> > Status: Open
> >> >
> >>
> >>
> >>
> >> --
> >> =======================================
> >> Dan Swank
> >> NCDC-NOMADS Project
> >> Team ERT/STG
> >> (NCDC Government Contractor)
> >> Data Access & Applications Branch
> >> Veach-Baley Federal Building
> >> 151 Patton Avenue
> >> Asheville, NC 28801-5001
> >> Email: address@hidden
> >> Phone: 828-271-4007
> >> ============================================
> >> Any opinions expressed in this message are mine personally and
> >> do not represent any position from STG, ERT, NCDC, or NOAA
> >> ============================================
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: GHR-456072
> > Department: Support THREDDS
> > Priority: Critical
> > Status: Open
> >
> 
> 
> 
> --
> =======================================
> Dan Swank
> NCDC-NOMADS Project
> Team ERT/STG
> (NCDC Government Contractor)
> Data Access & Applications Branch
> Veach-Baley Federal Building
> 151 Patton Avenue
> Asheville, NC 28801-5001
> Email: address@hidden
> Phone: 828-271-4007
> ============================================
> Any opinions expressed in this message are mine personally and
> do not represent any position from STG, ERT, NCDC, or NOAA
> ============================================
> 
> 


Ticket Details
===================
Ticket ID: GHR-456072
Department: Support THREDDS
Priority: Critical
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.