[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Paper on THREDDS Data Server (TDS)



Hi Leilei :

I have made a new release that hopefully will work, find it at the same place. 
It will have version 3.3.05.
Im not sure how you are installing, but its a good idea to delete the expanded 
webapp/thredds directory to be sure you get the new stuff.

Leilei Wang wrote:
I'd already tried the latest v3.3 downloaded from your tech status page:
http://www.unidata.ucar.edu/projects/THREDDS/tech/index.html#TDS*
*TDS server just can't up and run at all:*

**type* Exception report

*message*

*description* _The server encountered an internal error () that prevented it from fulfilling this request._

*exception*

javax.servlet.ServletException: Servlet.init() for servlet ThreddsDefault threw 
exception
        
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
        
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
        org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
        
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
        
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
        
org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
        
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
        java.lang.Thread.run(Thread.java:534)

*root cause*

java.lang.NoSuchMethodError: 
ucar.nc2.iosp.grib.GribServiceProvider.extendIndex(Z)V
        thredds.servlet.ThreddsDefault.init(ThreddsDefault.java:55)
        javax.servlet.GenericServlet.init(GenericServlet.java:211)
        
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
        
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
        org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
        
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
        
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
        
org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
        
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
        java.lang.Thread.run(Thread.java:534)

Leilei

Ethan Davis wrote:

Hi Leilei,

I grabbed the file mentioned below from your ftp site. Robb tried it out and could read it fine (and a few others I got from there).

We are not using TDS 3.2 on motherlode. We are using the code from our development (currently 3.3) branch. We're guessing there is a bug fix on that branch that didn't get into 3.2. If you are interested in trying that version, I can send you the thredds.war file. Since it isn't a stable release yet, we don't advise relying on it but it should be fine for testing and such.

Let me know if you want the 3.3 thredds.war file.

Ethan

Leilei Wang wrote:

Seems it is an unfortunate day for me.
I've reloaded correct GRIB-2 NDFD data files onto our TDS. Metadata seems fine with Netcdf Tool UI. But on "DODS Dataset Access Form",
when I try to get ascii for a subset of some variables, I've got an error. I'm using latest official thredds.war 3.2 and even reinstalled tomcat with the version you suggested, still got no luck. Part of related error msg is followed. Seems array index is out of bound. I remember we got similar problem back to Aug and you've fixed it for other data. Weirdest thing is same data file(downloaded one NDFD file from your site)works on your motherlode while not working on our TDS. Are you using v3.2 with your motherlode thredds?


http://eclipse.ncdc.noaa.gov:9090/thredds/dodsC/ndfd/20040901/YGUZ99_KWBN_200409012154.html

Error {

code = 0;
message = "Connection cannot be read http://eclipse.ncdc.noaa.gov:9090/thredds/dodsC/ndfd/20040901/YGUZ99_KWBN_200409012154.dods?Maximum_temperature[0:1:0][0:1:10][0:1:10]";;


};

2005-11-14T16:46:41.302 -0500 [ 2395772][ 348] INFO - thredds.servlet.ServletUtil - Remote host: 205.167.25.154 - Request: "GET /thredds/dodsC/ndfd/20040901/YGUZ99_KWBN_200409012154.dods?Maximum_temperature[0:1:0][0:1:10][0:1:10] HTTP/1.1"
2005-11-14T16:46:41.384 -0500 [ 2395854][ 348] ERROR - dods.servers.netcdf.NcSDArray - ERROR NcSDArray Read Maximum_temperature
java.lang.ArrayIndexOutOfBoundsException: 121
at ucar.ma2.ArrayFloat.setFloat(ArrayFloat.java:193)
at ucar.ma2.IteratorFast.setFloatNext(IteratorFast.java:64)
at ucar.nc2.iosp.grib.GribServiceProvider.readXY(GribServiceProvider.java:218)


at ucar.nc2.iosp.grib.GribServiceProvider.readData(GribServiceProvider.java:177)

at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:967)
at ucar.nc2.Variable._read(Variable.java:786)
at ucar.nc2.Variable.read(Variable.java:500)
at dods.servers.netcdf.NcSDArray.read(NcSDArray.java:110)
at dods.servers.netcdf.NcSDGrid.read(NcSDGrid.java:62)
at dods.dap.Server.SDGrid.serialize(SDGrid.java:490)
at dods.dap.Server.CEEvaluator.send(CEEvaluator.java:254)
at dods.servlet.DODSServlet.doGetDODS(DODSServlet.java:651)
at dods.servlet.DODSServlet.doGet(DODSServlet.java:1454)
at dods.servers.netcdf.NcDODSServlet.doGet(NcDODSServlet.java:291)
Thanks,
Leilei



Ethan Davis wrote:

Interesting. I wonder what is different about them. Can you send us (or make available for us to grab) an example file?

Thanks,

Ethan

Leilei Wang wrote:

Thanks, Ethan! I'm attaching the error messages. And also you can check our server for these data: http://eclipse.ncdc.noaa.gov:9090/thredds/dodsC/ndfd/20050928/catalog.html


This is the error message I've got when the opendap link is clicked.
***************************************************************
Error { code = 5; message = "NcDODSServlet ERROR opening NcDataset dodsC/ndfd/20050928/ndfd-all_020238_20050929_0000_fff.grb access = /raid/ftp/pub/ndfd/20050928/ndfd-all_020238_20050929_0000_fff.grb null"; };
*****************************************************************
And this is message in log file:
******************************************************************
2005-11-14T11:06:21.588 -0500 [ 331405962][ 268] INFO - thredds.servlet.ServletUtil - Remote host: 192.153.129.249 - Request: "GET /thredds/dodsC/ndfd/20050928/ndfd-all_020238_20050929_0000_fff.grb.html HTTP/1.1"
2005-11-14T11:06:21.653 -0500 [ 331406027][ 268] ERROR - dods.servers.netcdf.NcDODSServlet - NcDODSServlet ERROR opening NcDataset dodsC/ndfd/20050928/ndfd-all_020238_20050929_0000_fff.grb access = /raid/ftp/pub/ndfd/20050928/ndfd-all_020238_20050929_0000_fff.grb
null
java.lang.NullPointerException
at ucar.grib.grib1.Grib1Lookup.<init>(Grib1Lookup.java:36)
at ucar.nc2.iosp.grib.Grib1ServiceProvider.open(Grib1ServiceProvider.java:72)


at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:754)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:334)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:219)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:165)
at ucar.nc2.util.NetcdfFileCache.acquire(NetcdfFileCache.java:161)
at dods.servers.netcdf.GuardedDatasetImpl.<init>(GuardedDatasetImpl.java:53)


at dods.servers.netcdf.NcDODSServlet.getDataset(NcDODSServlet.java:338)
at dods.servlet.DODSServlet.doGetHTML(DODSServlet.java:951)
at dods.servlet.DODSServlet.doGet(DODSServlet.java:1462)
at dods.servers.netcdf.NcDODSServlet.doGet(NcDODSServlet.java:291)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)


at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:432)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)

at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:868)

at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:744)

at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)

at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)

at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)

at java.lang.Thread.run(Unknown Source)
2005-11-14T11:06:21.680 -0500 [ 331406054][ 268] INFO - thredds.servlet.ServletUtil - Request Completed - 200 - -1 - 92
*************************************************************************************


Seems it can't be opened at all and  Netcdf Tool verified this too.

Yes, GRIB-2 data files directly downloaded from NDFD work fine in TDS. But the files I've got problem are from NCDC archive system.
I'm puzzled because these files seem in GRIB-1 format when I use 'degrib' decoder to decode them. And current 'degrib' can't handle them too because center & subcenter are not included in local use tables.
Thanks,
Leilei


Ethan Davis wrote:

Hi Leilei,

The GRIB data is being read by the netCDF-java 2.2 library which has a number of adapters for reading various file formats. I'm CCing Robb Kambic who developed and maintains our GRIB code. I just checked on our server and looks like we are serving NDFD data. Here's the catalog for the NDFD data we're serving in case you want to take a look at that

http://motherlode.ucar.edu:8080/thredds/idd/model/NDFD/CONUS_5km/catalog.xml


Are you getting any relavent messages in the TDS logs in the ${TOMCAT_HOME}/content/thredds/logs directory?


Another thing you could do is use the netCDF Tools UI to look at one of your data files and see if it can read it in. In the "Viewer" tab you can click on the folder icon to find a local file and try to open onf the your NDFD files. That way we'll know if it is a TDS or nc/GRIB issue. If the ToolsUI can't open the file, can you make one available so Robb can take a closer look?

Let us know what kind of error messages (if any) you get from the ToolsUI or in the TDS logs.

Thanks,

Ethan

Leilei Wang wrote:

Hi Ethan:
Which decoder does TDS use to decode GRIB files? Is it degrib? We're trying to serve NCDC archived NDFD data files through TDS which in GRIB but failed.
Thanks,
Leilei


Ethan Davis wrote:

Hi all,

We will be presenting a paper on the THREDDS Data Server (TDS) at the 2006 AMS meeting in Atlanta. It is available online now if you want to take a look before January.

http://www.unidata.ucar.edu/projects/THREDDS/PublicationsAndPresentations/2006AMS-TDS/


Ethan






NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.