Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
I have to work with a lot of data, which are structured differently (in
naming and location conventions)
from a product to another one. I wonder if all of these would be easilly
configurable or if we have to
change their organisation. So I have a few questions:
- I would like to know more about all the scan fonctionnalities:
Are they working the same way for matching regExp, and is the full file
path tested by this regExp or only the file name ?
If the entire path is tested, does dateFormatMark, runDateMatcher and
other date markers use the full path to ?
In other word, is it possible to extract date informations from
directory name as from file name ?
- In which cases netcdf files are opened when building catalogs, and if I
provide needed metadata,
will they be opened to (optimizations)?
- I have a problem with the following datasetFmrc:
Dataset definition:
-------------------
<datasetFmrc name="TestDatasetFmrc" path="testDatasetFmrc">
<metadata inherited="true">
<serviceName>multiService</serviceName>
<dataType>Grid</dataType>
<dataFormat>NetCDF</dataFormat>
</metadata>
<netcdf xmlns="
http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; enhance="true">
<aggregation dimName="run" type="forecastModelRunCollection"
recheckEvery="1 hour">
<scan
location="/home/coriolis_exp/spool/co01/co0123/co012302/co01230204/archive/"
regExp="(.*)MM5_200710([0-9]{2})\.nc$"
suffix=".nc" subdirs="true"
dateFormatMark="MM5_#yyyyMMdd"
olderThan="5 min" />
</aggregation>
</netcdf>
</datasetFmrc>
The files are all in the same directory:
----------------------------------------
|-- /home/coriolis_exp/spool/co01/co0123/co012302/co01230204/archive/
|-- ...
|-- MM5_20071008.nc
|-- MM5_20071009.nc
|-- ...
These files are structured as below (ncdump -h):
------------------------------------------------
netcdf MM5_20071009 {
dimensions:
lon = 440 ;
lat = 185 ;
time = UNLIMITED ; // (41 currently)
variables:
float lon(lon) ;
lon:long_name = "longitude" ;
lon:units = "degree_east" ;
float lat(lat) ;
lat:long_name = "latitude" ;
lat:units = "degree_north" ;
double time(time) ;
time:long_name = "seconds since the
origin date" ;
time:time_origin = "01-JAN-0000
00:00:00" ;
time:units = "seconds" ;
float uwd(time, lat, lon) ;
uwd:long_name = "10-meter U Component" ;
uwd:units = "m/s" ;
float vwd(time, lat, lon) ;
vwd:long_name = "10-meter V Component" ;
vwd:units = "m/s" ;
float pmer(time, lat, lon) ;
pmer:long_name = "Sea level pressure" ;
pmer:units = "Pa" ;
float rad(time, lat, lon) ;
rad:long_name = "solar radiation flux" ;
rad:units = "w/m^2" ;
float sat(time, lat, lon) ;
sat:long_name = "2-meter Temperature" ;
sat:units = "C degrees" ;
float rain(time, lat, lon) ;
rain:long_name = "precipitation during
one period" ;
rain:units = "kg/m^2" ;
float hflw(time, lat, lon) ;
hflw:long_name = "long wave heat flux" ;
hflw:units = "w/m^2" ;
float rh(time, lat, lon) ;
rh:long_name = "Relative humidity" ;
rh:units = "%" ;
// global attributes:
:Conventions = "COARDS" ;
:host = "ftp.unidata.ucar.edu" ;
:directory = "pub/netcdf/Conventions/COARDS" ;
:title = "MM5 version 3 format output on sigma levels" ;
:history = "Tue Oct 9 04:46:11 2007: ncrename -d
j_cross,lon final.nc\n",
"Tue Oct 9 04:46:11 2007: ncrename -d
i_cross,lat final.nc\n",
"Tue Oct 9 04:46:11 2007: ncatted -O -a
history,global,o,c, final.nc\n",
"" ;
:institution = "CETEMPS - University of L\'Aquila,
Italy" ;
:source = "NCAR MM5 model version 3" ;
:references = "http://www.mmm.ucar.edu/mm5/mm5-home.html";
;
:comment = "No comments yet" ;
I can access the dataset with the web browser:
----------------------------------------------
TestDatasetFmrc
Forecast Model Run Collection (2D time coordinates)
Best Time Series
Forecast Model Run
Constant Forecast Offset
Constant Forecast Date
But when I try to get any of the proposed datasets, it results in:
------------------------------------------------------------------
"Etat HTTP 400 - NULL message java.lang.NullPointerException" in the
browser
And in the threddsServlet.log:
------------------------------
2007-10-09T14:31:42.373 +0200 [ 3177637][ 45] ERROR -
thredds.servlet.ServletUtil - handleException
java.lang.NullPointerException
at thredds.servlet.ViewServlet$IDV.getViewerLinkHtml(ViewServlet.java
:212)
at thredds.servlet.ViewServlet.showViewers(ViewServlet.java:159)
at thredds.servlet.CatalogServicesServlet.showDataset(
CatalogServicesServlet.java:324)
at thredds.servlet.CatalogServicesServlet.handleCatalogServiceRequest(
CatalogServicesServlet.java:201)
at thredds.servlet.CatalogServicesServlet.doGet(
CatalogServicesServlet.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:178)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:432)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
I can't see what's going wrong...
Thanks for your help.
Jonathan
thredds archives: