Re: [netcdf-java] [thredds] NetCDF GridSubsetService Out of memory errors

What are the settings on your java virtual machine that the tomcat is running under? You probably need to increase the heap available to the Tomcat and then do a shutdown and startup of the tomcat.

-Roy M.

On Oct 29, 2007, at 7:15 PM, Trevor.Daniels@xxxxxxxx wrote:


Hi,

I'm getting out of memory errors trying to aggregate some rainfall data
files (each 2.5Mb) using the netcdf grid subsetting service in thredds
(3.16.19) with the following..

<dataset name="Rainfall" ID="RainAggregated"
urlPath="content/bilo/aggregatedData.nc">
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ ncml-2.2">
        <aggregation dimName="time" type="joinNew">
          <variableAgg name="rain"/>
          <scan location="c:\data\bilo\2007\" suffix=".nc"
dateFormatMark="#yyyyMMdd" />
        </aggregation>
      </netcdf>

Generally the aggregation works for producing outputs of up to about
50Mb, whereby anything greater generates the following stack trace on
multiple pc's...

Oct 29, 2007 3:34:48 PM org.apache.catalina.core.StandardWrapperValve
invoke
SEVERE: Servlet.service() for servlet GridSubsetService threw exception
java.lang.OutOfMemoryError: Java heap space
 at ucar.ma2.ArrayFloat.<init>(ArrayFloat.java:86)
 at ucar.ma2.ArrayFloat$D3.<init>(ArrayFloat.java:291)
 at ucar.ma2.ArrayFloat$D3.<init>(ArrayFloat.java:283)
 at ucar.ma2.ArrayFloat.factory(ArrayFloat.java:50)
 at ucar.ma2.ArrayFloat.factory(ArrayFloat.java:36)
 at ucar.ma2.Array.factory(Array.java:130)
 at ucar.ma2.Array.factory(Array.java:79)
 at ucar.nc2.ncml.Aggregation.read(Aggregation.java:666)
 at ucar.nc2.dataset.VariableDS._read(VariableDS.java:277)
 at ucar.nc2.Variable.read(Variable.java:603)
 at ucar.nc2.Variable.read(Variable.java:556)
 at ucar.nc2.FileWriter.copySome(FileWriter.java:287)
 at ucar.nc2.FileWriter.copyVarData(FileWriter.java:225)
 at ucar.nc2.FileWriter.finish(FileWriter.java:459)
 at ucar.nc2.dt.grid.NetcdfCFWriter.makeFile(NetcdfCFWriter.java:198)
 at thredds.server.ncSubset.GridServlet.sendFile(GridServlet.java:395)
 at
thredds.server.ncSubset.GridServlet.processGrid(GridServlet.java:354)
 at thredds.server.ncSubset.GridServlet.doGet(GridServlet.java:148)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
 at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter (Applica
tionFilterChain.java:290)
 at
org.apache.catalina.core.ApplicationFilterChain.doFilter (ApplicationFilt
erChain.java:206)
 at
org.apache.catalina.core.StandardWrapperValve.invoke (StandardWrapperValv
e.java:233)
 at
org.apache.catalina.core.StandardContextValve.invoke (StandardContextValv
e.java:175)
 at
org.apache.catalina.authenticator.AuthenticatorBase.invoke (Authenticator
Base.java:433)
 at
org.apache.catalina.core.StandardHostValve.invoke (StandardHostValve.java
:128)
 at
org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java
:102)
 at
org.apache.catalina.core.StandardEngineValve.invoke (StandardEngineValve.
java:109)
 at
org.apache.catalina.connector.CoyoteAdapter.service (CoyoteAdapter.java:2
63)
 at
org.apache.coyote.http11.Http11Processor.process (Http11Processor.java:84
4)
 at
org.apache.coyote.http11.Http11Protocol $Http11ConnectionHandler.process(
Http11Protocol.java:584)
 at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java: 447)
        
Any suggestions on how to resolve or further diagnose?

Thanks for help,

Trevor

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http:// www.unidata.ucar.edu/mailing_lists/

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division 
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."