[thredds] NetCDF GridSubsetService Out of memory errors

[Roy Mendelssohn]

What are the settings on your java virtual machine that the tomcat is  
running under?  You probably need to increase the heap available to  
the Tomcat and then do a shutdown and startup of the tomcat.

-Roy M.

On Oct 29, 2007, at 7:15 PM, Trevor.Daniels at csiro.au wrote:

>
> Hi,
>
> I'm getting out of memory errors trying to aggregate some rainfall  
> data
> files (each 2.5Mb) using the netcdf grid subsetting service in thredds
> (3.16.19) with the following..
>
> <dataset name="Rainfall" ID="RainAggregated"
> urlPath="content/bilo/aggregatedData.nc">
>  <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ 
> ncml-2.2">
>         <aggregation dimName="time" type="joinNew">
>           <variableAgg name="rain"/>
>           <scan location="c:\data\bilo\2007\" suffix=".nc"
> dateFormatMark="#yyyyMMdd" />
>         </aggregation>
>       </netcdf>
>
> Generally the aggregation works for producing outputs of up to about
> 50Mb, whereby anything greater generates the following stack trace on
> multiple pc's...
>
> Oct 29, 2007 3:34:48 PM org.apache.catalina.core.StandardWrapperValve
> invoke
> SEVERE: Servlet.service() for servlet GridSubsetService threw  
> exception
> java.lang.OutOfMemoryError: Java heap space
>  at ucar.ma2.ArrayFloat.<init>(ArrayFloat.java:86)
>  at ucar.ma2.ArrayFloat$D3.<init>(ArrayFloat.java:291)
>  at ucar.ma2.ArrayFloat$D3.<init>(ArrayFloat.java:283)
>  at ucar.ma2.ArrayFloat.factory(ArrayFloat.java:50)
>  at ucar.ma2.ArrayFloat.factory(ArrayFloat.java:36)
>  at ucar.ma2.Array.factory(Array.java:130)
>  at ucar.ma2.Array.factory(Array.java:79)
>  at ucar.nc2.ncml.Aggregation.read(Aggregation.java:666)
>  at ucar.nc2.dataset.VariableDS._read(VariableDS.java:277)
>  at ucar.nc2.Variable.read(Variable.java:603)
>  at ucar.nc2.Variable.read(Variable.java:556)
>  at ucar.nc2.FileWriter.copySome(FileWriter.java:287)
>  at ucar.nc2.FileWriter.copyVarData(FileWriter.java:225)
>  at ucar.nc2.FileWriter.finish(FileWriter.java:459)
>  at ucar.nc2.dt.grid.NetcdfCFWriter.makeFile(NetcdfCFWriter.java:198)
>  at thredds.server.ncSubset.GridServlet.sendFile(GridServlet.java:395)
>  at
> thredds.server.ncSubset.GridServlet.processGrid(GridServlet.java:354)
>  at thredds.server.ncSubset.GridServlet.doGet(GridServlet.java:148)
>  at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>  at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
>  at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
> (Applica
> tionFilterChain.java:290)
>  at
> org.apache.catalina.core.ApplicationFilterChain.doFilter 
> (ApplicationFilt
> erChain.java:206)
>  at
> org.apache.catalina.core.StandardWrapperValve.invoke 
> (StandardWrapperValv
> e.java:233)
>  at
> org.apache.catalina.core.StandardContextValve.invoke 
> (StandardContextValv
> e.java:175)
>  at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke 
> (Authenticator
> Base.java:433)
>  at
> org.apache.catalina.core.StandardHostValve.invoke 
> (StandardHostValve.java
> :128)
>  at
> org.apache.catalina.valves.ErrorReportValve.invoke 
> (ErrorReportValve.java
> :102)
>  at
> org.apache.catalina.core.StandardEngineValve.invoke 
> (StandardEngineValve.
> java:109)
>  at
> org.apache.catalina.connector.CoyoteAdapter.service 
> (CoyoteAdapter.java:2
> 63)
>  at
> org.apache.coyote.http11.Http11Processor.process 
> (Http11Processor.java:84
> 4)
>  at
> org.apache.coyote.http11.Http11Protocol 
> $Http11ConnectionHandler.process(
> Http11Protocol.java:584)
>  at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java: 
> 447)
> 	
> Any suggestions on how to resolve or further diagnose?
>
> Thanks for help,
>
> Trevor
>
> _______________________________________________
> thredds mailing list
> thredds at unidata.ucar.edu
> For list information or to unsubscribe,  visit: http:// 
> www.unidata.ucar.edu/mailing_lists/

**********************
"The contents of this message do not reflect any position of the U.S.  
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division	
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."