Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

[thredds] Java heap problems with large ASCII downloads

Hi,
I use thredds as opendap server for some large netcdf-files (CF-1.0, size > 1.1GB). When selecting to download the whole file with the 'Get ASCII' or 'GET BINARY', thredds crashes with a 'Java heap error'.
I'm currently using a default memory setup, i.e. -Xmx512MB for tomcat. How much memory will I need to avoid this problem. Does DODS read the whole file at once or variable after variable, or even slice after slice? Or are there some adjustments I can make to reduce memory consumption? 

Best regards,

Heiko




Details:
DODServlet ERROR (anyExceptionHandler): java.lang.OutOfMemoryError: Java heap space 
ReqState:
  serverClassName:    'thredds.server.opendap.NcDODSServlet'
  dataSet:            'XXXX.nc'
  requestSuffix:      'ascii'
  CE:                 ''
  compressOK:          true
  InitParameters:
    DebugOn: ''
    maxNetcdfFilesCached: '10'

java.lang.OutOfMemoryError: Java heap space
        at ucar.nc2.N3raf.readData(N3raf.java:77)
        at ucar.nc2.N3iosp.readData(N3iosp.java:159)
        at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:1374)
        at ucar.nc2.Variable._read(Variable.java:923)
        at ucar.nc2.Variable.read(Variable.java:618)
        at thredds.server.opendap.NcSDArray.read(NcSDArray.java:102)
        at thredds.server.opendap.NcSDGrid.read(NcSDGrid.java:59)
        at opendap.servlet.AsciiWriter.writeAsc(AsciiWriter.java:95)
        at opendap.servlet.AsciiWriter.toASCII(AsciiWriter.java:56)
at opendap.servlet.AbstractServlet.doGetASC(AbstractServlet.java:1029) 
        at opendap.servlet.AbstractServlet.doGet(AbstractServlet.java:1630)
at thredds.server.opendap.NcDODSServlet.doGet(NcDODSServlet.java:269) 
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:698)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
...



Thredds-Version:
Built-By: caron
Built-On: 2008-02-20 22:56:05
Implementation-Title: THREDDS Data Server
Implementation-Version: 3.16.33
Implementation-Vendor: UCAR/Unidata

Running on tomcat 6.0.16 with jdk.1.6.0_04 on debian 4.0.

--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY
  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: