[thredds] Java heap problems with large ASCII downloads

Hi,

I use thredds as opendap server for some large netcdf-files (CF-1.0, size > 1.1GB). When selecting to download the whole file with the 'Get ASCII' or 'GET BINARY', thredds crashes with a 'Java heap error'.

I'm currently using a default memory setup, i.e. -Xmx512MB for tomcat. How much memory will I need to avoid this problem. Does DODS read the whole file at once or variable after variable, or even slice after slice? Or are there some adjustments I can make to reduce memory consumption?

Best regards,

Heiko




Details:

DODServlet ERROR (anyExceptionHandler): java.lang.OutOfMemoryError: Java heap space
ReqState:
  serverClassName:    'thredds.server.opendap.NcDODSServlet'
  dataSet:            'XXXX.nc'
  requestSuffix:      'ascii'
  CE:                 ''
  compressOK:          true
  InitParameters:
    DebugOn: ''
    maxNetcdfFilesCached: '10'

java.lang.OutOfMemoryError: Java heap space
        at ucar.nc2.N3raf.readData(N3raf.java:77)
        at ucar.nc2.N3iosp.readData(N3iosp.java:159)
        at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:1374)
        at ucar.nc2.Variable._read(Variable.java:923)
        at ucar.nc2.Variable.read(Variable.java:618)
        at thredds.server.opendap.NcSDArray.read(NcSDArray.java:102)
        at thredds.server.opendap.NcSDGrid.read(NcSDGrid.java:59)
        at opendap.servlet.AsciiWriter.writeAsc(AsciiWriter.java:95)
        at opendap.servlet.AsciiWriter.toASCII(AsciiWriter.java:56)
at opendap.servlet.AbstractServlet.doGetASC(AbstractServlet.java:1029)
        at opendap.servlet.AbstractServlet.doGet(AbstractServlet.java:1630)
at thredds.server.opendap.NcDODSServlet.doGet(NcDODSServlet.java:269)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:698)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
...



Thredds-Version:
Built-By: caron
Built-On: 2008-02-20 22:56:05
Implementation-Title: THREDDS Data Server
Implementation-Version: 3.16.33
Implementation-Vendor: UCAR/Unidata

Running on tomcat 6.0.16 with jdk.1.6.0_04 on debian 4.0.

--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY


  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: