Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

[decoders] Large GRIB files

NOTE: The decoders mailing list is no longer active. The list archives are made available for historical reasons.

Hi,

I'm trying to read a large GRIB file using the java decoder, but I'm
getting the following exception:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
 at java.lang.StringBuffer.append(StringBuffer.java:320)
at ucar.grib.grib1.Grib1Input.seekHeader(Grib1Input.java:247)
 at ucar.grib.grib1.Grib1Input.scan(Grib1Input.java:129)

Since the grib file has 3.8Gb, the exception is thrown when the class tries
to load the whole file to the memory.

Is there a way to seek the grib headers in-disk?

I'm a newbie in the grib file format, but looking at the source code for
GribInput, the seekHeader method loads a bunch of bytes into the memory and
then look for the header pattern. Is this load really necessary?

Att,

-- 
Ederson Ferreira de Jesus
Matemático Aplicado - UNICAMP
"Time is an illusion. Lunchtime doubly so."
Adams, Douglas
  • 2011 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the decoders archives: