[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #HSK-896839]: netCDF static lib



Hi Vitaliy,

Sorry to have taken so long to respond to your question:
> I've the following problem :
> 
> - unlimited dimension variable of 10 Gb that I need to read and filter its
> data in accoding to some criteria, and then save the results in new 10 Gb
> variable in new netcdf file.
> 
> I'm doing all on my desktop PC and limited in memory (3Gb only)
> 
> so I have exceptions when doing allocation(unlim_var(dim1_len, unlim_dimlen)
> 
> is there solution to handle such a case ?

A solution is to read the data one record at a time, instead of trying to read
it all into memory at once.  Then you only need to allocate enough memory to
hold one record's worth of data at a time.  That's what the nccopy utility that
comes with the netCDF C software does to be able to copy, convert, compress, 
chunk, or rechunk any netCDF file, no matter how large, without requiring 
memory needed to hold all of a variable in memory.

--Russ


Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: HSK-896839
Department: Support netCDF
Priority: Critical
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.