overcoming netcdf3 limits

[John Caron]

Hi Greg:

Can you send me the output from ncdump -h on a typical file ?

How does the parallel part work? do certain sections of the file get reserved for different nodes, or is there a master node that essentially serializes the writing?

Greg Sjaardema wrote:
> As a quick answer to the question, we (Sandia Labs) use netcdf
> underneath our exodusII
> file format for storing finite element results data.
> 
> If the mesh contains #nodes nodes and #elements elements, then there
> will be a dataset of the size #elements*8*4 (assuming a hex element with
> 8 nodes, 4 bytes/int) to store the nodal connectivity of each hex
> element in a group of elements (element block). Assuming 4GiB, this
> limits us to ~134 Million elements per element block (using CDF-2) which
> is large, but not enough to give us more than a few months breathing
> room.    Using CDF-1 format, we top out at about 30 million elements or
> less which is hit routinely.
> 
> There is a pdf file at
> http://endo.sandia.gov/SEACAS/Documentation/exodusII.pdf that shows
> (starting at page 177) how we map exodusII onto netcdf.  There have been
> some changes since the report was written to reduce some of the dataset
> sizes.  For example, we split the "coord" dataset into 3 separate
> datasets now and we also split the vals_nod_var into a single dataset
> per nodal variable.
> 
> --Greg
> 
> 
> John Caron wrote:
>> Hi Rob:
>>
>> Could you give use case(s) where the limits are being hit?
>> I'd be interested in actual dimension sizes, number of variables,
>> whether you are using a record dimension, etc.
>>
>> Robert Latham wrote:
>>> Hi
>>>
>>> Over in Parallel-NetCDF land we're running into users who find even
>>> the CDF-2 file format limitations, well, limiting.
>>> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/NetCDF-64-bit-Offset-Format-Limitations.html
>>>
>>>
>>> http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#Large%20File%20Support10
>>>
>>>
>>> If we worked up a CDF-3 file format for parallel-netcdf (off the top
>>> of my head, maybe a 64 bit integer instead of an unsigned 32 bit
>>> integer could be used to describe variables), would the serial netcdf
>>> folks be interested, or are you looking to the new netcdf-4 format to
>>> take care of these limits?
>>>
>>> Thanks
>>> ==rob
>>>
>> ==============================================================================
>>
>> To unsubscribe netcdfgroup, visit:
>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>> ==============================================================================
>>
>>
>>

==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================