Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [netcdfgroup] NetCDF4 MPI-IO and compression

In defense of the HDF team, this is difficult capability to enable in practice. This is because prior to compression you don't know how much space a given write will require, so parallelizing writes (especially if they aren't performed collectively) is not straightforward and requires a significant amount of coordination. 

Rob

On Feb 13, 2008, at 12:07 PM, Ed Hartnett wrote:


Thomas Green <thomas.green@xxxxxxxxxxxxxxxx> writes:


NetCDF users,

Just a quick question.  Can NetCDF4 use MPI-IO if also using
compression? I read on the HDFgroup website that MPI-IO and compression 
are not compatible but I am not sure if this still applies and how it
affects NetCDF4.

The HDFgroup FAQ related to MPI-IO and compression:
http://www.hdfgroup.com/hdf5-quest.html#p5comp

Many thanks,

Tom



No, sorry, that is not allowed by HDF5.

If you want parallel I/O you must use uncompressed data.

I have asked the HDF5 team whether this might be changed in a future
HDF5 release...

Ed

--
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx
_______________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/
  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: