Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [netcdfgroup] NetCDF4 MPI-IO and compression

That makes sense. -- Rob

On Feb 13, 2008, at 2:03 PM, Ed Hartnett wrote:

Rob Ross <rross@xxxxxxxxxxx> writes:

In defense of the HDF team, this is difficult capability to enable in
practice. This is because prior to compression you don't know how much
space a given write will require, so parallelizing writes (especially
if they aren't performed collectively) is not straightforward and
requires a significant amount of coordination.

Rob


Also I was mistaken when I wrote that parallel I/O and the deflate
filter can't be used together. This is only true for writing the data,
not reading it.

So you can read compressed data in parallel.

Thanks,

Ed
--
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx




  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: