Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

Re: [netcdfgroup] Wondering about when NetCDF data hits the disk...

Hi,

Am Wed, 28 Oct 2009 10:43:42 -0600
schrieb Russ Rew <russ@xxxxxxxxxxxxxxxx>:

> Just out of curiosity, did you try using the NF90_SHARE flag on the
> create/open calls in the writing and reading processes to see if that
> makes any difference?

I did not and it's already quite late or me here, but I am assured that it 
would not make a difference. The NF90_SHARE semantics are fine for processes on 
one machine. Without considering multiple machines accessing one file system 
(be it NFS or whatnot), it does not matter if the file is "written to disk"... 
since everyone shares the same filesystem buffers.

> I will have to do some
> research to see whether avoiding fsync() was entirely a portability
> issue for use on non-Unix systems, or whether there was some other
> intention. 

> But for now, I agree that it looks like calling nc_sync()
> ought to also call fsync() on the underlying file descriptor.  If
> nc_sync() did that, there would be no need for a new nc_fsync()
> function.

Having the fiasco of Firefox' madness with its huge databases for trivial stuff 
like bookmarks in mind, we should think before making fsync() default on every 
write, or, rather on every explicit call to nc_sync(). The latter should be 
sane, IMHO, but one might like to be aware of funny effects like 
https://bugzilla.mozilla.org/show_bug.cgi?id=421482 .
But, well, causing disk I/O on a call to a data I/O library feels rather sane 
and is not as unexpected as causing disk I/O on every character typed into a 
URL field, or a click on a hyperlink...

> Thanks for bringing this issue to our attention.

Oh, glad to have helped to get you another issue to worry about;-)


Alrighty then,

Thomas.

-- 
Dipl. Phys. Thomas Orgis
Atmospheric Modelling
Alfred-Wegener-Institute for Polar and Marine Research


 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690