[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #NVI-857322]: Re: [netcdfgroup] Wondering about when NetCDF data hits the disk...

Hi Thomas,

> OK, I tested the simple patch that inserts fsync().
> It does help the concurrent reading of data. No doubt.

Thanks for testing that.

> > 2. determine if there is a resulting performance penalty, and if so
> > whether you consider it acceptable or unacceptable.
> 
> Well, you of course can measure runtime differences, for example,
> largely irrelevant because you don't know what computations are
> behind that:
> 
> a run with little amount of data per snapshot
> 
> with fsync:
> 
> real 0m13.936s
> user 0m9.750s
> sys 0m0.255s
> 
> without fsync:
> real 0m10.389s
> user 0m9.758s
> sys 0m0.087s
> 
> another run with more data, also more computing because of that:
> 
> 
> without fsync:
> real 0m34.651s
> user 0m33.926s
> sys 0m0.188s
> 
> fsync:
> real 0m34.638s
> user 0m33.916s
> sys 0m0.177s
> 
> It simply boils down to measurable overhead with small amounts of
> data and less overhead with large amounts of data. That's the
> basic idea behind caching, so it's no surprise. People calling
> nc_synch() should not expect anything different.
> So I am voting for the inserted fsync() call.

My benchmarks showed similar results.  I also found the port to
Windows is very simple:
#ifdef __MSVCRT__
#define fsync _commit
#endif

so I'm leaning towards putting the fsync call into 4.1 and removing
the --enable-fsync option to configure, because I agree it's what
people calling nc_sync() would expect.  Whether we need a
--disable-fsync call for configure remains to be decided.

> PS: I noticed that nc-config --fflags under Solaris / Sun Studio
> does not add -M/prefix/include, which is needed for the Sun fortran
> compiler to find the Fortran 90 interface module, -I/prefix/include
> is not enough. This is not trivial, though, as other compilers (gcc)
> don't like -M .. so you cannot just add it always for Fortran (90)
> ... only when NetCDF has been built with Sun Studio -- or perhaps
> one can add a configure check for -M support?

It turns out to be a little more complicated than that.  The Solaris
flag should be -I/prefix/include when compiling an f77 source that has
an "include netcdf.inc" statement, but -M/prefix/include when
compiling an f90 program that has "use netcdf".  But --fflags doesn't
permit distinguishing between which kind of source is being compiled.
I guess we would really need an --f77flags in addition to --fflags to
get this right.  But for now, you'll see that nc-config --fflags will
use the -M/prefix/include in the snapshot and later releases.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: NVI-857322
Department: Support netCDF
Priority: Normal
Status: Closed