[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #VFX-630532]: Writing multiple variables at once using NetCDF4 parallel

> Unfortunately that does not change anything... I tried to include
> netcdf.h after netcdf_apr it doesn't change anything.
> 
> I tried not including netcdf.h but then the code does not compile at all
> as it needs thnings likeNC_CLOBBER etc...

Sorry for the bad advice.  I was hoping it was something simple, but now 
I see even the minimal documentation about netcdf_par.h is wrong, it 
obviously can't be used instead of netcdf.h.

I'll have to reproduce the problem here with your test_append.c and see
if I can get it working. Unfortunately our parallel I/O expertise is gone, so
it may take some time.

> Any other idea?

I'm not sure what you intend with nc_redef() and nc_enddef().
There is no need for either nc_redef() or nc_enddef() calls for 
netCDF-4 files, unless you're setting special variable properties 
such as compression. Quoting from the docs:

  For netCDF-4 files (i.e. files created with NC_NETCDF4 in the cmode in their 
  call to nc_create()), it is not necessary to call nc_redef() unless the file 
was 
  also created with NC_STRICT_NC3. For straight-up netCDF-4 files, 
  nc_redef() is called automatically, as needed.
    ...
  It's not necessary to call nc_enddef() for netCDF-4 files. With netCDF-4 
files, 
  nc_enddef() is called when needed by the netcdf-4 library. User calls to 
  nc_enddef() for netCDF-4 files still flush the metadata to disk.
    ...
  For netCDF-4/HDF5 format files there are some variable settings (the 
  compression, endianness, fletcher32 error correction, and fill value) which 
  must be set (if they are going to be set at all) between the nc_def_var() and 
  the next nc_enddef(). Once the nc_enddef() is called, these settings can no 
  longer be changed for a variable.

So, you might also try just removing the nc_enddef() and nc_redef() calls, 
though I don't want to send you on another wild goose chase. I'll get
around to testing that here soon ...

--Russ

> On 06/03/2015 09:55 AM, Unidata netCDF Support wrote:
> > Hi Charles,
> >
> >> I just set out to port our software to use NetCDF4 parallel (Netcdf 
> >> 4.3.3.1 HDF5 1.8.13)
> >>
> >> I’m trying to have multiple core wrote to different variables? Is that 
> >> possible, or can you only write to ONE variable at a time.
> >>
> >> When running the attached code when ONE core, it works happily, when 
> >> running with multiple core (mpirun –n 2 or more) only the variables 
> >> written by the highest rank are actually in the file.
> >>
> >> Is it possible to achieve what I’m trying to do? If so what am I doing 
> >> wrong?
> > Yes, I think it is.  One thing you're doing wrong is including netcdf_par.h 
> > after netcdf.h.
> > According to a note in some release notes that doesn't seem to have made it 
> > into
> > the actual documentation:
> >
> >   Users of parallel I/O with netCDF-4 please note: starting with the 4.1.2 
> > release the parallel I/O functions are prototyped in netcdf_par.h, not 
> > netcdf.h. You must include netcdf_par.h BEFORE netcdf.h to use parallel I/O 
> > with netCDF-4.
> >   http://www.unidata.ucar.edu/netcdf/release-notes-4.1.2.html
> >
> > That also seems to be in the 2012 netCDF workshop section on using parallel 
> > I/O:
> >
> >   For parallel builds you must include "netcdf_par.h" before (or instead 
> > of) netcdf.h.
> >   
> > http://www.unidata.ucar.edu/netcdf/workshops/2012/pnetcdf/BuildingParallel.html
> >
> > I don't know if that's the problem, but it's the first problem I spotted, 
> > so you might want
> > to try just deleting the "#include <netcdf.h>" statement from your 
> > test_append.c.
> >
> > --Russ
> >> Also I tried to stay in def mode only first (I.e. Two loop, one declaring 
> >> all variables, then move in redef mode and write variables in 2nd loop) 
> >> but it didn’t make a difference.
> >>
> >> Thanks for any pointer,
> >>
> >> C.
> >>
> >>
> >>
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: VFX-630532
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> >
> 
> 
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: VFX-630532
Department: Support netCDF
Priority: Normal
Status: Closed