Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

Re: Defining large NetCDF variables midway through an MPI program

Hello all,

Does anyone have any experience in reading and defining new NetCDF variables midway through a multiple processor MPI program? I've experienced a very strange bug: I have a MPI program where each processor reads in data from multiple NetCDF variables. If a variable is missing, one of the processors then defines the new variable while the other processors halt. Once the variable is defined, all of the processors move forward in the program. If the newly defined variable is small, this works with no problems. However, if the definition of the new variables takes a considerable amount of time, then the other processors actually _forget_ the data they previously read in. How weird is that? I am using the NO_Fill mode to define the variables, but it still takes a long time. The variable could be huge, though (500+ MB).

There's an easy-to-implement solution (define first, then read), but I was curious if anyone has had a similar problem and what causes it?


Thanks,

  Howard Salis
  Dept Chemical Eng. & Materials Science
  University of Minnesota


 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690