[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 19990711: extra header space

>To: address@hidden
>From: Michael Nolta <address@hidden>
>Subject: extra header space
>Organization: Princeton
>Keywords: 199907110517.XAA12464 netCDF

Mike,

> Sorry, my note was a bit muddled. Perhaps a few crude diagrams will help.
> If I comment out the section adding the "title" attribute, I get a file
> that looks like this (not to scale ;-):
> 
>       **000000000000000000abcdefghijklmnopqrstuvwxyz
> 
> where * is the header information, 0 is blank, and abc... is the data.
> Writing "title" gives me a file that looks like this:
> 
>       ********abcdefghijklmnopqrstuvwxyzopqrstuvwxyz
> 
> What I expected to get was:
> 
>       ********000000000000abcdefghijklmnopqrstuvwxyz
> 
> i.e., "title" is written into the blank space and the data is left
> untouched. What appears to happen is that the header expands and then the
> data is written a second time immediately preceding the (non-blank) header
> info.
> 
> Basically, what I'm trying to do is add some room for expansion to the
> header, so I can add to it later and avoid copying the data section.
> 
> I'm running version 3.4 on SunOS 5.6.

Thanks, that's a very clear explanation of the symptoms.

You have uncovered a bug in nc__enddef().  

The library is moving the fixed-size variable around unnecessarily,
but resetting the offset value in the header that points to the
beginning of the variable data so that it still points to the
beginning of the moved data.  Since the number of elements in the
variable is correct in the header, ncdump and other programs that just
use the netCDF interface read the correct values, but the netCDF file
has extraneous junk after the variable (so this is also a bug that can
make netCDF files larger than they should be).

It appears to be the kind of bug that does extra work, but doesn't
corrupt the data.  However, I still regard this as serious, because it
can result in unexpectedly poor performance.

I'm still looking at the situations under which the bug occurs.  I
think the problem is that nc_close() calls the 1-argument nc_enddef()
instead of the 5-argument nc__enddef() in this case, which causes the
reserved space in the header to be closed up again, but that may not
be the whole story.  I need to understand what's going on better
before I can provide a fix, or even a workaround.  I intend to fix
this before we release netCDF version 3.5.

Thanks again for the bug report.

--Russ

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu