[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #VQZ-726653]: data packing

Sue,

Good questions.

Hi Sue,

The formulas were presented in terms of unsigned integer ranges rather
signed bytes, shorts, and ints, because the description of packing was
supposed to be abstract and language neutral, but I can see why it
might be more useful to have a description of how it's actually
implemented in specific languages.

C has unsigned short and unsigned char types, that can be used in C or
C++ implementations.  On the other hand, the byte and short types for
netCDF data are considered signed for the purpose of type conversion,
so it might be better to present the formulas assuming signed bytes
and shorts.  However when you read netCDF values of byte or short type
directly into variables of the appropriate type, using
nc_get_vara_uchar() or nc_get_vara_short(), no type conversion takes
place and you can treat the values as unsigned.  The same is true for
the relevant "put" functions.

But we'll consider adapting the formulas to use signed ranges, as you
suggest, since it might make this clearer to developers who have to
implement the code.  Eventually we may be able to do what has been
done in the Java interface, handling packing and unpacking in the
library layer, though for C-based interfaces it may be the
higher-level libcf rather than the netCDF library.  Some of the issues
in packing and missing values are described in this thread of
discussion that took place on the netcdfgroup mailing list:

  
http://conan.unidata.ucar.edu/support/help/MailArchives/netcdfgroup-list/msg00177.html

To answer your specific question about fill values, we recommend using
a value at either end of the packed range for the fill value, and
using the (external on-disk) packed data type of the variable for the
netCDF type of the _FillValue attribute.  The default fill value for
shorts and ints already qualifies.  There is no default fill value
for bytes, so in this case you have to choose a value.

The NCO software includes code to handle packing and unpacking automatically,
so you might want to look at how it's done there.  I think NCL may also handle
this automatically, but I'm not sure.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: VQZ-726653
Department: Support netCDF
Priority: Normal
Status: Closed