2008 Unidata NetCDF Workshop for Developers and Data Providers > Best Practices
8.7 Missing Data Values
To indicate that data values are missing, invalid, or not written,
special values are conventionally used.
Since netCDF data can be written in random order, there may be values
that are not written, either intentionally or inadvertently.
To detect attempts to later read unwritten data,
the netCDF library initializes data with either
- the value of a variable's
_FillValue attribute,
if it exists, or else
- a type-specific default fill value, defined as part of the
netCDF interface, or
- nothing, leaving the value undetermined, if the writer sets "no
fill" mode.
Recommendations
- Unless you have a good reason, use the default fill values for
each type, because writing, reading, and
testing for equality with these default fill values works
portably on the platforms on which netCDF has been tested.
- Alternatively, for floating-point data consider using an IEEE
NaN (Not a Number) value, because any computation
using a NaN results in a NaN. Client software must know to look for NaNs,
however, and detection of NaNs may be tricky, since any comparison
with a NaN is required to return false.
- Alternatively or in addition, set the valid_range attribute for each
variable that uses missing values, and make sure all valid data is within
that range, and all missing or invalid data is outside of that range. Again,
the client software must recognize and make use of this information. Example:
variables:
float rel_humidity( z, y, x);
rel_humidity:units = "percent";
rel_humidity:valid_range = 0.0f, 1.0f;
- The
_FillValue attribute, if used, should have the
same data type as the variable
it describes. If the variable is packed into a narrower type,
for example with scale_factor and add_offset, _FillValue should be the same as
the packed type.
- Turning on "no fill" mode can make writes faster, but this may
be an unwise optimization unless you know you will write all data
values: it prevents later detection of unwritten data.
2008 Unidata NetCDF Workshop for Developers and Data Providers > Best Practices
