[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Scale factor in netCDF (fwd)

------- Forwarded Message

Date:    Tue, 15 Nov 2005 18:29:16 -0800
From:    "Steve Kopp" <address@hidden>
To:      "Russ Rew" <address@hidden>
Subject: RE: Scale factor in netCDF 


Thanks Russ, this is great, I'll get with Ajit in the morning on this. And I 
feel your pain about the transition from FrameMaker.  We went from FrameMaker 
to PageMaker, and now onto a homegrown online content system, and its about the 
least fun project I can imagine.

Thanks,

Steve

-----Original Message-----
From: Russ Rew [mailto:address@hidden] 
Sent: Tuesday, November 15, 2005 6:11 PM
To: Steve Kopp
Subject: Re: Scale factor in netCDF 

Hi Steve,

> We're implementing support for scale and offset to our netCDF
> support and have come across a problem that leads to a question.
>  
> How/where is the type of the scale factor stored and interpreted? 
> We don't see a type, such as "double" in the CDL for scale
> factor. However, when we programmatically get the scale factor, its
> attribute type is returned as double, even though for the dataset we
> are looking at the scale factor and resulting scaled value could
> easily be stored as a float. 

When you specify an attribute value with CDL, it's type is derived
from the type of the value assigned to it.  So, for example, the
following CDL statements would create attributes of type double, float,
int, short, and byte:

  var:d = -1.0;  // double
  var:f = -1.0f; // float
  var:i = -1;    // int
  var:s = -1s;   // short
  var:b = -1b;   // byte

This information and the syntax for constants are in the section of
the User's Guide under "CDL Syntax" and "CDL Data Types" in a chapter
where you might not expect them, "NetCDF Utilities":

  http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/CDL-Syntax.html
  http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/CDL-Data-Types.html

The best documentation on use of the scale_factor and add_offset is
under the "Packed Data Values" section of the "Writing NetCDF Files:
Best Practices" document, at:

  
http://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html#Packed%20Data%20Values
 
> Can you help us understand what's going on internally so we can
> decide what to do?  Is this always double regardless of the size of
> the value it needs to store? 

No, as documented in the Common Attributes section of the User's Guide
and in the Best Practices document above, the type of the scale_factor
and add_offset attributes should be the type that you want the
unpacked data to be, typically float or double.

Note the bullet about using rounding rather than truncating when you
are packing data values, it avoids a subtle bias that might otherwise
be introduced by packing.

> The problem this results in, is that if someone wants to do analysis
> on their netCDF as a raster in ArcGIS, we convert it to our GRID
> format, which is currently only 32 bit.  So if all data with a scale
> factor thinks its 64 bit, this could mean that you could not do
> raster analysis on netCDF data that was 64 or that had a scale
> factor.  Our only other option at this point would be to pretend any
> double scale factor really only needs to be a float, then the
> conversion and analysis will work fine for 32 bit data and data with
> scale factor, but then actual 64 bit values would not even display
> properly.

Please let us know if the above doesn't clarify things.  Someday,
we'll get the documentation organized so that things like this are
easier to find.  We lost most of our good indices in converting the
documents from FrameMaker to texinfo ...

--Russ


------- End of Forwarded Message