[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: HDF metadata



On 11/22/2010 3:36 PM, Robert B. Schmunk wrote:
John,

A panoply user has complained that soem variable metadata in an HDF
file is not being reported. I haven't been able to figure it out,
and since Panoply is getting the info via the NJ libraries, I thought
I'd pass this on to see if you have any ideas.

The sample problem dataset is at
ftp://ftp.nodc.noaa.gov/pub/data.nodc/crw/CRW50km/all_in_one/2001/sst.coralreef.fields.50km.n14.20010106.hdf

Looking at, for example, the variable CRW_HDW, there is critical
difference in how the NJ libraries and Panoply report the attributes
and how the HDFview application reports the attributes. Essentially,
NJ/Panoply are missing five attributes:

       _FillValue = -7777
       scale_factor = 0.01
       missing_value = -7777
       add_offset = 0.0
       valid_range = 0,10000

What NJ/Panoply report is:

  double CRW_DHW(latitude=331, longitude=720);
      :long_name = "Twice-weekly global 50km satellite coral bleaching Degree 
Heating Weeks";
      :units = "degree Celsius-weeks";
      :format = "I6";
      :inputdata_type = "16-bit (2-byte) integer";
      :coordsys = "geographic";
      :scale_factor_err = 0.0; // double
      :add_offset_err = 0.0; // double
      :calibrated_nt = 24; // int
      :C_format = "%6d";
      :fraction_digits = 1; // int
      :variable_info = "The values in this variable array and in the valid_range 
attribute are in the units of x100 degree Celsius-weeks. The data values should be 
multiplied by the value (0.01) contained in the scale_factor attribute to obtain the 
actual values in the units of degree Celsius-weeks.";
      :image_info_variable = "The fourth browse image is for bleaching Degree 
Heating Weeks.";
      :palette_info_variable = "The fourth palette, containing 256 indexed colors 
(0-255), is for CRW Degree Heating Weeks (DHW). To plot CRW DHW data using this palette, 
DHW values in the range of [0,16] are matched to color 0-255 using color_index=DHW*255/16 
rounded to the nearest integer. Any color index from the conversion that is greater than 
252 is set to 252. Negative and zero index values at valid water pixels should be changed 
to 1. Land, ice, and missing data pixels flagged in surface_flag array should be assigned 
to 253 (white), 254 (black), and 255 (gray), respectively.";

And what HDFview program reports is:

CRW_DHW
   16-bit integer,    331 x 720
   Number of attributes = 18
       long_name = Twice-weekly global 50km satellite coral bleaching Degree 
Heating Weeks
       units = degree Celsius-weeks
       format = I6
       inputdata-type = 16-bit (2-byte) integer
       coordsys = geographic
       _FillValue = -7777
       scale_factor = 0.01
       scale_factor_err = 0.0
       add_offset = 0.0
       add_offset_err = 0.0
       calibrated_nt = 24
       C_format = %6d
       missing_value = -7777
       fraction_digits = 1
       valid_range = 0,10000
       variable_info = The values in this variable array and in the valid_range 
attribute are in the units of x100 degree Celsius-weeks. The data values should 
be multiplied by the value (=0.01) contained in the scale_factor attribute to  
obtain the actual values in the units of degree Celsius-weeks.
       image_info_variable = The fourth browse image is for bleaching Degree 
Heating Weeks.
       palette_info_variable = The fourth palette, containing 256 indexed 
colors (0-255), is for CRW DHW. To plot CRW DHW data using this palette, DHW 
values in the range of [0,16] are matched to colors 1-252 using 
color_index=DHW*255/16 rounded to the nearest integer. Any color index from the 
conversion that is greater than 252 is set to 252. Negative and zero index 
values at valid water pixels should be changed to 1. Land, ice, and missing 
data pixels flagged in surface_flag array should be assigned to 253 (white), 
254 (black), and 255 (gray), respectively.


Any ideas?

Thanks,
rbs


--
Robert B. Schmunk, address@hidden
NASA Goddard Institute for Space Studies, 2880 Broadway, New York, NY 10025

Hi Robert:

If you look at in ToolsUI Viewer, you see:

  short CRW_DHW(latitude=331, longitude=720);
     :_FillValue = -7777S; // short
     :scale_factor = 0.01; // double
     :scale_factor_err = 0.0; // double
     :add_offset = 0.0; // double
     :add_offset_err = 0.0; // double
     :calibrated_nt = 24; // int
     :missing_value = -7777S; // short
     :fraction_digits = 1; // int
     :valid_range = 0S, 10000S; // short

(Im not showing the string attributes)

then, when you enhance it, it turns it into a double by applying the scale/offset, and the non-string attributes are:

  double CRW_DHW(latitude=331, longitude=720);
     :scale_factor_err = 0.0; // double
     :add_offset_err = 0.0; // double
     :calibrated_nt = 24; // int
     :fraction_digits = 1; // int

it removes scale/offset attributes so they they dont get applied again. the missing values are converted to NaNs, and those are also removed since they dont make sense anymore now that the variables is a double.

If you are using the GeoGrid interface, we always enhance.


John