Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
tl;dr There appears to be a bug in NetCDF Java with respect to reading linked HDF4 files which results in data that is read from the linked file(s) to be erroneous. Resources * ToolsUI * HDFView * The files mentioned below can be retrieved directly from OceanColor<http://oceancolor.gsfc.nasa.gov/cgi/l3/A20021822013212.L3b_MC_RRS.main.bz2?sub=bin> (one at a time), or (for convenience) as one tar.gz file fromhere<https://drive.google.com/uc?id=0B6UT7Mn4GZQhMjdLNDBBMFE0TTA&export=download> Details I'm reading data from the Aqua MODIS L3 Binned products available from the NASA OceanColor<http://oceancolor.gsfc.nasa.gov/> website. It should be noted that these files are HDF4 (4.2.9 according to NetCDF Java - ncdump). Many of the products, such as chlorophyll, Particulate Inorganic Carbon, and Sea Surface Temperature, come as a single file. The NetCDF library reads these files without any difficulty. However, one of the datasets of interest is the Remote Sensing Reflectance data, which is NOT provided as a single file, but as a "main" file and a set of subordinate files which are read via the "main" file as needed (see here for more information<http://oceancolor.gsfc.nasa.gov/PRODUCTS/modis_binned.html>): * A20021822013212.L3b_MC_RRS.main * A20021822013212.L3b_MC_RRS.x00 * A20021822013212.L3b_MC_RRS.x01 * A20021822013212.L3b_MC_RRS.x02 * A20021822013212.L3b_MC_RRS.x03 * A20021822013212.L3b_MC_RRS.x04 * A20021822013212.L3b_MC_RRS.x05 * A20021822013212.L3b_MC_RRS.x06 * A20021822013212.L3b_MC_RRS.x07 * A20021822013212.L3b_MC_RRS.x08 * A20021822013212.L3b_MC_RRS.x09 * A20021822013212.L3b_MC_RRS.x10 * A20021822013212.L3b_MC_RRS.x11 NetCDF Java (via ToolsUI) loads the .main file without issue, and permits reading of data variables (i.e. Rrs_412) without raising any errors. However, the data returned is not accurate. Below is a comparison of the data returned by ToolsUI and the same data returned by HDFView (which uses the HDF-java JNI<http://www.hdfgroup.org/products/java/JNI/> library): Retrieving the first 10 values for variable "Rrs_412" HDFView Screen Capture<http://cl.ly/WZnD> Opening the .main file in HDFView and looking at the Rrs_412 dataset gives a very different set of data: 0.0055423053, 0.0106070135, 0.006894292, -0.0040368317, -0.0020879991, -0.0020279996, 0.009794002, 0.011879213, 0.010874448, 0.012330733 ToolsUI Screen Capture<http://cl.ly/WZMW> Opening the .main file and performing an Ncdump Data of variable: "Level-3_Binned_Data/Rrs_412(0:10:1).Rrs_412_sum" Returns: float Rrs_412_sum; data: {1.86057E-40, 9.403955E-38, 6.4099753E-10, 2.6076459E-9, 1.0297978E21, 5.6431478E-11, 0.0, -2.9699963E36, 4.59183E-40, 3.67343E-40, 2.60329423E11} Also, in ToolsUI, all of the other data variables (i.e. angstrom, aot_869 & Rrs_*) all display very very similar (most are identical) values as the Rrs_412. This is not the case for HDFView. Incidentally, reading the data via OceanColor's SeaDas<http://seadas.gsfc.nasa.gov/> application (which uses NetCDF Java under the hood) results in the same data as ToolsUI. Wrap-up The evidence above appears to indicate that there is a bug in NetCDF Java related to linked HDF files which results in incorrect data reads from linked files. Does anyone have any idea: a) what could be causing the issue? b) how could it be addressed? Thanks in advance, Chris
netcdf-java archives: