Hi Don:
Funny you should mention this, I am just working on it.
John
Here's a possible hitch in this. Last week during the change to the GFS, the 192hr GEFS ensemble files now contain 2 accumulated precip fields - one 6hr and one a 12hr accumulation. If I load in a single file from the FTP server like:
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.20100806/00/pgrb2a/gec00.t00z.pgrb2af192
wgrib output shows:
52:2608353:d=2010080600:APCP:surface:186-192 hour acc fcst:ENS=low-res ctl
53:2651667:d=2010080600:APCP:surface:180-192 hour acc fcst:ENS=low-res ctl
Then I get a precip field with a time bounds that shows both time steps. However, if I load in the file from the THREDDS server which would presumably have these grib messages, all I see are the 6hr accumulation fields and the 12hr one gets lost. I was looking at:
dods://motherlode.ucar.edu:9080/thredds/dodsC/NCEP/GEFS/Global_1p0deg_Ensemble/member/GEFS_Global_1p0deg_Ensemble_20100806_0600.grib2
How would the new paradigm distinguish this case and how would I ask for the 12hr accumulation though the GeoGrid interface?
I guess the basic question is how would one distinguish these fields when you have a grib file with both 6 and 12 hour accumulated precip in it.
Don
On 7/26/10 1:14 PM, John Caron wrote:description of the time interval problem. i dont think its affects IDV use but not positive. in any case, older versions are also broken.
-------- Original Message -------- Subject: [netcdf-java] Problems with GRIB "time interval" variables Date: Tue, 15 Jun 2010 15:39:33 -0600 From: John Caron <address@hidden> To: Java NetCDF <address@hidden>, THREDDS community <address@hidden>
To netCDF-Java and TDS users:
We have discovered that our GRIB reading software is not correctly
reading GRIB variables whose time coordinates are time intervals, rather
than instantaneous time. This affects both clients that use the
netCDF-Java library, and TDS servers, which read GRIB1 or GRIB2 files.
If you are not reading GRIB files, then this problem will not affect you.
In the real-time NCEP model output datasets that we serve on motherlode, this problem occurs with about 20% of the variables. A list of the problem variables follows. In almost all cases, the data itself is correctly read, but the metadata describing the time coordinate is misleading. However, the RUC2_CONUS_20km_pressure dataset has inconsistent results for the variables "Convective_precipitation" and "Large_scale_precipitation".
"Time interval coordinates" have a start and ending time, and a "statistical type" like Accumulation, Average, Maximum, etc. which is applied over the time range. We have been inadvertently ignoring time interval coordinates and their statistical type, and instead presenting these variables as if they were instantaneous. The Common Data Model has a way to represent interval coordinates using the CF Conventions "bounds" attribute on the time coordinate, which points to a variable containing the starting and ending time coordinate for that interval. We will be modifying the GRIB IOSP to do that.
We will be putting out a new release of version 4.2 of the netCDF-Java library and the TDS that fixes this problem ASAP. This problem will not be fixed in version 4.1, however. If you have been using these datasets in a way that will be affected by these changes, or have concerns or questions about these fixes, please respond to this email group or privately to address@hidden
Sorry for the trouble, John Caron
======================================================== GRIB2 datasets with variables that are affected. The number in () is the grid definition template number.
Q:\cdmUnitTest\tds\new\DGEX_Alaska_12km_20100524_0000.grib2 Total_precipitation/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\DGEX_CONUS_12km_20100514_1800.grib2 Total_precipitation/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\GEFS_Global_1p0deg_Ensemble_20100424_0000.grib2 Categorical_Freezing_Rain/Ground or water surface (11) Categorical_Ice_Pellets/Ground or water surface (11) Categorical_Rain/Ground or water surface (11) Categorical_Snow/Ground or water surface (11) Downward_long_wave_rad_flux/Ground or water surface (11) Downward_short_wave_rad_flux/Ground or water surface (11) Latent_heat_net_flux/Ground or water surface (11) Maximum_temperature/Specified height level above ground (11) Minimum_temperature/Specified height level above ground (11) Sensible_heat_net_flux/Ground or water surface (11) Total_cloud_cover/Entire atmosphere layer (11) Total_precipitation/Ground or water surface (11) Upward_long_wave_rad_flux/Ground or water surface (11) Upward_long_wave_rad_flux/Nominal top of the atmosphere (11) Upward_short_wave_rad_flux/Ground or water surface (11)
Q:\cdmUnitTest\tds\new\GEFS_Global_1p0deg_Ensemble_derived_20100423_0000.grib2
Categorical_Freezing_Rain/Ground or water surface (12) Categorical_Ice_Pellets/Ground or water surface (12) Categorical_Rain/Ground or water surface (12) Categorical_Snow/Ground or water surface (12) Downward_long_wave_rad_flux/Ground or water surface (12) Downward_short_wave_rad_flux/Ground or water surface (12) Latent_heat_net_flux/Ground or water surface (12) Maximum_temperature/Specified height level above ground (12) Minimum_temperature/Specified height level above ground (12) Sensible_heat_net_flux/Ground or water surface (12) Total_cloud_cover/Entire atmosphere layer (12) Total_precipitation/Ground or water surface (12) Upward_long_wave_rad_flux/Ground or water surface (12) Upward_long_wave_rad_flux/Nominal top of the atmosphere (12) Upward_short_wave_rad_flux/Ground or water surface (12)
Q:\cdmUnitTest\tds\new\GFS_Global_0p5deg_20100602_0000.grib2 Albedo/Ground or water surface (8) Categorical_Freezing_Rain/Ground or water surface (8) Categorical_Ice_Pellets/Ground or water surface (8) Categorical_Rain/Ground or water surface (8) Categorical_Snow/Ground or water surface (8) Cloud_Work_Function/Entire atmosphere layer (8) Convective_Precipitation_Rate/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Maximum_temperature/Specified height level above ground (8) Meridional_Flux_of_Gravity_Wave_Stress/Ground or water surface (8) Minimum_temperature/Specified height level above ground (8) Momentum_flux_u_component/Ground or water surface (8) Momentum_flux_v_component/Ground or water surface (8) Precipitation_rate/Ground or water surface (8) Pressure/High cloud bottom level (8) Pressure/High cloud top level (8) Pressure/Low cloud bottom level (8) Pressure/Low cloud top level (8) Pressure/Middle cloud bottom level (8) Pressure/Middle cloud top level (8) Sensible_heat_net_flux/Ground or water surface (8) Temperature/High cloud top level (8) Temperature/Low cloud top level (8) Temperature/Middle cloud top level (8) Total_cloud_cover/Boundary layer cloud layer (8) Total_cloud_cover/Entire atmosphere layer (8) Total_cloud_cover/High cloud layer (8) Total_cloud_cover/Low cloud layer (8) Total_cloud_cover/Middle cloud layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_runoff/Ground or water surface (8) Zonal_Flux_of_Gravity_Wave_Stress/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\GFS_Global_2p5deg_20100602_1200.grib2 Albedo/Ground or water surface (8) Categorical_Freezing_Rain/Ground or water surface (8) Categorical_Ice_Pellets/Ground or water surface (8) Categorical_Rain/Ground or water surface (8) Categorical_Snow/Ground or water surface (8) Cloud_Work_Function/Entire atmosphere layer (8) Convective_Precipitation_Rate/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Maximum_temperature/Specified height level above ground (8) Meridional_Flux_of_Gravity_Wave_Stress/Ground or water surface (8) Minimum_temperature/Specified height level above ground (8) Momentum_flux_u_component/Ground or water surface (8) Momentum_flux_v_component/Ground or water surface (8) Precipitation_rate/Ground or water surface (8) Pressure/High cloud bottom level (8) Pressure/High cloud top level (8) Pressure/Low cloud bottom level (8) Pressure/Low cloud top level (8) Pressure/Middle cloud bottom level (8) Pressure/Middle cloud top level (8) Sensible_heat_net_flux/Ground or water surface (8) Temperature/High cloud top level (8) Temperature/Low cloud top level (8) Temperature/Middle cloud top level (8) Total_cloud_cover/Boundary layer cloud layer (8) Total_cloud_cover/Entire atmosphere layer (8) Total_cloud_cover/High cloud layer (8) Total_cloud_cover/Low cloud layer (8) Total_cloud_cover/Middle cloud layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_runoff/Ground or water surface (8) Zonal_Flux_of_Gravity_Wave_Stress/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\GFS_Global_onedeg_20100529_0600.grib2 Albedo/Ground or water surface (8) Categorical_Freezing_Rain/Ground or water surface (8) Categorical_Ice_Pellets/Ground or water surface (8) Categorical_Rain/Ground or water surface (8) Categorical_Snow/Ground or water surface (8) Cloud_Work_Function/Entire atmosphere layer (8) Convective_Precipitation_Rate/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Maximum_temperature/Specified height level above ground (8) Meridional_Flux_of_Gravity_Wave_Stress/Ground or water surface (8) Minimum_temperature/Specified height level above ground (8) Momentum_flux_u_component/Ground or water surface (8) Momentum_flux_v_component/Ground or water surface (8) Precipitation_rate/Ground or water surface (8) Pressure/High cloud bottom level (8) Pressure/High cloud top level (8) Pressure/Low cloud bottom level (8) Pressure/Low cloud top level (8) Pressure/Middle cloud bottom level (8) Pressure/Middle cloud top level (8) Sensible_heat_net_flux/Ground or water surface (8) Temperature/High cloud top level (8) Temperature/Low cloud top level (8) Temperature/Middle cloud top level (8) Total_cloud_cover/Boundary layer cloud layer (8) Total_cloud_cover/Entire atmosphere layer (8) Total_cloud_cover/High cloud layer (8) Total_cloud_cover/Low cloud layer (8) Total_cloud_cover/Middle cloud layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_runoff/Ground or water surface (8) Zonal_Flux_of_Gravity_Wave_Stress/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_Alaska_11km_20100519_0000.grib2 Convective_precipitation/Ground or water surface (8) Probability_of_freezing_precipitation/Ground or water surface (8) Probability_of_frozen_precipitation/Ground or water surface (8) Probability_of_p01_inch_precipitation/Ground or water surface (8) Thunderstorm_probability/Ground or water surface (8) Total_precipitation/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_Alaska_45km_conduit_20100524_0000.grib2 Baseflow_groundwater_runoff/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Evaporation/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Potential_Evaporation/Ground or water surface (8) Sensible_heat_net_flux/Ground or water surface (8) Snow_melt/Ground or water surface (8) Snow_phase_change_heat_flux/Ground or water surface (8) Storm_surface_runoff/Ground or water surface (8) Total_cloud_cover/Entire atmosphere layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_CONUS_12km_20100425_1800.grib2 Baseflow_groundwater_runoff/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Deep_convective_heat_rate/Entire atmosphere layer (8) Downward_short_wave_rad_flux/Ground or water surface (8) Large_scale_condensate_heat_rate/Entire atmosphere layer (8) Snow_melt/Ground or water surface (8) Storm_surface_runoff/Ground or water surface (8) Total_precipitation/Ground or water surface (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_CONUS_12km_20100520_0000.grib2 Convective_precipitation/Ground or water surface (8) Probability_of_freezing_precipitation/Ground or water surface (8) Probability_of_frozen_precipitation/Ground or water surface (8) Probability_of_p01_inch_precipitation/Ground or water surface (8) Thunderstorm_probability/Ground or water surface (8) Total_precipitation/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_CONUS_20km_selectsurface_20100426_1800.grib2 Baseflow_groundwater_runoff/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Evaporation/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Potential_Evaporation/Ground or water surface (8) Probability_of_freezing_precipitation/Ground or water surface (8) Probability_of_frozen_precipitation/Ground or water surface (8) Probability_of_p01_inch_precipitation/Ground or water surface (8) Sensible_heat_net_flux/Ground or water surface (8) Snow_depth/Ground or water surface (8) Snow_melt/Ground or water surface (8) Snow_phase_change_heat_flux/Ground or water surface (8) Storm_surface_runoff/Ground or water surface (8) Thunderstorm_probability/Ground or water surface (8) Total_cloud_cover/Entire atmosphere layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_CONUS_20km_surface_20100429_0600.grib2 Baseflow_groundwater_runoff/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Deep_convective_heat_rate/Entire atmosphere layer (8) Downward_short_wave_rad_flux/Ground or water surface (8) Large_scale_condensate_heat_rate/Entire atmosphere layer (8) Snow_melt/Ground or water surface (8) Storm_surface_runoff/Ground or water surface (8) Total_precipitation/Ground or water surface (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_CONUS_40km_conduit_20100506_1800.grib2 Baseflow_groundwater_runoff/Ground or water surface (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Evaporation/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Potential_Evaporation/Ground or water surface (8) Probability_of_freezing_precipitation/Ground or water surface (8) Probability_of_frozen_precipitation/Ground or water surface (8) Probability_of_p01_inch_precipitation/Ground or water surface (8) Sensible_heat_net_flux/Ground or water surface (8) Snow_depth/Ground or water surface (8) Snow_melt/Ground or water surface (8) Snow_phase_change_heat_flux/Ground or water surface (8) Storm_surface_runoff/Ground or water surface (8) Thunderstorm_probability/Ground or water surface (8) Total_cloud_cover/Entire atmosphere layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NAM_Polar_90km_20100525_1200.grib2 Baseflow_groundwater_runoff/Ground or water surface (8) Convective_cloud_cover/Entire atmosphere layer (8) Convective_precipitation/Ground or water surface (8) Downward_long_wave_rad_flux/Ground or water surface (8) Downward_short_wave_rad_flux/Ground or water surface (8) Evaporation/Ground or water surface (8) Ground_Heat_Flux/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Latent_heat_net_flux/Ground or water surface (8) Non-Convective_Cloud_Cover/Entire atmosphere layer (8) Potential_Evaporation/Ground or water surface (8) Sensible_heat_net_flux/Ground or water surface (8) Snow_melt/Ground or water surface (8) Snow_phase_change_heat_flux/Ground or water surface (8) Storm_surface_runoff/Ground or water surface (8) Total_cloud_cover/Entire atmosphere layer (8) Total_precipitation/Ground or water surface (8) Upward_long_wave_rad_flux/Ground or water surface (8) Upward_long_wave_rad_flux/Nominal top of the atmosphere (8) Upward_short_wave_rad_flux/Ground or water surface (8) Upward_short_wave_rad_flux/Nominal top of the atmosphere (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\NDFD_CONUS_5km_20100527_1200.grib2 Maximum_temperature/Specified height level above ground (8) Minimum_temperature/Specified height level above ground (8) Probability_of_p01_inch_precipitation/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\RUC2_CONUS_20km_hybrid_20100421_2000.grib2 Convective_precipitation/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\RUC2_CONUS_20km_pressure_20100509_1300.grib2 Convective_precipitation/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\RUC2_CONUS_20km_surface_20100516_1600.grib2 Convective_precipitation/Ground or water surface (8) Large_scale_precipitation/Ground or water surface (8) Water_equivalent_of_accumulated_snow_depth/Ground or water surface (8)
Q:\cdmUnitTest\tds\new\SREF_Alaska_45km_ensprod_20100525_0300.grib2 Large_scale_snow/Ground or water surface (9) Total_precipitation/Ground or water surface (9)
Q:\cdmUnitTest\tds\new\SREF_CONUS_40km_ensprod_20100513_1500.grib2 Large_scale_snow/Ground or water surface (12) Large_scale_snow/Ground or water surface (9) Total_precipitation/Ground or water surface (9) Total_precipitation/Ground or water surface (12)
Q:\cdmUnitTest\tds\new\SREF_PacificNE_0p4_ensprod_20100509_2100.grib2 Total_precipitation/Ground or water surface (9) Total_precipitation/Ground or water surface (12)
===================== GRIB1 datasets with variables that are affected. The number in () is the time range indicator code.
/share/testdata/cdmUnitTest/tds/new/GFS_Alaska_191km_20100507_1800.grib1 APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/GFS_CONUS_80km_20100513_0600.grib1 APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/GFS_CONUS_95km_20100506_0600.grib1 ACPCP Convective_precipitation (4) APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/GFS_Hawaii_160km_20100428_0000.grib1 ACPCP Convective_precipitation (4) APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/GFS_N_Hemisphere_381km_20100516_0600.grib1
ACPCP Convective_precipitation (4) APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/NAM_Alaska_22km_20100504_0000.grib1 ACPCP Convective_precipitation (4) APCP Total_precipitation (4) WEASD Accumulated_snow (4)
/share/testdata/cdmUnitTest/tds/new/NAM_Alaska_95km_20100502_0000.grib1 ACPCP Convective_precipitation (4) APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/NAM_CONUS_20km_noaaport_20100602_0000.grib1
ACPCP Convective_precipitation (4) APCP Total_precipitation (4) CPOFP Prob_of_frozen_precipitation (4) CPOZP Prob_of_freezing_precipitation (4) POP Prob_of_precipitation (4) SNOD Snow_depth (4) TSTM Thunderstorm_probability (4)
/share/testdata/cdmUnitTest/tds/new/NAM_CONUS_80km_20100508_1200.grib1 ACPCP Convective_precipitation (4) APCP Total_precipitation (4)
/share/testdata/cdmUnitTest/tds/new/RUC2_CONUS_40km_20100515_0200.grib1 ACPCP Convective_precipitation (4) NCPCP Large_scale_precipitation (4) WEASD Accumulated_snow (4)
/share/testdata/cdmUnitTest/tds/new/RUC_CONUS_80km_20100430_0000.grib1 ACPCP Convective_precipitation (4) APCP Total_precipitation (4) NCPCP Large_scale_precipitation (4)
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.