[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Fwd: Re: [netcdf-java] announce: stable release netcdf-java 4.0]



Robb,

works like a charm!

Thanks very much.

-Bill

On 3/25/2009 3:12 PM, Robb Kambic wrote:
On Wed, 25 Mar 2009, Bill Moninger wrote:

Thanks very much Robb.

I fear I was a little impolitic with my "its your problem not mine" comment earlier. I much appreciate the effort you've gone to to get things working right. I know the grib format is very complicated indeed.

Just so I don't grab the wrong thing, should I get netcdfAll-4.0.jar from http://www.unidata.ucar.edu/software/netcdf-java/, or something else from somewhere else?

Bill,

the netcdfAll doesn't get updated as fast as the Grib package so go to:

http://www.unidata.ucar.edu/software/decoders/

go to the downloads page and get the GRIBJava package. The jar is in the lib dir. In your classpath reference the grib jar before the netcdfAll-4.0.jar to get the new grib code.

Robb...



-Bill

On 3/25/2009 2:28 PM, Robb Kambic wrote:
On Wed, 25 Mar 2009, Bill Moninger wrote:

Hi Robb,

Problem solved--but its your problem not mine.

At upper levels, VPTMP doesn't vary in xy--this is a physical result. But the values aren't (or shouldn't be) missing. Rather they're a 'reference' temperature that differs for each hybrid level. (and may differ for different times--I'm not sure about that.)

So, even though the field is constant, netcdf-java shouldn't be putting out the 'missing' value, but the actual constant value.

Bill,

i was looking at the problem this morning, it's a Missing Value Management problem. The docs don't say anything about this applying mvm to jpeg2000 decoding but I guess it does. So i implement it and you get constant values for the 42-50 hybrid levels for the ruc_sample file. I made a new distribution that you can get from the decoders web page.

RObb...


Here's some output from wgrib that shows that the max and min are equal in the upper levels--but are not missing!

Hope this helps.

-Bill

wgrib2.x -min -max -var -lev -lon -104.660004 37.830002
/public/data/grids/ruc/hyb_A252/grib2/0908416000003 | grep VPTMP
...
139:7720392:min=346:max=348.79:VPTMP:39 hybrid level:lon=255.323,lat=37.7791,val=346 140:7721589:min=349:max=353:VPTMP:40 hybrid level:lon=255.323,lat=37.7791,val=349 141:7722209:min=352:max=353.35:VPTMP:41 hybrid level:lon=255.323,lat=37.7791,val=352 142:7722698:min=355:max=355:VPTMP:42 hybrid level:lon=255.323,lat=37.7791,val=355 143:7722888:min=359:max=359:VPTMP:43 hybrid level:lon=255.323,lat=37.7791,val=359 144:7723078:min=365:max=365:VPTMP:44 hybrid level:lon=255.323,lat=37.7791,val=365 145:7723268:min=372:max=372:VPTMP:45 hybrid level:lon=255.323,lat=37.7791,val=372 146:7723458:min=385:max=385:VPTMP:46 hybrid level:lon=255.323,lat=37.7791,val=385 147:7723648:min=400:max=400:VPTMP:47 hybrid level:lon=255.323,lat=37.7791,val=400 148:7723838:min=422:max=422:VPTMP:48 hybrid level:lon=255.323,lat=37.7791,val=422 149:7724028:min=450:max=450:VPTMP:49 hybrid level:lon=255.323,lat=37.7791,val=450 150:7724218:min=500:max=500:VPTMP:50 hybrid level:lon=255.323,lat=37.7791,val=500


On 3/24/2009 5:33 PM, Robb Kambic wrote:
Bill,

I downloaded the new file, same thing. DataSection is only 5 bytes long and all the data is set to missing value. I'll look at it again.

vptmp is variable 0 0 15, are you sure  your looking at the right var.

i looked at hybrid level 45, one that showed missing data.


i'll do some more tests but...

RObb...





On Mon, 23 Mar 2009, Bill Moninger wrote:

Hi Robb,

pretty strange. But wgrib2 definitely shows data up there, and the data make perfect sense, because they are totally consistent with the (40km) grib(1) files. I'm generating soundings from the wind, VPTMP, and RH data--and the soundings look totally normal and consistent.

You can try a new file at http://ruc.noaa.gov/0908221000000.grib2

running
wgrib2 -var -lev -lon -104.660004 39.830002 <filename>
shows good data above lon/lat = -104.66/39.83 all the way up to the top.

I'm afraid I don't know the internals of the grib formats, but I can use wgrib2 per its instructions. I do notice that VPTMP does seem to be rounded off to the nearest degree starting at level 25. But VPTMP varies with lat/lon up there, so it isn't a constant field.

-Bill

On 3/23/2009 3:57 PM, Robb Kambic wrote:
On Mon, 23 Mar 2009, Bill Moninger wrote:

Hi Robb,

yes, the netcdf4-java code seems to be consistent in reporting fill values in the upper hybrid levels. But wgrib2 reports real data up there.

-Bill

Here's the output from
-----------------------------------------------------------
wgrib2 -var -lev -lon -105 40 ruc_sample.grib2 |grep VPTMT
101:6841742:VPTMP:1 hybrid level:lon=254.88,lat=40.0639,val=297.03
102:6897701:VPTMP:2 hybrid level:lon=254.88,lat=40.0639,val=298.76
103:6951440:VPTMP:3 hybrid level:lon=254.88,lat=40.0639,val=300.13
104:7003829:VPTMP:4 hybrid level:lon=254.88,lat=40.0639,val=300.3
105:7054644:VPTMP:5 hybrid level:lon=254.88,lat=40.0639,val=301.78
106:7104212:VPTMP:6 hybrid level:lon=254.88,lat=40.0639,val=303.21
107:7153410:VPTMP:7 hybrid level:lon=254.88,lat=40.0639,val=304.33
108:7202486:VPTMP:8 hybrid level:lon=254.88,lat=40.0639,val=305.15
109:7251048:VPTMP:9 hybrid level:lon=254.88,lat=40.0639,val=305.8
110:7297830:VPTMP:10 hybrid level:lon=254.88,lat=40.0639,val=306.3
111:7343669:VPTMP:11 hybrid level:lon=254.88,lat=40.0639,val=306.66
112:7388633:VPTMP:12 hybrid level:lon=254.88,lat=40.0639,val=306.9
113:7431303:VPTMP:13 hybrid level:lon=254.88,lat=40.0639,val=307.12
114:7470554:VPTMP:14 hybrid level:lon=254.88,lat=40.0639,val=307.23
115:7506808:VPTMP:15 hybrid level:lon=254.88,lat=40.0639,val=307.32
116:7540913:VPTMP:16 hybrid level:lon=254.88,lat=40.0639,val=307.34
117:7573226:VPTMP:17 hybrid level:lon=254.88,lat=40.0639,val=307.39
118:7603011:VPTMP:18 hybrid level:lon=254.88,lat=40.0639,val=307.39
119:7631630:VPTMP:19 hybrid level:lon=254.88,lat=40.0639,val=307.51
120:7658113:VPTMP:20 hybrid level:lon=254.88,lat=40.0639,val=307.51
121:7683603:VPTMP:21 hybrid level:lon=254.88,lat=40.0639,val=307.67
122:7707088:VPTMP:22 hybrid level:lon=254.88,lat=40.0639,val=307.67
123:7729290:VPTMP:23 hybrid level:lon=254.88,lat=40.0639,val=307.99
124:7750109:VPTMP:24 hybrid level:lon=254.88,lat=40.0639,val=308.01
125:7768065:VPTMP:25 hybrid level:lon=254.88,lat=40.0639,val=310
126:7783549:VPTMP:26 hybrid level:lon=254.88,lat=40.0639,val=312
127:7796762:VPTMP:27 hybrid level:lon=254.88,lat=40.0639,val=314
128:7807624:VPTMP:28 hybrid level:lon=254.88,lat=40.0639,val=316
129:7816452:VPTMP:29 hybrid level:lon=254.88,lat=40.0639,val=318
130:7823924:VPTMP:30 hybrid level:lon=254.88,lat=40.0639,val=320
131:7829560:VPTMP:31 hybrid level:lon=254.88,lat=40.0639,val=322
132:7834231:VPTMP:32 hybrid level:lon=254.88,lat=40.0639,val=325
133:7836697:VPTMP:33 hybrid level:lon=254.88,lat=40.0639,val=328
134:7837681:VPTMP:34 hybrid level:lon=254.88,lat=40.0639,val=331
135:7839324:VPTMP:35 hybrid level:lon=254.88,lat=40.0639,val=334
136:7840657:VPTMP:36 hybrid level:lon=254.88,lat=40.0639,val=337
137:7843491:VPTMP:37 hybrid level:lon=254.88,lat=40.0639,val=340
138:7846424:VPTMP:38 hybrid level:lon=254.88,lat=40.0639,val=343
139:7848140:VPTMP:39 hybrid level:lon=254.88,lat=40.0639,val=346
140:7849384:VPTMP:40 hybrid level:lon=254.88,lat=40.0639,val=349
141:7850100:VPTMP:41 hybrid level:lon=254.88,lat=40.0639,val=352
142:7850835:VPTMP:42 hybrid level:lon=254.88,lat=40.0639,val=355
143:7851025:VPTMP:43 hybrid level:lon=254.88,lat=40.0639,val=359
144:7851215:VPTMP:44 hybrid level:lon=254.88,lat=40.0639,val=365



145:7851405:VPTMP:45 hybrid level:lon=254.88,lat=40.0639,val=372

Bill,

For vptmp hybrid 45

i did some detail debugging for the actual data read. Data section 7 is only 5 bytes long. The reference value is 372, maybe that's what you are seeing.
the data template is 40, that's jpeg decoding.
the number of bits is 0, which means all the data values are the same.

This is the code that we use for all jpeg decoding, so I have confidence it's correct. Are you certain that you are request the correct information with wgrib2? Can you come up with another sample?

Robb...


146:7851595:VPTMP:46 hybrid level:lon=254.88,lat=40.0639,val=385
147:7851785:VPTMP:47 hybrid level:lon=254.88,lat=40.0639,val=400
148:7851975:VPTMP:48 hybrid level:lon=254.88,lat=40.0639,val=422
149:7852165:VPTMP:49 hybrid level:lon=254.88,lat=40.0639,val=450
150:7852355:VPTMP:50 hybrid level:lon=254.88,lat=40.0639,val=500
------------------------------------------------------------------

On 3/20/2009 4:33 PM, Robb Kambic wrote:
On Fri, 20 Mar 2009, Bill Moninger wrote:

Hi Robb,

Thanks for this. The new netcdf4-java works nearly perfectly.

Unfortunately, for one of the variables I'm trying to read out of the file, upper levels are filled with fill-values. Its odd, but I've written a bare-bones simple program to read the variable, and I'm getting nothing but fill values above hybrid level 41 for variable "VPTMP", also known as Virtual_potential_temperature in netCDF-land.


Bill,

i display the ruc_sample.grig2 in toolsUI, hybrid levels ~42 to 50 had fill values. i even dumped Virtual_potential_temperature hybrid level using the low level reader Grib2GetData it had all fill values.

this was for hybrid level 45

Grib2GetData C:\data\grib\ruc_sample.grib2 7851442 7851523

If you look at the java docs on the home page it has routines Grib2Dump, Grib2Indexer, ShowGrib, etc that lets one inspect low level values.

RObb...



The grib2 file is at http://ruc.noaa.gov/ruc_sample.grib2

The relevant part of the code that reads it below.

   ncfile = NetcdfFile.open(filename);
   Debug.println("ncfile is "+ncfile);
   //Debug.println(""+ncfile.getDetailInfo());
   Array data4D;
   Variable v = null;
   Attribute a = null;

   // get grid parameters for most variables
   Dimension d = ncfile.findDimension("hybrid");
   if(d == null) {
     Debug.println("Bad dimension for hybrid");
     System.exit(1);
   }
   tuv_levels = d.getLength();

   // get variables
   int[] origin = new int[] {0, 0, 112,121};
   int[] tuv_size = new int[] {1, tuv_levels, 1,1};
   v  = ncfile.findVariable("Virtual_potential_temperature");
   data4D = v.read(origin, tuv_size);
   Tvar = (double[])data4D.reduce().get1DJavaArray(double.class);
...
 for(int i=0;i<tuv_levels;i++) {
   Debug.println("i: "+i+" t "+Tvar[i]);
 }

This seems pretty obscure, inasmuch as it doesn't effect other variables like pressure, but I fail to find any problem in my code, even after simplifying it down to bare bones.

wgrib2 shows data in all the levels for the specific column specified above.

I'd appreciate whatever help you can provide.

-Bill

On 3/19/2009 1:12 PM, Robb Kambic wrote:
On Thu, 19 Mar 2009, Bill Moninger wrote:

Hi Robb,

any update on this? Is a new version of netcdf4-java available that will read the grib2 RUC format with its hybrid coordinates?


The netcdf4-java with the hybrid level fixes are available at:

http://www.unidata.ucar.edu/software/netcdf-java/

let me know if you find any problems.

Robb...

Robb Kambic                       Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research address@hidden WWW: http://www.unidata.ucar.edu/ ===============================================================================






--
William R. Moninger http://www-frd.fsl.noaa.gov/~moninger/ NOAA / Earth Systems Research Laboratory / Global Systems Division 325 Broadway, R/GSD1 voice: 303-497-6435 Boulder, CO 80305 fax: 303-497-3329


=============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research address@hidden WWW: http://www.unidata.ucar.edu/ ===============================================================================





--
William R. Moninger         http://www-frd.fsl.noaa.gov/~moninger/
NOAA / Earth Systems Research Laboratory / Global Systems Division
325 Broadway, R/GSD1                           voice: 303-497-6435
Boulder, CO 80305                              fax:   303-497-3329


=============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research
address@hidden           WWW: http://www.unidata.ucar.edu/
===============================================================================




--
William R. Moninger         http://www-frd.fsl.noaa.gov/~moninger/
NOAA / Earth Systems Research Laboratory / Global Systems Division
325 Broadway, R/GSD1                           voice: 303-497-6435
Boulder, CO 80305                              fax:   303-497-3329


=============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research
address@hidden           WWW: http://www.unidata.ucar.edu/
===============================================================================



--
William R. Moninger         http://www-frd.fsl.noaa.gov/~moninger/
NOAA / Earth Systems Research Laboratory / Global Systems Division
325 Broadway, R/GSD1                           voice: 303-497-6435
Boulder, CO 80305                              fax:   303-497-3329


=============================================================================== Robb Kambic Unidata Program Center
Software Engineer III               Univ. Corp for Atmospheric Research
address@hidden           WWW: http://www.unidata.ucar.edu/
===============================================================================


--
William R. Moninger         http://www-frd.fsl.noaa.gov/~moninger/
NOAA / Earth Systems Research Laboratory / Global Systems Division
325 Broadway, R/GSD1                           voice: 303-497-6435
Boulder, CO 80305                              fax:   303-497-3329


===============================================================================
Robb Kambic                       Unidata Program Center
Software Engineer III               Univ. Corp for Atmospheric Research
address@hidden           WWW: http://www.unidata.ucar.edu/
===============================================================================

--
William R. Moninger         http://www-frd.fsl.noaa.gov/~moninger/
NOAA / Earth Systems Research Laboratory / Global Systems Division
325 Broadway, R/GSD1                           voice: 303-497-6435
Boulder, CO 80305                              fax:   303-497-3329