[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #IUX-310433]: Aggregating with Groups in NCML

If he extracted the records he wants and put them in a single file, that would 
work. 

> Hi John...
> 
> Many thanks for looking into this.  I'm not sure about the projection
> issues, since HP was only working with the
> "proj1/Instantaneous_Rain_Rate" and this display just fine in the IDV.
> His problem is he has 96 files and wants to show
> them in a loop (without having to select each of them....).
> 
> So, I must conclude from your comments that the 2 GDS's are the real
> issue with the aggregation, and because of that he is out-of-luck for
> now, unless your "stab" works out...?  I'll keep my fingers crossed!
> 
> Thank you again...let us know if you have any success!
> 
> Best wishes,
> 
> tom
> 
> On Mon, Jun 20, 2011 at 2:43 PM, Unidata netCDF Java Support
> <address@hidden> wrote:
> > Hi Tom, HP:
> >
> > I have two files, each with 3 records.
> >
> > 1) 1 record uses "space view perspective" (attached MSG1.png), 2 records 
> > use a lat/lon GDS (attached MSG2.png). The first has an interesting banded 
> > pattern. The second has -9999 as the first lat/lon point (see GDS dump 
> > below). Im guessing that this might mean "calculate the domain using only 
> > the second lat/lon point and nx/ny/dx/dy". But theres nothing in the GRIB 
> > spec that seems to allow this. HP, is this the case? Is "use -9999 to mean 
> > calculate the grid from the other info" part of GRIB standard or is it some 
> > local convention from EUmetsat?
> >
> > 2) the 2 lat/lon have unknown parameters 3-1-98 and 3-1-99 (3=discipline, 
> > 1=category,98/99=parameter). These are not in the latest WMO parameter 
> > tables, and they are not in the "local parameter" area of the table (>191). 
> > HP, can you comment on what these are, and if EUM is using WMO parameter 
> > numbers for local parameters ?
> >
> > 3) theres some interesting metadata in the local use section that we are 
> > not picking up. Ive copied a complete dump of one of the records below.
> >
> > 4) because there are 2 GDS, there are 2 groups. the aggregation is failing 
> > with groups. im not sure if i can fix this quickly, although im taking a 
> > stab at it.
> >
> > 4) TDS 4.3 has a complete refactoring of GRIB-2, with a new feature to 
> > specify a collection of grib2 files, which are then correctly aggregated. 
> > So it works correctly because NcML aggregation is not needed. I can give 
> > you an advance copy, but this wont be released for a few months, and then 
> > theres the question of IDV integration.
> >
> > John
> >
> > ----
> >
> > File=0 
> > E:/work/whittaker/MSG2-SEVI-MSGMPEG-0100-0100-20090830163000.000000000Z-1002820.grb
> > Header="(0)(2)(1)(6)(1)(2)(0)(0)(234)u(10)(20)2(4)(5)(1)(0)!V(9)(13)(10)HeaderVersionNo
> >           0(13)(10)FileType           
> >   Â2(13)(10)SubHeaderType           
> > 1(13)(10)SourceFacilityID         Â6(13)(10)SourceEnvId  
> >           1(13)(10)SourceInstanceId       
> >   Â2(13)(10)SourceSUId          
> > Â60021(13)(10)SourceCPUId     Â10 Â20 Â50  
> > 4(13)(10)DestFaciliyId           5(13)(10)DestEnvId  
> >            1(13)(10)DataFieldLength       
> > 2184713(13)(10)(0)(171)(24)I(181)(3)(153)(10) 
> > (1)B(13)(10)SubHeaderVersionNo        Â0(13)(10)ServiceType 
> >           171(13)(10)ServiceSubType       
> >    24(13)(10)FileTime       
> > 18869:60361248(13)(10)SpacecraftId          
> > Â322(13)(10)(0)(16)(0)(0)(1)(1)I(181)(3)(138)
 
address@hidden(1)(0)(0)I(181)(3)(138)_@(1)(0)I(181)(3)(138)_@(0)(0)I(181)(3)(138)_@(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0
> > Â)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(0)(0)(0)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)"
> >
> > Grib2IndicatorSection
> > ÂDiscipline = (3) Space products
> > ÂLength   = 2184244
> >
> > Grib2IdentificationSection
> > ÂCenter    Â= (254) EUMETSAT Operation Centre
> > ÂSubCenter   = (0) null
> > ÂMaster Table Â= 3
> > ÂLocal Table  = 0
> > ÂRefTimeSignif = 3 (Observation time)
> > ÂRefTime    = 2009-08-30T16:30:00.000Z
> > ÂRefTime Fields = 2009-8-30 16:30:0
> > ÂProductionStatus   Â= 1 (Operational test products)
> > ÂTypeOfProcessedData  = 6 (Processed satellite observations)
> >
> > Grib2GridDefinitionSection hash=-1919541760 crc=3711080673
> > ÂLength       = 72
> > ÂSource Â(3.0) Â Â Â= 0 (Specified in Code table 3.1)
> > ÂNpts        = 576
> > ÂTemplate (3.1) Â Â = 0
> >
> > (3.0) Grid definition template 3.0 - latitude/longitude (or equidistant 
> > cylindrical, or Plate CarrÃe)
> > Â1: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â GDS length == 72
> > Â5: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂSection == 3
> > Â6: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Source of Grid Definition (see code table 3.0) == 0 (table 3.0: Specified 
> > in Code table 3.1)
> > Â7: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â ÂNumber of data points == 576
> > Â11: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Number of octects for optional list of numbers == 0
> > Â12: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â ÂInterpretation of list of numbers == 0 (table 3.11: There 
> > is no appended list)
> > Â13: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â ÂGrid Definition Template Number == 0
> > Â15: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Shape of the Earth == 3 (table 3.2: 
> > Earth assumed oblate spheroid with major and minor axes specified (in km) 
> > by data producer)
> > Â16: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â ÂScale factor of radius of spherical Earth == 255
> > Â17: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â ÂScaled value of radius of spherical Earth == -9999
> > Â21: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂScale 
> > factor of major axis of oblate spheroid Earth == 4
> > Â22: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂScaled 
> > value of major axis of oblate spheroid Earth == 63781400
> > Â26: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂScale 
> > factor of minor axis of oblate spheroid Earth == 4
> > Â27: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂScaled 
> > value of minor axis of oblate spheroid Earth == 63567550
> > Â31: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Ni - number of points along a parallel == 24
> > Â35: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Nj - number of points along a meridian == 24
> > Â39: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Basic angle of the initial production domain == 0
> > Â43: Subdivisions of basic angle used to define extreme longitudes and 
> > latitudes, and direction increments == -9999
> > Â47: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â La1 - latitude of first grid point == -9999
> > Â51: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â ÂLo1 - longitude of first grid point == -9999
> > Â55: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Resolution and component flags == 56
> > Â56: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â ÂLa2 - latitude of last grid point == 60000000
> > Â60: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Lo2 - longitude of last grid point == 60000000
> > Â64: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Di - i direction increment == 5000000
> > Â68: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Dj - j direction increment == 5000000
> > Â72: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂScanning mode == 64
> > Â73: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂList of number of 
> > points along each meridian or parallel. == -9999
> >
> > Grib2ProductDefinitionSection
> >
> > (4.30) Product definition template 4.30 - satellite product
> > Â1: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â PDS length == 24
> > Â5: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂSection == 4
> > Â6: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > ÂNumber of coordinates values after Template == 0
> > Â8: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Product Definition Template Number == 30
> > Â10: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Parameter category == 1
> > Â11: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Parameter number == 98
> > Â12: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Â Â Â Â Â Â Â Â Type of generating process == 8 (table 4.3: 
> > Observation)
> > Â13: Â Â Â Â Â Â Â Â ÂObservation generating process identifier 
> > (defined by originating centre) == 7
> > Â14: Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> > Â Number of contributing spectral bands (NB) == 1
> >
> > Grib2SectionDataRepresentation
> > ÂTemplate      = 0 (Grid point data - simple packing)
> > ÂNPoints      Â= 576
> >
> > Grib2SectionData
> > ÂStarting Pos    = 2183854
> > ÂData Length    Â= 581
> >
> >
> >> Hi John....
> >>
> >> Did the data files sample get through okay? ÂIf not, I have also posted 
> >> this at:
> >> <ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/hpgrids.zip>
> >>
> >> Thanks again for looking into this...
> >>
> >> tom
> >>
> >> On Mon, Jun 13, 2011 at 2:32 PM, Tom Whittaker <address@hidden> wrote:
> >> > Hi John...
> >> >
> >> > Thanks....I had simply mis-typed it (guess I should use
> >> > cut-and-paste)....but since it does not work for me, I'm attaching a
> >> > zip file with the 2 nc files and the attempt at ncml...
> >> >
> >> > Thanks ahead for your help!!
> >> >
> >> > tom
> >> >
> >> > On Fri, Jun 10, 2011 at 4:11 PM, Unidata netCDF Java Support
> >> > <address@hidden> wrote:
> >> >> hi tom:
> >> >>
> >> >> jointExisting should be joinExisting.
> >> >>
> >> >> if thats not the actual problem, can you send me 2 files to check?
> >> >>
> >> >> john
> >> >>
> >> >>> I have several files that I want to aggregate over time. ÂEach file
> >> >>> has two Groups, with a dimension "time" which is used for the desired
> >> >>> variable...for example:
> >> >>>
> >> >>> Group proj1 {
> >> >>> dimensions:
> >> >>> time = 1;
> >> >>> y = 3712;
> >> >>> x = 3712;
> >> >>> variables:
> >> >>> float Instantaneous_Rain_Rate(time=1, y=3712, x=3712);
> >> >>> ...
> >> >>>
> >> >>> I am trying to use NCML to accomplish this, with an "aggregation" 
> >> >>> element like:
> >> >>>
> >> >>> <aggregation dimName="time" type="jointExisting">
> >> >>> ...
> >> >>>
> >> >>> But apparently "time" is not the right syntax for this. ÂI've tried
> >> >>> "proj1/time" as well, but to no available. ÂI'm hoping you can help.
> >> >>>
> >> >>> Thanks!
> >> >>>
> >> >>> tom
> >> >>>
> >> >>> --
> >> >>> Tom Whittaker
> >> >>> University of Wisconsin-Madison
> >> >>> Space Science & Engineering Center (SSEC)
> >> >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> >>> 1225 W. Dayton Street
> >> >>> Madison, WIÂ 53706Â USA
> >> >>> ph: +1 608 262 2759
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >> Ticket Details
> >> >> ===================
> >> >> Ticket ID: IUX-310433
> >> >> Department: Support netCDF Java
> >> >> Priority: Normal
> >> >> Status: Open
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Tom Whittaker
> >> > University of Wisconsin-Madison
> >> > Space Science & Engineering Center (SSEC)
> >> > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> > 1225 W. Dayton Street
> >> > Madison, WIÂ 53706Â USA
> >> > ph: +1 608 262 2759
> >> >
> >>
> >>
> >>
> >> --
> >> Tom Whittaker
> >> University of Wisconsin-Madison
> >> Space Science & Engineering Center (SSEC)
> >> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> 1225 W. Dayton Street
> >> Madison, WIÂ 53706Â USA
> >> ph: +1 608 262 2759
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: IUX-310433
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> >
> >
> 
> 
> 
> --
> Tom Whittaker
> University of Wisconsin-Madison
> Space Science & Engineering Center (SSEC)
> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> 1225 W. Dayton Street
> Madison, WIÂ 53706Â USA
> ph: +1 608 262 2759
> 
> 


Ticket Details
===================
Ticket ID: IUX-310433
Department: Support netCDF Java
Priority: Normal
Status: Closed

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.