[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #IUX-310433]: Aggregating with Groups in NCML



forgot attachments

> Hi Tom, HP:
> 
> I have two files, each with 3 records.
> 
> 1) 1 record uses "space view perspective" (attached MSG1.png), 2 records use 
> a lat/lon GDS (attached MSG2.png). The first has an interesting banded 
> pattern. The second has -9999 as the first lat/lon point (see GDS dump 
> below). Im guessing that this might mean "calculate the domain using only the 
> second lat/lon point and nx/ny/dx/dy". But theres nothing in the GRIB spec 
> that seems to allow this. HP, is this the case? Is "use -9999 to mean 
> calculate the grid from the other info" part of GRIB standard or is it some 
> local convention from EUmetsat?
> 
> 2) the 2 lat/lon have unknown parameters 3-1-98 and 3-1-99 (3=discipline, 
> 1=category,98/99=parameter). These are not in the latest WMO parameter 
> tables, and they are not in the "local parameter" area of the table (>191). 
> HP, can you comment on what these are, and if EUM is using WMO parameter 
> numbers for local parameters ?
> 
> 3) theres some interesting metadata in the local use section that we are not 
> picking up. Ive copied a complete dump of one of the records below.
> 
> 4) because there are 2 GDS, there are 2 groups. the aggregation is failing 
> with groups. im not sure if i can fix this quickly, although im taking a stab 
> at it.
> 
> 4) TDS 4.3 has a complete refactoring of GRIB-2, with a new feature to 
> specify a collection of grib2 files, which are then correctly aggregated. So 
> it works correctly because NcML aggregation is not needed. I can give you an 
> advance copy, but this wont be released for a few months, and then theres the 
> question of IDV integration.
> 
> John
> 
> ----
> 
> File=0 
> E:/work/whittaker/MSG2-SEVI-MSGMPEG-0100-0100-20090830163000.000000000Z-1002820.grb
> Header="(0)(2)(1)(6)(1)(2)(0)(0)(234)u(10)(20)2(4)(5)(1)(0)!V(9)(13)(10)HeaderVersionNo
>                    0(13)(10)FileType                          
> 2(13)(10)SubHeaderType                     1(13)(10)SourceFacilityID          
>         6(13)(10)SourceEnvId                       1(13)(10)SourceInstanceId  
>                 2(13)(10)SourceSUId                    
> 60021(13)(10)SourceCPUId          10  20  50   4(13)(10)DestFaciliyId         
>             5(13)(10)DestEnvId                         
> 1(13)(10)DataFieldLength             
> 2184713(13)(10)(0)(171)(24)I(181)(3)(153)(10) (1)B(13)(10)SubHeaderVersionNo  
>               0(13)(10)ServiceType                     
> 171(13)(10)ServiceSubType                   24(13)(10)FileTime             
> 18869:60361248(13)(10)SpacecraftId                    
> 322(13)(10)(0)(16)(0)(0)(1)(1)I(181)(3)(138)address@hidden(1)(0)(0)I(181)(3)(138)_@(1)(0)I(181)(3)(138)_@(0)(0)I(181)(3)(138)_@(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)
 
(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(0)(0)(0)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)"
> 
> Grib2IndicatorSection
> Discipline = (3) Space products
> Length     = 2184244
> 
> Grib2IdentificationSection
> Center        = (254) EUMETSAT Operation Centre
> SubCenter     = (0) null
> Master Table  = 3
> Local Table   = 0
> RefTimeSignif = 3 (Observation time)
> RefTime       = 2009-08-30T16:30:00.000Z
> RefTime Fields = 2009-8-30 16:30:0
> ProductionStatus      = 1 (Operational test products)
> TypeOfProcessedData   = 6 (Processed satellite observations)
> 
> Grib2GridDefinitionSection hash=-1919541760 crc=3711080673
> Length             = 72
> Source  (3.0)      = 0 (Specified in Code table 3.1)
> Npts               = 576
> Template (3.1)     = 0
> 
> (3.0) Grid definition template 3.0 - latitude/longitude (or equidistant 
> cylindrical, or Plate CarrÃe)
> 1:                                                                            
>      GDS length == 72
> 5:                                                                            
>         Section == 3
> 6:                                             Source of Grid Definition (see 
> code table 3.0) == 0 (table 3.0: Specified in Code table 3.1)
> 7:                                                                      
> Number of data points == 576
> 11:                                             Number of octects for 
> optional list of numbers == 0
> 12:                                                          Interpretation 
> of list of numbers == 0 (table 3.11: There is no appended list)
> 13:                                                            Grid 
> Definition Template Number == 0
> 15:                                                                         
> Shape of the Earth == 3 (table 3.2: Earth assumed oblate spheroid with major 
> and minor axes specified (in km) by data producer)
> 16:                                                  Scale factor of radius 
> of spherical Earth == 255
> 17:                                                  Scaled value of radius 
> of spherical Earth == -9999
> 21:                                        Scale factor of major axis of 
> oblate spheroid Earth == 4
> 22:                                        Scaled value of major axis of 
> oblate spheroid Earth == 63781400
> 26:                                        Scale factor of minor axis of 
> oblate spheroid Earth == 4
> 27:                                        Scaled value of minor axis of 
> oblate spheroid Earth == 63567550
> 31:                                                     Ni - number of points 
> along a parallel == 24
> 35:                                                     Nj - number of points 
> along a meridian == 24
> 39:                                               Basic angle of the initial 
> production domain == 0
> 43: Subdivisions of basic angle used to define extreme longitudes and 
> latitudes, and direction increments == -9999
> 47:                                                         La1 - latitude of 
> first grid point == -9999
> 51:                                                        Lo1 - longitude of 
> first grid point == -9999
> 55:                                                             Resolution 
> and component flags == 56
> 56:                                                          La2 - latitude 
> of last grid point == 60000000
> 60:                                                         Lo2 - longitude 
> of last grid point == 60000000
> 64:                                                                 Di - i 
> direction increment == 5000000
> 68:                                                                 Dj - j 
> direction increment == 5000000
> 72:                                                                           
>    Scanning mode == 64
> 73:                                  List of number of points along each 
> meridian or parallel. == -9999
> 
> Grib2ProductDefinitionSection
> 
> (4.30) Product definition template 4.30 - satellite product
> 1:                                                                            
>      PDS length == 24
> 5:                                                                            
>         Section == 4
> 6:                                                Number of coordinates 
> values after Template == 0
> 8:                                                         Product Definition 
> Template Number == 30
> 10:                                                                         
> Parameter category == 1
> 11:                                                                           
> Parameter number == 98
> 12:                                                                 Type of 
> generating process == 8 (table 4.3: Observation)
> 13:                  Observation generating process identifier (defined by 
> originating centre) == 7
> 14:                                                 Number of contributing 
> spectral bands (NB) == 1
> 
> Grib2SectionDataRepresentation
> Template           = 0 (Grid point data - simple packing)
> NPoints            = 576
> 
> Grib2SectionData
> Starting Pos       = 2183854
> Data Length        = 581
> 
> 
> > Hi John....
> >
> > Did the data files sample get through okay?  If not, I have also posted 
> > this at:
> > <ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/hpgrids.zip>
> >
> > Thanks again for looking into this...
> >
> > tom
> >
> > On Mon, Jun 13, 2011 at 2:32 PM, Tom Whittaker <address@hidden> wrote:
> > > Hi John...
> > >
> > > Thanks....I had simply mis-typed it (guess I should use
> > > cut-and-paste)....but since it does not work for me, I'm attaching a
> > > zip file with the 2 nc files and the attempt at ncml...
> > >
> > > Thanks ahead for your help!!
> > >
> > > tom
> > >
> > > On Fri, Jun 10, 2011 at 4:11 PM, Unidata netCDF Java Support
> > > <address@hidden> wrote:
> > >> hi tom:
> > >>
> > >> jointExisting should be joinExisting.
> > >>
> > >> if thats not the actual problem, can you send me 2 files to check?
> > >>
> > >> john
> > >>
> > >>> I have several files that I want to aggregate over time. ÂEach file
> > >>> has two Groups, with a dimension "time" which is used for the desired
> > >>> variable...for example:
> > >>>
> > >>> Group proj1 {
> > >>> dimensions:
> > >>> time = 1;
> > >>> y = 3712;
> > >>> x = 3712;
> > >>> variables:
> > >>> float Instantaneous_Rain_Rate(time=1, y=3712, x=3712);
> > >>> ...
> > >>>
> > >>> I am trying to use NCML to accomplish this, with an "aggregation" 
> > >>> element like:
> > >>>
> > >>> <aggregation dimName="time" type="jointExisting">
> > >>> ...
> > >>>
> > >>> But apparently "time" is not the right syntax for this. ÂI've tried
> > >>> "proj1/time" as well, but to no available. ÂI'm hoping you can help.
> > >>>
> > >>> Thanks!
> > >>>
> > >>> tom
> > >>>
> > >>> --
> > >>> Tom Whittaker
> > >>> University of Wisconsin-Madison
> > >>> Space Science & Engineering Center (SSEC)
> > >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > >>> 1225 W. Dayton Street
> > >>> Madison, WIÂ 53706Â USA
> > >>> ph: +1 608 262 2759
> > >>>
> > >>>
> > >>
> > >>
> > >> Ticket Details
> > >> ===================
> > >> Ticket ID: IUX-310433
> > >> Department: Support netCDF Java
> > >> Priority: Normal
> > >> Status: Open
> > >>
> > >>
> > >
> > >
> > >
> > > --
> > > Tom Whittaker
> > > University of Wisconsin-Madison
> > > Space Science & Engineering Center (SSEC)
> > > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > > 1225 W. Dayton Street
> > > Madison, WIÂ 53706Â USA
> > > ph: +1 608 262 2759
> > >
> >
> >
> >
> > --
> > Tom Whittaker
> > University of Wisconsin-Madison
> > Space Science & Engineering Center (SSEC)
> > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> > 1225 W. Dayton Street
> > Madison, WIÂ 53706Â USA
> > ph: +1 608 262 2759
> >
> >
> 


Ticket Details
===================
Ticket ID: IUX-310433
Department: Support netCDF Java
Priority: Normal
Status: Open

Attachment: MSG1.png
Description: PNG image

Attachment: MSG2.png
Description: PNG image


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.