[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #XLF-986148]: aggregating ensembles



youll have to do time first, since joins are always on the outer dimension.

these files have to be highly homogeneous - exact same coords, or else you will 
get wrong results. id probably write a program to double check that is true.

eventually i will write some agg code that will do that for you - and do the 
right thing is the coordinates dont match. for now, were stuck with these 
homgeneity constraints.

> Thanks.  I understand about the unzipping - hopefully they can provide
> datasets that are not zipped.
> 
> Aggregating by time will be important to get a view of the entire 21st
> century, not just individual decades.  If you could give a sample of
> what that would look like, I'd appreciate that.  In particular whether
> it's better to aggregate by time first and then ensemble, or ensemble
> then time.
> 
> Don
> 
> On 6/10/11 3:45 PM, Unidata THREDDS Support wrote:
> > You can generalize this:
> >
> > <?xml version='1.0' encoding='UTF-8'?>
> > <netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'>
> >    <variable name='ens' type='String' shape='ens'>
> >      <attribute name='long_name' value='ensemble coordinate' />
> >      <attribute name='_CoordinateAxisType' value='Ensemble' />
> >    </variable>
> >    <aggregation dimName='ens' type='joinNew'>
> >      <variableAgg name='tasmax'/>
> >      <netcdf 
> > location='E:/work/dmurray/A1B_HadCM3Q3_DM_25km_2001-2010_tasmax.nc.gz' 
> > coordValue='HadCM'/>
> >      <netcdf 
> > location='E:/work/dmurray/A1B_ECHAM5-r3_DM_25km_2001-2010_tasmax.nc.gz' 
> > coordValue='ECHAM5'/>
> >    </aggregation>
> > </netcdf>
> >
> > I would not do the time aggregation if possible, as i think it will slow 
> > things down. CDM will need to uncompressed the files which will be slow the 
> > first time. So it would be better to work on subsets, to minimize space.
> >
> >> So, for the site below, I would think you'd want to aggregate on the
> >> ensemble dimension for:
> >>
> >> *25km_2021-2030_tasmax.nc
> >>
> >> then you'd probably want to aggreggate across time (joinExisting) on
> >> time for:
> >>
> >> *_tasmax.nc
> >>
> >> of those ensembles.  More examples under a TDS are at:
> >>
> >> http://clima-dods.ictp.it/thredds/catalog/aap/ENSEMBLES_AMMA/catalog.html
> >>
> >> with a particular run at:
> >>
> >> http://clima-dods.ictp.it/thredds/catalog/aap/ENSEMBLES_AMMA/ERAIN/Months/catalog.html
> >>
> >> Don
> >>
> >> On 6/6/11 6:32 PM, Unidata THREDDS Support wrote:
> >>> ok, ive looked at some random files. can you give me an example of a list 
> >>> of files for different models that they want to aggregate? they will have 
> >>> to be on the exact same grid, is that true?
> >>>
> >>>> They are netCDF files.  Samples are at:
> >>>>
> >>>> http://clima-dods.ictp.it/repository/entry/show/Top/netCDF/d2/ENSEMBLES/A1B?max=500&entryid=509ba64f-eea3-4d6a-9fee-be908f770a77
> >>>>
> >>>> Don
> >>>>
> >>>> On 6/6/11 9:57 AM, Unidata THREDDS Support wrote:
> >>>>> what kind of files are these? We are working on some special mechanisms 
> >>>>> for GRIB2. Otherwise, ensemble support is pretty sketchy.
> >>>>>
> >>>>>
> >>>>>> Ethan/John-
> >>>>>>
> >>>>>> At the ICTP workshop this past week, the question came up about
> >>>>>> aggregating files into an ensemble.  I think this can be done through
> >>>>>> NcML (and thus TDS/IDV/RAMADDA), but I couldn't find an example in the
> >>>>>> docs (although I didn't look very closely).  They have different model
> >>>>>> runs that cover the same geographic/temporal domains, but they want to
> >>>>>> be able to view the files as one dataset in the IDV to be able to do
> >>>>>> ensemble means, etc.  This is akin to the SREF data accessible under
> >>>>>> this catalog:
> >>>>>>
> >>>>>> http://motherlode.ucar.edu/thredds/catalog/model/NCEP/SREF/CONUS_40km/pgrb_biasc/catalog.xml
> >>>>>>
> >>>>>> where each file contains the data for a particular ensemble member.  If
> >>>>>> I wanted to aggregate these files to look like a single dataset with an
> >>>>>> ensemble dimension, what would the NcML look like?
> >>>>>>
> >>>>>> Thanks for your help.
> >>>>>>
> >>>>>> Don
> >>>>>> --
> >>>>>> Don Murray
> >>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>> 303-497-3596
> >>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> Ticket Details
> >>>>> ===================
> >>>>> Ticket ID: XLF-986148
> >>>>> Department: Support THREDDS
> >>>>> Priority: High
> >>>>> Status: Open
> >>>>>
> >>>>
> >>>> --
> >>>> Don Murray
> >>>> NOAA/ESRL/PSD and CIRES
> >>>> 303-497-3596
> >>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>
> >>>>
> >>>
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: XLF-986148
> >>> Department: Support THREDDS
> >>> Priority: High
> >>> Status: Open
> >>>
> >>
> >> --
> >> Don Murray
> >> NOAA/ESRL/PSD and CIRES
> >> 303-497-3596
> >> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: XLF-986148
> > Department: Support THREDDS
> > Priority: High
> > Status: Open
> >
> 
> --
> Don Murray
> NOAA/ESRL/PSD and CIRES
> 303-497-3596
> http://www.esrl.noaa.gov/psd/people/don.murray/
> 
> 


Ticket Details
===================
Ticket ID: XLF-986148
Department: Support THREDDS
Priority: High
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.