update stardate 2349873.432874 ok, enhance actually already works, but i ran into another problem with joinExisting. So I retreated to joinNew using the kludge of extracting the date from the filename. this only works if the filenames are uniform: <?xml version='1.0' encoding='UTF-8'?> <netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'> <aggregation dimName='time' type='joinNew'> <variableAgg name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' /> <scan location='G:/work/blodgett/' suffix='.hdf' subdirs='false' dateFormatMark="MOD13Q1.A#yyyyDDD"/> </aggregation> </netcdf> you also have to list the variables you want aggregated explicitly; i only did one, you can add the others: <variableAgg name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' /> this form should work with the 4.3.16 release. let me know if this is adequate for you for now, and any problems. john ps: use this as reference: http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/AnnotatedSchema4.html#aggregation On 3/26/2013 1:29 PM, Blodgett, David wrote: > ... enhance ... enhance ... go back ... enhance ... > > Love it! > > Thanks John, so no easy path forward on aggregations yet, but the master > plan includes it? > > I'm happy with that. > > > On Tue, Mar 26, 2013 at 1:59 PM, John Caron <address@hidden > <mailto:address@hidden>> wrote: > > Hi David: > > I had a look at this again. I changed HdfEosModisConvention to use a > 1D instead of a scalar time coordinate. > > However, joinExisting isnt working because the dataset needs to be > "enhanced" (with HdfEosModisConvention) first, before the time > coordinate is available. Currently joinExisting aggregation doesnt > do that. > > I can probably hack that in. Really I want to get the GRID feature > collection working, but thats going to be a few months. Aggregation > is old and crufty. > > Anyway, Ill let you know if i can get "enhanced" working. > > > John > > > > On 3/20/2013 7:50 AM, Blodgett, David wrote: > > Morning John, > > I pulled down the latest thredds war and have been testing a few > things > out. Did you get temporal aggregation working on the structured > data at > all? Any hints on how to structure the ncml for that? > > Looks like the spatial stuff is working. Fantastic. > > Thanks! > > - Dave > > > > On Mon, Mar 4, 2013 at 9:07 AM, Blodgett, David > <address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>>> wrote: > > OK, we'll keep an eye out for the release, thanks so much John! > > No worries on the aggregation, the tiles are big enough > that I think > we can deal without aggregation. > > > On Mon, Mar 4, 2013 at 8:43 AM, John Caron > <address@hidden <mailto:address@hidden> > <mailto:address@hidden > <mailto:address@hidden>__>> wrote: > > On 3/1/2013 12:03 PM, Blodgett, David wrote: > > John, > > Really happy you added this stuff in, what's your > plan for > releasing it? > > > going to try to release 4.3.16 in next week or two. > > > > You think there's any chance that tiled aggregation > will work? > > > im reluctant to recommend using whats there currently. > But doing > it right is a challenge and wont be done soon. > > > > > - Dave > > > On Tue, Feb 26, 2013 at 8:30 AM, address@hidden > <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>> > <address@hidden <mailto:address@hidden> > > <mailto:address@hidden <mailto:address@hidden>>> > wrote: > > Yes the tiling scheme is consistent for all the > MODIS tiles. > > Thanks, > > Jason Werpy > Enterprise Architect > Information Dynamics > USGS/EROS > Sioux Falls, SD 57198 > phone: 605-594-2723 <tel:605-594-2723> > cell: 605-690-3576 <tel:605-690-3576> > fax: 605-594-2530 <tel:605-594-2530> > address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>> > > > NOTICE: This email may contain confidential, > proprietary, > or competition > sensitive bid or proposal procurement > information. Unauthorized > disclosure of this information may carry > criminal penalties as set forth > in the Procurement Integrity Act, 41 U.S.C. 423, as > amended. Further, > the unauthorized disclosure of certain > commercial information by civil > servants may result in fines or imprisonment > under the > Trade Secrets Act > (18 U.S.C. 1905). If you have received this > information in > error, please > delete it, including all copies, and notify the > sender of > the error > immediately. > > > On Feb 26, 2013, at 8:26 AM, "Blodgett, David" > <address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>>> wrote: > > In this case, the data is 16 day from about > 2000 to > present. So, on the order of 275 time > steps. There are > daily products and annual products though, > in wich case, > we are looking 12 to 4000+ time steps. > > Jason, is the tile scheme you sent over > used for all the > different resolutions? Looking at a few of > them it looks > like that is the case. > > - Dave > > > > On Tue, Feb 26, 2013 at 7:48 AM, John Caron > <address@hidden > <mailto:address@hidden> > <mailto:address@hidden > <mailto:address@hidden>__>> > > wrote: > > Heres an image of this file. > > questions about aggregations: how many > time steps > will be in each aggregation ? how many > different > aggregations (tiles?) will you serve? > > > > On 2/24/2013 6:55 PM, David Blodgett wrote: > > Sweet, thanks John. There was data > in the tile, > but was hard to find... BORING, I > know. Sorry > about that. > > Here's a tile over the great lakes > to give you a > coast to compare locations to. > > > http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__BROWSE.MOD13A1.A2000065.__h11v04.005.2008238024700.1.jpg > > <http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/BROWSE.MOD13A1.A2000065.h11v04.005.2008238024700.1.jpg> > > http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf > > <http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf> > > http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf.xml > > <http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf.xml> > > - Dave > > On Feb 24, 2013, at 6:33 PM, John > Caron wrote: > > Hi: > > Im hacking away at this, but > the datafiles i > downloaded _seem_ to have > nothing but missing > values. Or im doing something > wrong. Can you > send me a file that you know > has real data in > it. thanks > > John > > On 2/24/2013 9:22 AM, David > Blodgett wrote: > > Hi John, > > Actually, I was assuming > that the > sinusoidal projection was > in there, doh! > That was based on > gdal_translate creating > a geotiff with this > projection info: > > PROJCS["unnamed", > GEOGCS["Unknown datum > based upon the > custom spheroid", > > DATUM["Not_specified_based_on___custom_spheroid", > SPHEROID["Custom > spheroid",6371007.181,0]], > PRIMEM["Greenwich",0], > > UNIT["degree",0.__0174532925199433]], > PROJECTION["Sinusoidal"], > > PARAMETER["longitude_of___center",0], > > PARAMETER["false_easting",0], > PARAMETER["false_northing",0], > UNIT["metre",1, > AUTHORITY["EPSG","9001"]]] > > There's a georeferencing > section here > > (http://www.gdal.org/frmt___hdf4.html > <http://www.gdal.org/frmt_hdf4.html>) that > has a bit more info about > gdal's > implementation. Might be > something > helpful there? Looks like > the ODL text > you point to is where its > coming from. > > 4 and 5 are questions for > Jason and maybe > Jordan. Guys? > > - Dave > > > > > > On Feb 23, 2013, at 1:22 > PM, John Caron > wrote: > > > On 2/22/2013 2:07 PM, > David Blodgett > wrote: > > Welcome back John. > Hope you had a > good trip! > > thanks! > > I actually just > spent a bunch of > time with this the > other day and have > been meaning to > summarize what > I'm finding. > > The first issue we > are running > into is your #1 > below. In this > case there > is an external xml > file that > contains a lot of > metadata, including > information about > the projection > (sinusoidal), which > is currently > unsupported by > NetCDF-Java? > > 1) where do you get > this xml file? > > 2) im looking at > > > MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf > > MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf.xml > > in your union > subdirectory. The xml > file does not seem to > have projection > info in it. > > 3) the internal ODL text > (StructMetadata) has > > ... > > UpperLeftPointMtrs=(-20015109.__354000,1111950.519667) > > LowerRightMtrs=(-18903158.__834333,-0.000000) > Projection=GCTP_SNSOID > > ProjParams=(6371007.181000,0,__0,0,0,0,0,0,0,0,0,0,0) > > SphereCode=-1 > > ... > > which may be enough to > calculate the > lat/lon values. Since > this is the > best choice, lets > investigate it > before we go down any > other paths. > > 4) Can anyone track > down the code in > the hyrax handler that > does this? Or > does the hyrax handler > actually link > in the hds-eos libraries ?? > > 5) how confident are > you that all the > files that you want to > aggregate have > the same lat/lon > geolocation ? > > > John > > > > > > > >
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.
