hope so.
note that the dateFormatMark="MOD13Q1.A#yyyyDDD" is just counting chars, then applying the java SimpleDateFormat, so the date in the filename just has to start at character 9, it doesnt have to match MOD13Q1.A
On 3/26/2013 1:38 PM, Blodgett, David wrote:
Ahhh, so the "/" in the variable name will work to represent groups. I
didn't want to believe that would work, glad it will.
Thanks!
- Dave
On Tue, Mar 26, 2013 at 2:36 PM, John Caron <address@hidden> <mailto:address@hidden <mailto:address@hidden>>><mailto:address@hidden>> wrote:
update stardate 2349873.432874
ok, enhance actually already works, but i ran into another problem
with joinExisting.
So I retreated to joinNew using the kludge of extracting the date
from the filename.
this only works if the filenames are uniform:
<?xml version='1.0' encoding='UTF-8'?>
<netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'>
<aggregation dimName='time' type='joinNew'>
<variableAgg
name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' />
<scan location='G:/work/blodgett/' suffix='.hdf'
subdirs='false' dateFormatMark="MOD13Q1.A#yyyyDDD"/>
</aggregation>
</netcdf>
you also have to list the variables you want aggregated explicitly;
i only did one, you can add the others:
<variableAgg
name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' />
this form should work with the 4.3.16 release.
let me know if this is adequate for you for now, and any problems.
john
ps: use this as reference:
http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/AnnotatedSchema4.html#aggregation
On 3/26/2013 1:29 PM, Blodgett, David wrote:
> ... enhance ... enhance ... go back ... enhance ...
>
> Love it!
>
> Thanks John, so no easy path forward on aggregations yet, but the
master
> plan includes it?
>
> I'm happy with that.
>
>
> On Tue, Mar 26, 2013 at 1:59 PM, John Caron
<address@hidden <mailto:address@hidden>
wrote:
>
> Hi David:
>
> I had a look at this again. I changed HdfEosModisConvention
to use a
> 1D instead of a scalar time coordinate.
>
> However, joinExisting isnt working because the dataset needs
to be
> "enhanced" (with HdfEosModisConvention) first, before the time
> coordinate is available. Currently joinExisting aggregation
doesnt
> do that.
>
> I can probably hack that in. Really I want to get the GRID
feature
> collection working, but thats going to be a few months.
Aggregation
> is old and crufty.
>
> Anyway, Ill let you know if i can get "enhanced" working.
>
>
> John
>
>
>
> On 3/20/2013 7:50 AM, Blodgett, David wrote:
>
> Morning John,
>
> I pulled down the latest thredds war and have been
testing a few
> things
> out. Did you get temporal aggregation working on the
structured
> data at
> all? Any hints on how to structure the ncml for that?
>
> Looks like the spatial stuff is working. Fantastic.
>
> Thanks!
>
> - Dave
>
>
>
> On Mon, Mar 4, 2013 at 9:07 AM, Blodgett, David
> <address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>
> <mailto:address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>>> wrote:
>
> OK, we'll keep an eye out for the release, thanks so
much John!
>
> No worries on the aggregation, the tiles are big enough
> that I think
> we can deal without aggregation.
>
>
> On Mon, Mar 4, 2013 at 8:43 AM, John Caron
> <address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden>
> <mailto:address@hidden
<mailto:address@hidden>>__>> wrote:
>
> On 3/1/2013 12:03 PM, Blodgett, David wrote:
>
> John,
>
> Really happy you added this stuff in, what's
your
> plan for
> releasing it?
>
>
> going to try to release 4.3.16 in next week or two.
>
>
>
> You think there's any chance that tiled
aggregation
> will work?
>
>
> im reluctant to recommend using whats there
currently.
> But doing
> it right is a challenge and wont be done soon.
>
>
>
>
> - Dave
>
>
> On Tue, Feb 26, 2013 at 8:30 AM,
address@hidden <mailto:address@hidden>
> <mailto:address@hidden <mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>>
> <address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>
>
> <mailto:address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>>>
> wrote:
>
> Yes the tiling scheme is consistent for
all the
> MODIS tiles.
>
> Thanks,
>
> Jason Werpy
> Enterprise Architect
> Information Dynamics
> USGS/EROS
> Sioux Falls, SD 57198
> phone: 605-594-2723 <tel:605-594-2723>
> cell: 605-690-3576 <tel:605-690-3576>
> fax: 605-594-2530 <tel:605-594-2530>
> address@hidden <mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>
> <mailto:address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>>
>
>
> NOTICE: This email may contain confidential,
> proprietary,
> or competition
> sensitive bid or proposal procurement
> information. Unauthorized
> disclosure of this information may carry
> criminal penalties as set forth
> in the Procurement Integrity Act, 41
U.S.C. 423, as
> amended. Further,
> the unauthorized disclosure of certain
> commercial information by civil
> servants may result in fines or imprisonment
> under the
> Trade Secrets Act
> (18 U.S.C. 1905). If you have received this
> information in
> error, please
> delete it, including all copies, and
notify the
> sender of
> the error
> immediately.
>
>
> On Feb 26, 2013, at 8:26 AM, "Blodgett,
David"
> <address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>>> wrote:
>
> In this case, the data is 16 day
from about
> 2000 to
> present. So, on the order of 275 time
> steps. There are
> daily products and annual products
though,
> in wich case,
> we are looking 12 to 4000+ time steps.
>
> Jason, is the tile scheme you sent over
> used for all the
> different resolutions? Looking at a
few of
> them it looks
> like that is the case.
>
> - Dave
>
>
>
> On Tue, Feb 26, 2013 at 7:48 AM,
John Caron
> <address@hidden
<mailto:address@hidden>
> <mailto:address@hidden
<mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden>
> <mailto:address@hidden
<mailto:address@hidden>>__>>
>
> wrote:
>
> Heres an image of this file.
>
> questions about aggregations:
how many
> time steps
> will be in each aggregation ?
how many
> different
> aggregations (tiles?) will you
serve?
>
>
>
> On 2/24/2013 6:55 PM, David
Blodgett wrote:
>
> Sweet, thanks John. There
was data
> in the tile,
> but was hard to find...
BORING, I
> know. Sorry
> about that.
>
> Here's a tile over the great
lakes
> to give you a
> coast to compare locations to.
>
>
http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__BROWSE.MOD13A1.A2000065.__h11v04.005.2008238024700.1.jpg
>
<http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/BROWSE.MOD13A1.A2000065.h11v04.005.2008238024700.1.jpg>
>
http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf
>
<http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf>
>
http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf.xml
>
<http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf.xml>
>
> - Dave
>
> On Feb 24, 2013, at 6:33 PM,
John
> Caron wrote:
>
> Hi:
>
> Im hacking away at this, but
> the datafiles i
> downloaded _seem_ to have
> nothing but missing
> values. Or im doing
something
> wrong. Can you
> send me a file that you know
> has real data in
> it. thanks
>
> John
>
> On 2/24/2013 9:22 AM, David
> Blodgett wrote:
>
> Hi John,
>
> Actually, I was assuming
> that the
> sinusoidal
projection was
> in there, doh!
> That was based on
> gdal_translate creating
> a geotiff with this
> projection info:
>
> PROJCS["unnamed",
> GEOGCS["Unknown
datum
> based upon the
> custom spheroid",
>
> DATUM["Not_specified_based_on___custom_spheroid",
>
SPHEROID["Custom
>
spheroid",6371007.181,0]],
>
PRIMEM["Greenwich",0],
>
> UNIT["degree",0.__0174532925199433]],
>
PROJECTION["Sinusoidal"],
>
> PARAMETER["longitude_of___center",0],
>
> PARAMETER["false_easting",0],
>
PARAMETER["false_northing",0],
> UNIT["metre",1,
>
AUTHORITY["EPSG","9001"]]]
>
> There's a georeferencing
> section here
>
> (http://www.gdal.org/frmt___hdf4.html
> <http://www.gdal.org/frmt_hdf4.html>) that
> has a bit more info
about
> gdal's
> implementation. Might be
> something
> helpful there? Looks
like
> the ODL text
> you point to is
where its
> coming from.
>
> 4 and 5 are
questions for
> Jason and maybe
> Jordan. Guys?
>
> - Dave
>
>
>
>
>
> On Feb 23, 2013, at 1:22
> PM, John Caron
> wrote:
>
>
> On 2/22/2013
2:07 PM,
> David Blodgett
> wrote:
>
> Welcome back
John.
> Hope you had a
> good trip!
>
> thanks!
>
> I actually just
> spent a bunch of
> time with
this the
> other day and have
> been meaning to
> summarize what
> I'm finding.
>
> The first
issue we
> are running
> into is your #1
> below. In this
> case there
> is an
external xml
> file that
> contains a
lot of
> metadata, including
> information
about
> the projection
>
(sinusoidal), which
> is currently
> unsupported by
> NetCDF-Java?
>
> 1) where do you get
> this xml file?
>
> 2) im looking at
>
>
> MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf
>
> MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf.xml
>
> in your union
> subdirectory. The xml
> file does not
seem to
> have projection
> info in it.
>
> 3) the internal
ODL text
> (StructMetadata) has
>
> ...
>
>
UpperLeftPointMtrs=(-20015109.__354000,1111950.519667)
>
> LowerRightMtrs=(-18903158.__834333,-0.000000)
>
Projection=GCTP_SNSOID
>
> ProjParams=(6371007.181000,0,__0,0,0,0,0,0,0,0,0,0,0)
>
> SphereCode=-1
>
> ...
>
> which may be
enough to
> calculate the
> lat/lon values.
Since
> this is the
> best choice, lets
> investigate it
> before we go
down any
> other paths.
>
> 4) Can anyone track
> down the code in
> the hyrax
handler that
> does this? Or
> does the hyrax
handler
> actually link
> in the hds-eos
libraries ??
>
> 5) how confident are
> you that all the
> files that you
want to
> aggregate have
> the same lat/lon
> geolocation ?
>
>
> John
>
>
>
>
>
>
>
>