I've finally had time to test this out
New-style featureCollection FMRC:
http://64.9.200.113:8080/thredds/catalog/LakeErieSST-FMRC/catalog.html
Old-style joinExisting Agg:
http://64.9.200.113:8080/thredds/aoc.html?dataset=LakeErieSST-Agg
The TDS catalog entries: https://gist.github.com/4164765
The NetCDF files are one satellite pass per file. The time dimension is
unlimited with a length of 1 in the individual files. The time units are
seconds since epoch (all even seconds).
The joinExisting Agg works as expected just like in 4.2. All of the services
(including WMS) work.
The featureCollection FMRC doesn't work quite as well, but I do get a valid
dataset.
* There are rounding issues when converting the original files' "seconds
since 1970-01-01 00:00:00" to the FMRC's "hours since
2012-03-11T08:00:00.000Z". Data is no longer on even seconds in the FMRC.
Agg: http://64.9.200.113:8080/thredds/dodsC/LakeErieSST-Agg.ascii?time[0:1:132]
VS. FMRC:
http://64.9.200.113:8080/thredds/dodsC/LakeErieSST-FMRC/LakeErieSST-FMRC_best.ncd.ascii?time[0:1:132]
* The datasets has a "time_run" variable even though there are no "runs". I
can't remove the "time_run" variable from the resulting dataset using the
protoDataset tag.
* The "coordinates" attribute of the "sst" variable gets set to "time_run
time lon lat". I don't want 'time_run' to be part of the coordinates. I can't
change the attribute using the protoDataset tag.
* The "File_Access" link takes me to a directory listing that doesn't apply
the regex of the collection. It lists all of the files in the directory rather
than just the .nc4 files:
http://64.9.200.113:8080/thredds/catalog/LakeErieSST-FMRC/files/catalog.html.
* The WMS service does not work (it is a known issues with ncWMS and FMRC
datasets... it does not handle the run dimension correctly).
Is a possible solution to create another option in the fmrcConfig's
datasetTypes attribute to remove all concept of a "run" from the dataset?
This will be useful for gridded files with no overlapping time. <fmrcConfig
datasetTypes="NoRunsPlzKThnx Files" />
Another solution could be to remove the concept of a "run" from a FMRC dataset
if the calculated time_offset variable is filled with all zeros.
Thoughts?
> -----Original Message-----
> From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx]
> Sent: Friday, October 26, 2012 7:48 PM
> To: Kyle Wilcox
> Cc: Roy Mendelssohn; thredds@xxxxxxxxxxxxxxxx; Charlton Galvarino
> (2Creek)
> Subject: Re: [thredds] Scan Aggregation vs. Feature Collection
>
> Hi Kyle:
>
> For GRID data aggregations, If they are GRIB format, use Feature Collection,
> type=GRIB. Otherwise use type=FMRC, even if theres only one runtime. Let us
> know any problems.
>
> John
>
> On 10/3/2012 10:09 AM, Kyle Wilcox wrote:
> > Hi John -
> >
> > Just trying to clarify a few things when testing 4.3:
> >
> > If we have a homogenous set of non-forecast GRID FeatureTypes (satellite),
> we should still use an 'old-style' NcML aggregation because there is no GRID
> FeatureCollection.
> >
> > If we have a heterogeneous set of non-forecast GRID FeatureTypes (satellite
> passes on slightly different grids), there is currently no way to aggregate
> them
> in 4.3 because there is no GRID FeatureCollection.
> >
> > Thanks ,
> > Kyle
> >
> >> -----Original Message-----
> >> From: thredds-bounces@xxxxxxxxxxxxxxxx [mailto:thredds-
> >> bounces@xxxxxxxxxxxxxxxx] On Behalf Of John Caron
> >> Sent: Monday, July 23, 2012 5:50 PM
> >> To: Roy Mendelssohn
> >> Cc: thredds@xxxxxxxxxxxxxxxx
> >> Subject: Re: [thredds] Scan Aggregation vs. Feature Collection
> >>
> >> Well, its complicated.
> >>
> >> Old style aggregations, aka "NcML aggegations" remain supported. I
> >> think of these as low level index manipulations. These are embedded
> >> in NcML, either on the client or in TDS config catalog, see:
> >>
> >> http://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.3/tutorial/NcM
> >> L.ht
> >> m
> >>
> >> We can do much better when we know the "feature type", and the new
> >> way is called "feature collections":
> >>
> >> 1) type=FMRC is for gridded data. Works mostly of in 4.2, but
> >> refactored in 4.2 and still testing.
> >> 2) type=GRIB is for GRIB data, which is a special kind of gridded data.
> >> This supersedes FMRC when the data is in GRIB. Only in 4.3, ready for
> >> beta test.
> >> 3) type=point are for point feature types. Still in alpha in 4.3
> >>
> >> NcML works fins when you know what you are doing and you have
> >> homogenous files. feature collections are intededd to be easier to
> >> use and tolerate heterogeneous files.
> >>
> >> John
> >>
> >>
> >> On 7/23/2012 3:27 PM, Roy Mendelssohn wrote:
> >>> But is this just for FMRC aggregation, or for all aggregations? I
> >>> could swear I
> >> remember a web age with feature collections that "grid" was a feature
> >> type, but when I looked the other day, it was point, station and grib I
> believe.
> >> Maybe it was my old age and bad vision seeing "grid" for "grib". If
> >> there is an aggregation for featureType=grid, can you point me to the
> >> page that describes this.
> >>>
> >>> We do not have any FMRC aggregations, but if there all of our
> >>> aggregations
> >> should be feature collections, then I have a lot of editing to do on
> >> our xml files.
> >>>
> >>> Thanks,
> >>>
> >>> -Roy
> >>>
> >>> On Jul 23, 2012, at 1:59 PM, John Caron wrote:
> >>>
> >>>> On 7/19/2012 11:50 AM, Roy Mendelssohn wrote:
> >>>>> Hi All:
> >>>>>
> >>>>> I have asked this before and am not certain that I ever got a response.
> >> For TDS 3.0 is it correct to assume that scan aggregation is
> >> deprecated and we should be using feature collections for aggregation
> instead?
> >>>> Hi Roy:
> >>>>
> >>>> Yes correct, though we are still debugging the FMRCs in 4.3.
> >>>>
> >>>> For GRIB files, you really want to use feature collections , type = GRIB.
> >>>>
> >>>> 4.3.12 is the last alpha, next release will be beta. It should be
> >>>> ready this
> >> week.
> >>>>
> >>>> Would appreciate use and feedback.
> >>>>
> >>>> John
> >>>>
> >>>> _______________________________________________
> >>>> thredds mailing list
> >>>> thredds@xxxxxxxxxxxxxxxx
> >>>> For list information or to unsubscribe, visit:
> >>>> http://www.unidata.ucar.edu/mailing_lists/
> >>> **********************
> >>> "The contents of this message do not reflect any position of the U.S.
> >> Government or NOAA."
> >>> **********************
> >>> Roy Mendelssohn
> >>> Supervisory Operations Research Analyst NOAA/NMFS Environmental
> >>> Research Division Southwest Fisheries Science Center
> >>> 1352 Lighthouse Avenue
> >>> Pacific Grove, CA 93950-2097
> >>>
> >>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> >>> voice: (831)-648-9029
> >>> fax: (831)-648-8440
> >>> www: http://www.pfeg.noaa.gov/
> >>>
> >>> "Old age and treachery will overcome youth and skill."
> >>> "From those who have been given much, much will be expected"
> >>> "the arc of the moral universe is long, but it bends toward justice" -MLK
> Jr.
> >>
> >> _______________________________________________
> >> thredds mailing list
> >> thredds@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe, visit:
> >> http://www.unidata.ucar.edu/mailing_lists/
