Kyle,
Thanks for posting this to the THREDDS list. Just today a colleague
finalizing her poster for AGU asked me why the WMS command that used
to work (in the pre-collection days) for her FMRC no longer works.
I had no idea, and spent an hour trying to figure it out before
giving up. I guess that "time_run" variable is the culprit. Didn't
know "new style FMRC" + WMS was a "known" problem. Hope it gets
fixed soon.
-Rich
On Wed, Nov 28, 2012 at 5:07 PM, Kyle Wilcox <KWilcox@xxxxxxxxxxxxxx> wrote:
> I've finally had time to test this out
>
> New-style featureCollection FMRC:
> http://64.9.200.113:8080/thredds/catalog/LakeErieSST-FMRC/catalog.html
> Old-style joinExisting Agg:
> http://64.9.200.113:8080/thredds/aoc.html?dataset=LakeErieSST-Agg
>
> The TDS catalog entries: https://gist.github.com/4164765
>
> The NetCDF files are one satellite pass per file. The time dimension is
> unlimited with a length of 1 in the individual files. The time units are
> seconds since epoch (all even seconds).
>
> The joinExisting Agg works as expected just like in 4.2. All of the services
> (including WMS) work.
>
> The featureCollection FMRC doesn't work quite as well, but I do get a valid
> dataset.
> * There are rounding issues when converting the original files' "seconds
> since 1970-01-01 00:00:00" to the FMRC's "hours since
> 2012-03-11T08:00:00.000Z". Data is no longer on even seconds in the FMRC.
> Agg:
> http://64.9.200.113:8080/thredds/dodsC/LakeErieSST-Agg.ascii?time[0:1:132]
> VS. FMRC:
> http://64.9.200.113:8080/thredds/dodsC/LakeErieSST-FMRC/LakeErieSST-FMRC_best.ncd.ascii?time[0:1:132]
> * The datasets has a "time_run" variable even though there are no "runs".
> I can't remove the "time_run" variable from the resulting dataset using the
> protoDataset tag.
> * The "coordinates" attribute of the "sst" variable gets set to "time_run
> time lon lat". I don't want 'time_run' to be part of the coordinates. I
> can't change the attribute using the protoDataset tag.
> * The "File_Access" link takes me to a directory listing that doesn't apply
> the regex of the collection. It lists all of the files in the directory
> rather than just the .nc4 files:
> http://64.9.200.113:8080/thredds/catalog/LakeErieSST-FMRC/files/catalog.html.
> * The WMS service does not work (it is a known issues with ncWMS and FMRC
> datasets... it does not handle the run dimension correctly).
>
> Is a possible solution to create another option in the fmrcConfig's
> datasetTypes attribute to remove all concept of a "run" from the dataset?
> This will be useful for gridded files with no overlapping time. <fmrcConfig
> datasetTypes="NoRunsPlzKThnx Files" />
>
> Another solution could be to remove the concept of a "run" from a FMRC
> dataset if the calculated time_offset variable is filled with all zeros.
>
> Thoughts?
>
>
>
>> -----Original Message-----
>> From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx]
>> Sent: Friday, October 26, 2012 7:48 PM
>> To: Kyle Wilcox
>> Cc: Roy Mendelssohn; thredds@xxxxxxxxxxxxxxxx; Charlton Galvarino
>> (2Creek)
>> Subject: Re: [thredds] Scan Aggregation vs. Feature Collection
>>
>> Hi Kyle:
>>
>> For GRID data aggregations, If they are GRIB format, use Feature Collection,
>> type=GRIB. Otherwise use type=FMRC, even if theres only one runtime. Let us
>> know any problems.
>>
>> John
>>
>> On 10/3/2012 10:09 AM, Kyle Wilcox wrote:
>> > Hi John -
>> >
>> > Just trying to clarify a few things when testing 4.3:
>> >
>> > If we have a homogenous set of non-forecast GRID FeatureTypes (satellite),
>> we should still use an 'old-style' NcML aggregation because there is no GRID
>> FeatureCollection.
>> >
>> > If we have a heterogeneous set of non-forecast GRID FeatureTypes (satellite
>> passes on slightly different grids), there is currently no way to aggregate
>> them
>> in 4.3 because there is no GRID FeatureCollection.
>> >
>> > Thanks ,
>> > Kyle
>> >
>> >> -----Original Message-----
>> >> From: thredds-bounces@xxxxxxxxxxxxxxxx [mailto:thredds-
>> >> bounces@xxxxxxxxxxxxxxxx] On Behalf Of John Caron
>> >> Sent: Monday, July 23, 2012 5:50 PM
>> >> To: Roy Mendelssohn
>> >> Cc: thredds@xxxxxxxxxxxxxxxx
>> >> Subject: Re: [thredds] Scan Aggregation vs. Feature Collection
>> >>
>> >> Well, its complicated.
>> >>
>> >> Old style aggregations, aka "NcML aggegations" remain supported. I
>> >> think of these as low level index manipulations. These are embedded
>> >> in NcML, either on the client or in TDS config catalog, see:
>> >>
>> >> http://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.3/tutorial/NcM
>> >> L.ht
>> >> m
>> >>
>> >> We can do much better when we know the "feature type", and the new
>> >> way is called "feature collections":
>> >>
>> >> 1) type=FMRC is for gridded data. Works mostly of in 4.2, but
>> >> refactored in 4.2 and still testing.
>> >> 2) type=GRIB is for GRIB data, which is a special kind of gridded data.
>> >> This supersedes FMRC when the data is in GRIB. Only in 4.3, ready for
>> >> beta test.
>> >> 3) type=point are for point feature types. Still in alpha in 4.3
>> >>
>> >> NcML works fins when you know what you are doing and you have
>> >> homogenous files. feature collections are intededd to be easier to
>> >> use and tolerate heterogeneous files.
>> >>
>> >> John
>> >>
>> >>
>> >> On 7/23/2012 3:27 PM, Roy Mendelssohn wrote:
>> >>> But is this just for FMRC aggregation, or for all aggregations? I
>> >>> could swear I
>> >> remember a web age with feature collections that "grid" was a feature
>> >> type, but when I looked the other day, it was point, station and grib I
>> believe.
>> >> Maybe it was my old age and bad vision seeing "grid" for "grib". If
>> >> there is an aggregation for featureType=grid, can you point me to the
>> >> page that describes this.
>> >>>
>> >>> We do not have any FMRC aggregations, but if there all of our
>> >>> aggregations
>> >> should be feature collections, then I have a lot of editing to do on
>> >> our xml files.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> -Roy
>> >>>
>> >>> On Jul 23, 2012, at 1:59 PM, John Caron wrote:
>> >>>
>> >>>> On 7/19/2012 11:50 AM, Roy Mendelssohn wrote:
>> >>>>> Hi All:
>> >>>>>
>> >>>>> I have asked this before and am not certain that I ever got a response.
>> >> For TDS 3.0 is it correct to assume that scan aggregation is
>> >> deprecated and we should be using feature collections for aggregation
>> instead?
>> >>>> Hi Roy:
>> >>>>
>> >>>> Yes correct, though we are still debugging the FMRCs in 4.3.
>> >>>>
>> >>>> For GRIB files, you really want to use feature collections , type =
>> >>>> GRIB.
>> >>>>
>> >>>> 4.3.12 is the last alpha, next release will be beta. It should be
>> >>>> ready this
>> >> week.
>> >>>>
>> >>>> Would appreciate use and feedback.
>> >>>>
>> >>>> John
>> >>>>
>> >>>> _______________________________________________
>> >>>> thredds mailing list
>> >>>> thredds@xxxxxxxxxxxxxxxx
>> >>>> For list information or to unsubscribe, visit:
>> >>>> http://www.unidata.ucar.edu/mailing_lists/
>> >>> **********************
>> >>> "The contents of this message do not reflect any position of the U.S .
>> >> Government or NOAA."
>> >>> **********************
>> >>> Roy Mendelssohn
>> >>> Supervisory Operations Research Analyst NOAA/NMFS Environmental
>> >>> Research Division Southwest Fisheries Science Center
>> >>> 1352 Lighthouse Avenue
>> >>> Pacific Grove, CA 93950-2097
>> >>>
>> >>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>> >>> voice: (831)-648-9029
>> >>> fax: (831)-648-8440
>> >>> www: http://www.pfeg.noaa.gov/
>> >>>
>> >>> "Old age and treachery will overcome youth and skill."
>> >>> "From those who have been given much, much will be expected"
>> >>> "the arc of the moral universe is long, but it bends toward justice" -MLK
>> Jr.
>> >>
>> >> _______________________________________________
>> >> thredds mailing list
>> >> thredds@xxxxxxxxxxxxxxxx
>> >> For list information or to unsubscribe, visit:
>> >> http://www.unidata.ucar.edu/mailing_lists/
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
Dr. Richard P. Signell (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598
