Re: [thredds] Search or Query Main Catalog

Kevin,

If you have a hierarchical directory structure that is being scanned
by THREDDS Data Server using <datasetScan>, and you have the ncISO
service enabled, you can tell GI-CAT to harvest the metadata from the
top level catalog and it will harvest all the NcISO metadata in the
hierarchical directory structure.     Is that what you are asking?

If you have detailed GI-CAT questions, probably best to ask on the
GI-DEVEL mailing list:
<http://essi-lab.eu/cgi-bin/mailman/listinfo/gi-devel>

-Rich

On Sat, Mar 26, 2011 at 11:28 AM, Kevin Manross <kevin.manross@xxxxxxxx> wrote:
>
> I've been digging around with GI-cat and it is very impressive.  I'm afraid
> that it may be a bit to "heavy weighted" for my needs, and/or not quite what
> I need.  However, there are some features that I may use (i.e., the eXist
> library).
> I also stumbled upon the thredds.catalog.crawl
> and thredds.catalog.query packages.  I'm wondering if I could leverage this
> to crawl through the hierarchical THREDDS catalog and extract the
> products/date-times that I'm interested in.
> My next step would be to extract a bounding box, and then perform an
> operation (i.e. getting the max value at each common gridpoint across all
> aggregated netCDF files matching my query).
> Perhaps this question is better suited for the netcdf-java mail list, but is
> the crawl and query tools intended to be used this way?  How efficient are
> they?
> Many thanks!
> -kevin.
>
> On Wed, Mar 23, 2011 at 1:16 PM, Kevin Manross <kevin.manross@xxxxxxxxx>
> wrote:
>>
>> Very intriguing!!
>>
>> I'm in a meeting and trying to view this on my phone. What output
>> format(s) does GI-Cat provide?  Is it mainly graphical, or can this
>> produce also produce CF netCDF?
>>
>> -kevin.
>>
>> (Sent from my iPhone)
>>
>> On Mar 23, 2011, at 13:10, Rich Signell <rsignell@xxxxxxxx> wrote:
>>
>> > Kevin,
>> > Just to follow up on Roy's comments: GI-CAT will harvest either the
>> > THREDDS metadata or the ncISO metadata (THREDDS or THREDDS-ncISO).
>> >
>> > ncISO has the advantage that if your data is CF-compliant, the
>> > geographic and time extents will be computed directly from the
>> > coordinate variables in the THREDDS datasets.
>> >
>> > See my 5 minute instructional video here for how to set up GI-CAT to
>> > harvest THREDDS ncISO metadata.  Soon to go viral!  ;-)
>> > http://www.youtube.com/watch?v=28biJHTQSrM
>> >
>> > -Rich
>> >
>> > On Wed, Mar 23, 2011 at 1:58 PM, Roy Mendelssohn
>> > <Roy.Mendelssohn@xxxxxxxx> wrote:
>> >> GI-cat : --http://essi-lab.eu/cgi-bin/twiki/view/GIcat
>> >>
>> >> -Roy
>> >>
>> >> On Mar 23, 2011, at 10:56 AM, Kevin Manross wrote:
>> >>
>> >>>
>> >>> Howdy!
>> >>>
>> >>> I think I may have asked this in person at one point, but I don't
>> >>> recall.  Is there a way to search or query the main (and sub) catalog in
>> >>> TDS?
>> >>>
>> >>> We have our data set up hierarchically as such:
>> >>>
>> >>>
>> >>> UTCDAY/sensor/product/[subtype]/YYYYMMDD-HHMMSS.nc
>> >>>
>> >>>
>> >>> It would be tremendously advantageous for extracting subsetted data
>> >>> from our archives if we would be able to query the TDS catalog and obtain
>> >>> the paths and/or create aggregations based on the query input.
>> >>>
>> >>> For example, assume I have a TDS that has crawled through our data
>> >>> organized as above and created it's catalog structure.  If I wanted to 
>> >>> get
>> >>> all the grids of MESH (multi-radar maximum expected size of hail) 
>> >>> between 04
>> >>> June 2009 21z and 06 June 2009 04 z, is there a way that I can query the
>> >>> catalog(s) to get the span of data I am requesting?
>> >>>
>> >>> I'd be eager to hear if anything like this is possible - or how i
>> >>> might set up my configuration to possibly allow for something similar.
>> >>>
>> >>> Many thanks!
>> >>>
>> >>> -kevin.
>> >>>
>> >>> --
>> >>> +-----------------------------------------------------+
>> >>> Kevin L. Manross           |  ** New Address **
>> >>> CIMMS Research Associate   |     120 David L. Boren Bvd
>> >>> NSSL : WRDD : SWAT         |     Rm 3923
>> >>>
>> >>> <kevin.manross@xxxxxxxx>
>> >>>    |     405.325.6385
>> >>>
>> >>> www.cimms.ou.edu/~kmanross
>> >>>  |
>> >>> "My opinions are my own and not representative of
>> >>> CIMMS, NSSL, NOAA or any affiliates"
>> >>> +-----------------------------------------------------+
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> thredds mailing list
>> >>> thredds@xxxxxxxxxxxxxxxx
>> >>> For list information or to unsubscribe,  visit:
>> >>> http://www.unidata.ucar.edu/mailing_lists/
>> >>
>> >> **********************
>> >> "The contents of this message do not reflect any position of the U.S.
>> >> Government or NOAA."
>> >> **********************
>> >> Roy Mendelssohn
>> >> Supervisory Operations Research Analyst
>> >> NOAA/NMFS
>> >> Environmental Research Division
>> >> Southwest Fisheries Science Center
>> >> 1352 Lighthouse Avenue
>> >> Pacific Grove, CA 93950-2097
>> >>
>> >> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>> >> voice: (831)-648-9029
>> >> fax: (831)-648-8440
>> >> www: http://www.pfeg.noaa.gov/
>> >>
>> >> "Old age and treachery will overcome youth and skill."
>> >> "From those who have been given much, much will be expected"
>> >>
>> >> _______________________________________________
>> >> thredds mailing list
>> >> thredds@xxxxxxxxxxxxxxxx
>> >> For list information or to unsubscribe,  visit:
>> >> http://www.unidata.ucar.edu/mailing_lists/
>> >>
>> >
>> >
>> >
>> > --
>> > Dr. Richard P. Signell   (508) 457-2229
>> > USGS, 384 Woods Hole Rd.
>> > Woods Hole, MA 02543-1598
>
>
>
> --
>
> +-----------------------------------------------------+
> Kevin L. Manross           |  ** New Address **
> CIMMS Research Associate   |     120 David L. Boren Bvd
> NSSL : WRDD : SWAT         |     Rm 3923
> <kevin.manross@xxxxxxxx>   |     405.325.6385
> www.cimms.ou.edu/~kmanross |
> "My opinions are my own and not representative of
> CIMMS, NSSL, NOAA or any affiliates"
> +-----------------------------------------------------+
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>



-- 
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598



  • 2011 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: