Re: [thredds] Search or Query Main Catalog

Hi Kevin:

If I understand what you want, you have several options, depending on how much 
programming you want to do.

1.  You can use the ncISO tool at http://www.ngdc.noaa.gov/eds/tds/.  However,, 
while this gives you all the metadata it will not give you the results based on 
a search response.  You would have to implement that.

2.  You can use GI-cat, preferably after adding the ncISO service to your 
thredds  (this will produce more metadata than otherwise), have GI-cat harvest 
this information, and then use one of the services that GI-cat supports to 
query GI-cat.  This will return the URLs of the data that meet your search 
criteria.   You would then have to send those requests to the TDS to actually 
get the data.

HTH,

-Roy



On Mar 26, 2011, at 8:28 AM, Kevin Manross wrote:

> 
> I've been digging around with GI-cat and it is very impressive.  I'm afraid 
> that it may be a bit to "heavy weighted" for my needs, and/or not quite what 
> I need.  However, there are some features that I may use (i.e., the eXist 
> library).
> 
> I also stumbled upon the thredds.catalog.crawl and thredds.catalog.query 
> packages.  I'm wondering if I could leverage this to crawl through the 
> hierarchical THREDDS catalog and extract the products/date-times that I'm 
> interested in.  
> 
> My next step would be to extract a bounding box, and then perform an 
> operation (i.e. getting the max value at each common gridpoint across all 
> aggregated netCDF files matching my query).
> 
> Perhaps this question is better suited for the netcdf-java mail list, but is 
> the crawl and query tools intended to be used this way?  How efficient are 
> they?
> 
> Many thanks!
> 
> -kevin.
> 
> 
> On Wed, Mar 23, 2011 at 1:16 PM, Kevin Manross <kevin.manross@xxxxxxxxx> 
> wrote:
> Very intriguing!!
> 
> I'm in a meeting and trying to view this on my phone. What output
> format(s) does GI-Cat provide?  Is it mainly graphical, or can this
> produce also produce CF netCDF?
> 
> -kevin.
> 
> (Sent from my iPhone)
> 
> On Mar 23, 2011, at 13:10, Rich Signell <rsignell@xxxxxxxx> wrote:
> 
>> Kevin,
>> Just to follow up on Roy's comments: GI-CAT will harvest either the
>> THREDDS metadata or the ncISO metadata (THREDDS or THREDDS-ncISO).
>> 
>> ncISO has the advantage that if your data is CF-compliant, the
>> geographic and time extents will be computed directly from the
>> coordinate variables in the THREDDS datasets.
>> 
>> See my 5 minute instructional video here for how to set up GI-CAT to
>> harvest THREDDS ncISO metadata.  Soon to go viral!  ;-)
>> http://www.youtube.com/watch?v=28biJHTQSrM
>> 
>> -Rich
>> 
>> On Wed, Mar 23, 2011 at 1:58 PM, Roy Mendelssohn
>> <Roy.Mendelssohn@xxxxxxxx> wrote:
>>> GI-cat : --http://essi-lab.eu/cgi-bin/twiki/view/GIcat
>>> 
>>> -Roy
>>> 
>>> On Mar 23, 2011, at 10:56 AM, Kevin Manross wrote:
>>> 
>>>> 
>>>> Howdy!
>>>> 
>>>> I think I may have asked this in person at one point, but I don't recall.  
>>>> Is there a way to search or query the main (and sub) catalog in TDS?
>>>> 
>>>> We have our data set up hierarchically as such:
>>>> 
>>>> 
>>>> UTCDAY/sensor/product/[subtype]/YYYYMMDD-HHMMSS.nc
>>>> 
>>>> 
>>>> It would be tremendously advantageous for extracting subsetted data from 
>>>> our archives if we would be able to query the TDS catalog and obtain the 
>>>> paths and/or create aggregations based on the query input.
>>>> 
>>>> For example, assume I have a TDS that has crawled through our data 
>>>> organized as above and created it's catalog structure.  If I wanted to get 
>>>> all the grids of MESH (multi-radar maximum expected size of hail) between 
>>>> 04 June 2009 21z and 06 June 2009 04 z, is there a way that I can query 
>>>> the catalog(s) to get the span of data I am requesting?
>>>> 
>>>> I'd be eager to hear if anything like this is possible - or how i might 
>>>> set up my configuration to possibly allow for something similar.
>>>> 
>>>> Many thanks!
>>>> 
>>>> -kevin.
>>>> 
>>>> --
>>>> +-----------------------------------------------------+
>>>> Kevin L. Manross           |  ** New Address **
>>>> CIMMS Research Associate   |     120 David L. Boren Bvd
>>>> NSSL : WRDD : SWAT         |     Rm 3923
>>>> 
>>>> <kevin.manross@xxxxxxxx>
>>>>   |     405.325.6385
>>>> 
>>>> www.cimms.ou.edu/~kmanross
>>>> |
>>>> "My opinions are my own and not representative of
>>>> CIMMS, NSSL, NOAA or any affiliates"
>>>> +-----------------------------------------------------+
>>>> 
>>>> 
>>>> _______________________________________________
>>>> thredds mailing list
>>>> thredds@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe,  visit: 
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>> 
>>> **********************
>>> "The contents of this message do not reflect any position of the U.S. 
>>> Government or NOAA."
>>> **********************
>>> Roy Mendelssohn
>>> Supervisory Operations Research Analyst
>>> NOAA/NMFS
>>> Environmental Research Division
>>> Southwest Fisheries Science Center
>>> 1352 Lighthouse Avenue
>>> Pacific Grove, CA 93950-2097
>>> 
>>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>>> voice: (831)-648-9029
>>> fax: (831)-648-8440
>>> www: http://www.pfeg.noaa.gov/
>>> 
>>> "Old age and treachery will overcome youth and skill."
>>> "From those who have been given much, much will be expected"
>>> 
>>> _______________________________________________
>>> thredds mailing list
>>> thredds@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe,  visit: 
>>> http://www.unidata.ucar.edu/mailing_lists/
>>> 
>> 
>> 
>> 
>> --
>> Dr. Richard P. Signell   (508) 457-2229
>> USGS, 384 Woods Hole Rd.
>> Woods Hole, MA 02543-1598
> 
> 
> 
> -- 
> +-----------------------------------------------------+
> Kevin L. Manross           |  ** New Address **
> CIMMS Research Associate   |     120 David L. Boren Bvd
> NSSL : WRDD : SWAT         |     Rm 3923
> 
> <kevin.manross@xxxxxxxx>
>   |     405.325.6385
> 
> www.cimms.ou.edu/~kmanross
> |
> "My opinions are my own and not representative of
> CIMMS, NSSL, NOAA or any affiliates"
> +-----------------------------------------------------+
> 
> 
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/

**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 



  • 2011 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: