[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #BTY-229773]: THREDDS - catalog generation of remote sites

Hi Katherine,

OK, I think I get the big picture. Still a few more questions though.

1) How do you currently access the data on the various machines? Mounted disks? 
Web servers? OPeNDAP?

2) Is access to the data restricted? 

3) Besides XML and HTML catalogs, sounds like you want to serve the data over 
OPeNDAP. Are you interested in WCS as well?

4) Sounds like you want to restrict access to catalogs as well as data. Is that 
right?

5) Are you running a stand-alone Tomcat (servlet container) or do you have an 
Apache server front-end?I

Thanks,

Ethan 

> Thanks Ethan
> 
> The scenario is this.
> We are a modelling centre with data here at our specific center, data on a
> e-grid computer and data on two supercomputers elsewhere in England (which
> as their contracts come and go move around)
> We are having a heck of a time keeping track of what is where, and keeping
> our web page catalog pages up to date. What I'd like to get back is, if the
> user is authenticated, get back an autogenerated catalog which the user
> could view the metadata in html, view the file (netcdf, grib) in the opendap
> form, use tools to browse (your IDV and toolsui) and click to download.
> 
> This centralized web page from our computational modelling support webpage
> would be the first/only page they come to when looking for data. There would
> be one catalog.xml page which would have datasetScan commands which would
> create a webpage that looks like this (for example):
> 
> Local Computer which I can install software on
> ECMWF Era-40
> Temperature Data - click and get list of ggat*.nc (or grb) files in a
> directory
> Pressure Data - click and get list of ggap*.nc (or grb) files in a
> directory
> Forecast Model Data - click and get list of ggfm*.nc (or grb) files in
> a directory
> ECMWF Era-15
> Temperature Data  - ""
> Pressure Data   - ""
> Forecast Model Data   - ""
> Misc Local Stuff
> ...
> 
> Egrid Computer which I can ask others install software on
> ECMWF Era-40
> Temperature Data - as above
> Pressure Data
> Forecast Model Data
> ECMWF Era-15
> Temperature Data
> Pressure Data
> Forecast Model Data
> Misc Egrid Stuff
> ...
> 
> Supercomputer One which it is not likely to install software on
> ECMWF Era-40
> Temperature Data - as above
> Pressure Data
> Forecast Model Data
> ECMWF Era-15
> Temperature Data
> Pressure Data
> Forecast Model Data
> Misc Supercomputer One Stuff
> ...
> 
> Supercomputer Two which it is not likely to install software on
> ECMWF Era-40
> Temperature Data - as above
> Pressure Data
> Forecast Model Data
> ECMWF Era-15
> Temperature Data
> Pressure Data
> Forecast Model Data
> Misc Supercomputer Two Stuff
> ...
> 
> >What is your current setup like? What do you get back from the location
> URLs above?
> >A web index page that lists data files that are accessible over HTTP?
> >It would certainly be possible to scan, generate catalogs, and serve from a
> TDS.
> >It would take some coding but we've tried to allow for this kind of
> extensibility.
> >The problem, similar to the OPeNDAP scanning I mention above, is that web
> index pages are not a standard format.
> >They are HTML but can vary from server to server. So, it is hard to write
> code that handles all variants.
> 
> >This would be an easy solution.
> >A THREDDS Data Servers (TDS) at each site and a central catalog that
> references with a catalogRef element the main catalog for each site.
> >Also, there are performance advantages to this route.
> >The single, central TDS would take a hit from accessing the data it served
> over HTTP rather than from local disk.
> >The distributed TDS could have local access to the data they serve.
> 
> Thing is I have no control over the supercomputers and can't install
> software on them most likely. I could put a catalog.xml in a directory, but
> no TDS or opendap.  That all has to be done from our home site...
> 
> Thanks for working with me on this.
> 
> Katherine
> 
> 


Ticket Details
===================
Ticket ID: BTY-229773
Department: Support THREDDS
Priority: High
Status: Open