THREDDS Data Repository

A data repository needs to have the following "atomic" capabilities:

Storage Locator
Data Mover
ID Generator
Name Resolver
Metadata Generator
Metadata Crosswalk
Cataloger

The THREDDS Data Repository (TDR) is designed to provide a single interface to these operations so that a process such as "putData" can encapsulate all this functionality behind a single, "atomic" API call.

The TDR defines a Java (and eventually Web Service) interface for each of these components such that alternate implementations can be plugged in. For example, the Storage Locator implementation could simply return some scratch space or it could use user preferences and resource management to find the optimal storage location.

A subset of these capabilities, without the metadata and catalog, can be described as a "Storage Repository".

High Level API

The goal of the TDR is to provide a very simple high level interface to support users with a drag and drop mentality. In addition, we want to be able to expose other APIs to the more advanced user.

The TDR API is also designed to support an abstract concept of Data that goes well beyond a file centric view. Data in the TDR is any geo-science data that can be represented by a URL. It could be a file, a collection of files, a subset of a file, a database result set, or an in-memory representation of the data.

Proposed Interfaces

These interfaces are being used in early prototypes of the THREDDS Data Repository. They are not written in stone and we are very willing to discuss modifications to the API. Optional arguments are in "[]".

THREDDS Data Repository:

putData(source:URL, logicalName:String, [user:User], [metadata:Metadata]):UniqueID
putData(data:Data, logicalName:String, [user:User], [metadata:Metadata]):UniqueID
getData(id:UniqueID, [user:User], [protocol:String]):Data
getData(logicalName:String, [user:User], [protocol:String]):Data
getDataURL(id:UniqueID, [user:User], [protocol:String]):URL
getDataURL(logicalName:String, [user:User], [protocol:String]):URL
retrieveData(id:UniqueID, destination:URL, [user:User])
retrieveData(logicalName:String, destination:URL, [user:User])

Component Interfaces

Storage Locator

getStorageLocation([user:User], [size:int], [protocol:String]):URL

Data Mover

moveData(source:URL, destination:URL, [user:User])

ID Generator

generateUniqueID():UniqueID

Metadata Generator

generateMetadata(data:URL, [metadata:Metadata]):Metadata

Metadata Crosswalk

translateMetadata(metadata:Metadata, targetSchema:String):Metadata

Cataloger

catalog(logicalName:String, metadata:Metadata, [user:User])
getMetadata(logicalName:String, [user:User]):Metadata

Name Resolver

register(id:UniqueID, location:URL)
getLocation(id:UniqueID):URL
getAllLocations(id:UniqueID):List

Supporting Interfaces

User

Metadata

UniqueID

Data

TDR and LEAD

To determine how the Storage Repositiry or TDR might best work for LEAD, we need to better understand the capabilities and interfaces of existing technologies (e.g. RLS, trebuchet).

For example:

Can RLS implement the Name Resolver interface?
Can trebuchet implement the Data Mover interface?
Can myLEAD implement the Cataloger interface?

			Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
			P.O. Box 3000 • Boulder, CO 80307-3000 USA • Tel: 303-497-8643 • Fax: 303-497-8690