RAMADDA Status Report
Winter 2007/2008
Jeff McWhirter
April 4, 2008
RAMADDA
RAMADDA (Repository for Archiving, Managing and Accessing Diverse DAta) is a
is a new effort at the UPC. This project, spearheaded by Jeff McWhirter, began in December 2007.
The goal of this project is to provide an open and extensible data repository framework/server.
RAMADDA is Java-based and uses a relational database as its underlying metadata storage facility.
It has been designed to run both stand-alone as well as running under a Tomcat server.
- To manage and provide access to an IDD/LDM real time archive.
- To support the development of community centric case studies. More generally, this is the "youtube of data" scenario. RAMADDA provides a rich environment
to support end users in publishing, organizing and sharing data and other information.
- Embedded in end-user clients (e.g., the Integreated Data Viewer) to provide a richer level of data access services for local disk-based data holdings.
- Field project support.
And provides a suite of comprehensive data management, archiving and repository services:
- Data file ingest, organization, meta-data creation and access control.
- Search and browse capabilities.
- Catalog and RSS feeds.
- Event notification.
- etc.
RAMADDA and TDR
This work builds on and advances the prior efforts on the Thredds Data Repository (TDR). The focus of the TDR was on providing data upload and
management facilities for the Thredds Data Server (TDS). As this work progressed it became clear that the scope of required functionality went
beyond the specific needs of the TDS that the TDR provides.
Current Status
A prototype implementation has been developed and initial use of it has begun by Unidata staff members. This implementation supports:
- An extensible data and metadata type system
- A variety of ingest mechanisms (user oriented upload, file system scanning, programmatic web service)
- Basic CRUD (Create, Read, Update, Delete) facilities
- A variety of navigational and display/presentation facilities (e.g., timeline, RSS feeds, Thredds catalog generation, image player, etc.)
- Access control mechanism and user management
- Metadata search
- Web site templating
We have begun preliminary work using the IDV to generate image and movie products within the repository
Performance testing has been done, using an internal IDD/LDM archive. We have a repository with 7 million entries in the database. We have also
investigated linking the LDM with RAMADDA to support real-time event notification of incoming data products. The Level-III was used which averages
5 data files/second. RAMADDA was able to support this load.
We are engaged in discussion with UCAR/EOL analyzing the feasability to use RAMADAA as the underpinnings of their next generation
field catalog and data management server.
Jeff McWhirter
