Unidata IDD Architecture

The primary goal of the IDD is to deliver data reliably to networks of computers in the departments of participating universities. The receiving sites specify the data products of interest, and the IDD delivers those products to the local site as soon as possible after the data are available from the source. From the beginning, the system was based on a set of principles adopted early on by the Unidata Advanced Technical Advisory Committee (ATAC ), and the Unidata Policy Committee.

Briefly stated the IDD Principles are:

1. Data reception implies relay responsibilities
2. The UPC acquires data of very high interest
3. The UPC chooses routes for high-interest data
4. Routing is ad hoc for data of lesser interest
5. The high-interest category is defined by actual use
6. Incentives and criteria exist for high-level relays
7. The LDM design facilitates a community endeavor
8. The Internet will evolve to simplify the IDD

Fan Out for Scalability

Considering only one data source, the system is designed to use a heirarchical distribution scheme as shown in the Fan Out diagram. This design addresses the issue of scalability. The source moves the data to a fixed number of top-level relay nodes; these relays in turn move the data to end user sites or to a second level of relay nodes and so on until the data products reach their destination at "leaf nodes."
The main point is that the number of downstream nodes served by any given relay is fixed, so the system scales with the number of end user or leaf nodes if:

• a sufficient number of relay sites can be set up,
• computer systems at relay nodes are sufficiently reliable
• the network connections between nodes are sufficiently reliable
• the software used for moving the data can deliver data reliably in the face of typical problems with computer systems and the underlying networks.

Flexibility

As the Network Distribution Schematic shows, the IDD approach allows any site to act as a data source. A Unidata software package called the Local Data Manager (LDM) uses remote procedure calls (RPC) to inject the data into the system at source sites, to capture it and pass it on to downstream nodes at the relays, and to capture the data at the "leaf" nodes. The full LDM runs on Unix systems, but a special receive-only version of it can be used on OS/2 systems at leaf nodes.

The Older Approach

Contrast the flexibility of the IDD with the previously used satellite broadcast system, where all datastreams were sent over special communications lines to the uplink site where they were broadcast via the satellite.

The satellite system worked well for delivering data quickly to many sites, but proved to be inflexible (or expensive) in terms of making new data available. This was especially true at the stage where only a few sites were interested in experimenting with a new class of data. The IDD technology enables any site to become a data source for products that only a few other sites may be interested in initially.