0.0 Introduction
The IDD reports and charts display different view points of the system that are hourly, system perspective, and a site perspective. The hourly reports and charts show the most detail but it is difficult to obtain long term trends without browsing much data. The system perspective shows the IDD as a complete unit without regard to individual site's performance. The site perspective shows the individual sites but it has drawbacks on interpretation of missing reports that cause performance distortions.
0.1 Analysis
Using the different results from the statistics it appears that a single conclusion cannot be made about the system as a whole. But, add the communication factor from the sites, and the conclusion is any site that wants data can obtain the desired data at 95%-100% reception rate if they have permission from an upstream site or the proper arrangements have been made.
Some of the statistic distortions that have been observed are some leaf sites only monitor the LDM status if there is a immediate need for data, otherwise the LDM is subjected to machine reboots, hardware failures, overuse, routing changes, etc. This is usually due to departmental tasks and time constraints placed on the site contact. On the positive side, the relay sites are more stable and aware of problems because of their community commitment to the IDD and their responsibility to the downstream sites.
Other aspects that distort the statistics are missing reports from the remote sites. The information has been included in different ways: 1) Counted as 0%, 2) Ignored, 3) Counted as 100%. Each option creates a different type of distortion, some charts show the effect of the options.
The network connection effects have become more apparent recently with the move from the NSFnet to commercial networks. The charts that only show the status of the relays often relate this information better because they have good network connections.
The IDD is a complex system that uses the statistic reports and charts to show different view points of the hardware aspects of the system infrastructure. To gain a better feel for the community IDD, monitor the system over a period of time including the IDD mailing lists.
0.2 Data
A set of active sites is used in combination with the data collected from the remote sites. The set contains information about the level in the hierarchy of IDD with regard to type of feed and if the site is a source, relay node or a leaf node.
1.0 Hourly statistics summary
This report is calculated hourly at 10 past the hour using a set of LDM active sites as a base with the raw data collected from the remote sites. The report calculates the last 24 hours so it includes late arriving data. It has the most detail because it displays the level of the site in the IDD routing hierarchy, the feeds, the number of products received, and the percentage of of total products sent from the source. For example:
Sink_1 drier.atmos.washington.edu DDS 1601 100.00
At the end there is a summary where the site feed data is combined into an overall site percentage. Each site is then categorized into the following columns:
Total 100% [99-100)% [95-99)% [80-95)% (0-80]% 0% LDM NO LDM's DOWN STATS
A site is placed in the NO STATS column if it is in the set of active sites and no report was received for the hour.
There is a overall system percentage that is calculated by summing all the products that were received by sites by all the products that were sent by the source.
Overall System Percentage is 99.16% for all Sites except NO STATS
2.0 Last 24 Hours Histogram based on Site performance
This chart displays overall site counts for the following categories for the last 24 hours using the "Hourly Statistics summary" report as the source of the data:
100% [99-100)% [95-99)% [80-95)% (0-80]% 0% LDM NO DOWN STATS
3.0 Last 24 Hours Percentage based on Site performance.
This chart displays overall system percentages for the last 24 hours using the "Hourly Statistics summary" report as the source of the data.
4.0 Last 24 Hours Histogram based on Relay's performance
This chart displays the same data as 2.0 except it is limited to the relay sites. Currently there are 10 relays.
5.0 Last 24 Hours Percentage based on Relay's performance
This chart displays the same data as 3.0 except it is limited to the relay sites. Currently there are 10 relays.
6.0 Daily Histogram based on System performance
This chart summarizes the data used in the chart "Last 24 Hours Histogram based on Site performance" into daily data. The name uses System performance because it states that on average this many sites are receiving a certain percentage of data. It does not state that a single site received a certain percentage of data for the 24 hour period.
7.0 Daily Percentage based on System performance
This chart summarizes the data used in the chart "Last 24 Hours Percentage based on Site performance" into daily data. The name uses System performance because it states the total number of products received over the total number of products sent for all the sites in the system.
8.0 Daily Histogram based on Site performance
This is the most sensitive chart because it uses the site as a base unit for all the feeds over a 24 hour period. If a site only receives 22 reports for a 24 hour period, 2 hours of zero are averaged into the percentages. Also, there must be 24 hours of reports for all the feeds a site is receiving or zeros are average in the total. Another sensitivity is the source sites must have sent 24 hours of reports, if not all the sites would be unable to receive 100% even if the site receives all the data that the source sent.
9.0 Daily Percentage based on Site performance
This chart displays the Site percentage on a daily average where missing reports are interpreted in the following manner: 1) Counted as 0%, 2) Ignored, 3) Counted as 100%
10.0 Cumulative and daily statistics summaries
The cumulative section of the report spans from 1 to 3 months of data and it displays the level of the site in the IDD routing hierarchy, the feeds, the percentage of of total products received from the source and the number of reports collected. For example:
Sink_1 drier.atmos.washington.edu DDS 99.58 2099
The daily section displays the level of the site in the IDD routing hierarchy, the feeds, the percentage of of total products received from the source, the number of reports collected, and the number of products received. For example:
Sink_1 drier.atmos.washington.edu DDS 97.88 23 32641
11.0 Last 24 Feed Histogram of Number of Products
This chart shows the Number of Probucts per Feed, the Feeds are stacked on top of each other for the hour. The drawback is some Feeds Numbers are relatively small to others, therefore a zoom chart was created to display smaller counts.
12.0 Zoom of Last 24 Feed Histogram of Number of Products
This charts show Feed Numbers less than 150.
13.0 Last 24 Feed Histogram of Number of Bytes
This chart shows the byte count for all the Feeds. The drawback is some Feeds byte count is relatively small to others, therefore a zoom chart was created to display smaller byte count.
14.0 Last 24 Feed Histogram of Number of Bytes
This chart show the Feed Bytes count less than 750,000 bytes.