IDD Status
Anne Wilson
May 13, 2002
This report updates the IDD Status report from the February, 2002, Unidata Policy Committee Meeting.
Participation
| Date |
Number of Institutions |
Number of hosts |
| October, 2001 |
131 |
244 |
| February, 2002 |
155 |
268 |
| May, 2002 |
165 |
292 |
The international reach of the IDD is growing as it is now serving sites in Australia and Brazil.
Data Feed Volumes
Feeds by popularity, of 142 sites reporting, are virtually unchanged
from the February report.
Since February, although the data volumes of most feeds have remained about the same, there have been significant increases on the
following feeds:
| Feed Type |
Feb MB/hour |
May MB/hour |
Comments |
| CONDUIT |
399.4 |
862.1 |
Due to the addition of 20km RUC output,
and the extension of the AVN forecast from 120 hours to 180 hours |
| HDS |
91.4 |
99.1 |
Additional products |
| NEXRAD |
79.0 |
139.1 |
Seasonal weather |
| FNEXRAD |
4.5 |
17.8 |
Due to addition of 1 km mosaics generated every 10 minutes |
| GEM |
0.0 |
.7 |
New feed (see below) |
| Total - all feeds |
597.6 |
1142.5 |
A cumulative increase of 91% |
Impact of Increased Data Volumes
One impact of the increase in data volumes is that
servers must increase their queue size in order to be able to store
the recommended minimum of one hour's worth of data.
A new problem, however, is that many
sites are reaching or have reached the maximum queue size of 2 gigabytes.
To remedy this situation on Unidata's own server (thelma) it was
necessary to rebuild the LDM with a 7 gigabyte queue. Building
such a large queue requires that a host have large file
support, that is, that it can support 64 bit addresses. Not all
machines have this capability. Yet, it may soon be necessary to
require at least top tier sites to have hardware that supports
such large queues.
Using a large queue incurs a performance
cost. In particular, upon establishment of a new connection, the time for a server to
find its place in the queue is greater.
New Data Feeds
The Canadian Meteorological Center's Global Environmental Multiscale
(GEM) model is now being relayed
on a point to point basis via the IDD. The volume of the feed is
relatively small:
8.3 megabytes, 430 products, twice per day.
Unidata is now receiving the entire CRAFT feed on an experimental
basis. Currently 38 radar sites are participating. Regarding data
volumes, one small sample showed 163 megabytes/hour, about
10,400 products per hour.
LDM 5.1.5
The new version of the LDM, 5.1.5, is still under development. It is
anticipated to be released in June.
Anne Wilson
Last modified: Mon May 13 16:22:44 MDT 2002
