Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
>
> > Since disk/storage is so cheap, having a large queue seems to make sense
> > these days. However, I would agree with Unidata's concerns that if there
> > was a larger scale outage such that many sites started trying to catch up
> > on many hours worth of data, this could present a networking load issue.
> > Hopefully this kind of situation wouldn't occur very often, though.
> >
>
> Maybe we can just restrict that to sites feeding others? Or if you're in
> paranoid mode, just top nodes (like UIUC, SSEC, etc)? In either case,
> a note to Daryl...thanks for the great public service. If I need it, I'll
> use it, but only in an emergency.
>
I would like to say a few words about the use of this archive. We
supported Daryl's effort to build the large queue and stress test the
archive. It is great that he is willing to provide this service to our
community.
If you are considering using the archive, please bear in mind that the
IDD exists to serve near real-time data to all participating sites.
This is its primary goal.
We're actually recommending that IDD sites that feed others *not* use
the archive, or use it in a very limited way. Our concern is that a
relay site will get so busy recovering and processing old data that it
won't propagate the real-time data in a timely manner. And, it could be
problematic if a relay site was to propagate old data, which is possible
depending on the configuration of the downstream site. For leaf nodes
this is not an issue. Relay nodes, however, have an obligation to their
downstream sites to ensure that they only serve near real-time data and
that other processing does not interfere with performing that role.
BAD MSG:
relatively small gaps in a site's data record.
Regarding the possibility of outages at the IDD top level, all of our
top level nodes already have a redundant feed. Thus, if one site loses
the input from their dish, they will still get data. It would take a
failure from both their dish and their redundant feed for them to lose
data. So far, this has not happened.
If you use this archive, please turn off pqbinstats when you point your
machine at the archive so that our statistics are not inappropriately
skewed. When returning to normal operation, please remember to turn
pqbinstats back on and remove the -o and -m flags to the rpc.ldmd call.
We appreciate your cooperation in this matter.
Anne
--
***************************************************
Anne Wilson UCAR Unidata Program
anne@xxxxxxxxxxxxxxxx P.O. Box 3000
Boulder, CO 80307
----------------------------------------------------
Unidata WWW server http://www.unidata.ucar.edu/
****************************************************
ldm-users archives: