Re: [ldm-users] user-stories for the LDM replacement

Steve Emmerson wrote:
Dear LDM user,

I'm working on a replacement for the LDM and I'd like your help. I need user-stories (i.e., high-level usage scenarios) so I don't forget anything or leave out some useful feature. I've created a wiki to help gather the stories (and for anything else for which it might be useful). The wiki is at <>. It contains the link "User Stories" in which I've put one story. Please read it and if you think something is missing, then please add your own story, edit any existing one, add comments, etc.

In order to add or edit, I'll have to create an account for you on the UCAR wiki and add you to the "LDM-Replacement" group. Just reply to this email with your full name and the email address you'll want to use as your identifier. Naturally, I reserve the right to be human (i.e., picky :-).

Now's your chance!

Lacking access to the wiki, and also having a very strong opinion that Wiki's are for documentation, and not active communication, I'm gonna start here.

One of the listed Design Goals is an automatic adaptation to topology. I strongly agree with this one. I think the static approach we're so comfortable with could benefit here from some change.

Automatic adaptation to bandwidth is a little fuzzier. If the idea is to adapt on the fly to varying bandwidth requirements, TCP does a fairly good job of that. It also handles congestion management pretty well. I'd not recommend hacking a UDP distribution mechanism to make UDP artificially ack packets, but IIRC, there's been some changes to TCP ack requirements over time for LDM to improve throughput. There might be some places where we can work on this. Having said this, another data distribution system I'm involved in moves a lot of data around its high-level servers using UDP and sees precious little packet loss: With strict accounting we see something approximating 0.2% but practical appreciation says we see no adverse impact from that. Using binary files instead of short files certainly helps this, though: For our use, 0.2% packet loss could be a major failure real fast.

I strongly support the ability to obtain/retrieve realtime _and_ archived data. I'm not so sure what static and dynamic directories are/mean but I think I've got a good idea... Ditto finite vs infinite files, and "local data holdings". I think we already do that with LDM.

I'm already using LDM and PQACT for event driven activities and they just work nicely. Am I doning something wrong? If DDDAS had discovered LDM 5 years ago, they'd be a bit further ahead.

Minimizing b/w and requiring a site to receive just wha tit needs are laudable goals and appropriate for leaf nodes, but further upstream unless we have a really adaptable system (not impossible but this is starting to sound like a major CompSci project) but non-trivial if you're also feeding downstream nodes. I can envision a couple of ways to approach this, however, that might work.

Stand-alone app AND libraries is good.  Very good.

Enhancing scalability is laudable but save with limitation some feedtypes and some file sizes, I think we're OK here. Scalability WRT the number of sites? I think there's likely an upper bound but I'm not positive we've reached it yet.

I don't do Windows. I can't stand the downtime. Server installs are (still) increasingly going to *nix. My particular preference is Linux but it's not burned in stone. I can survive with BSD (with or without Apple) or one of the commercial Unices, including Solaris, if I really have to. But Windows? Really? Maybe, just maybe for an end-user leaf node.

Cool names are always good.

And after a little more reading... I don't have a browser on my LDM machines. They're servers. Nor would I (like Tyler) support java in a high performance data handling environment. Java's got its place, but in my experience (and unlike Tyler, I do write C/C++/java code) it's not here.

So that's my initial impression.
Gerry Creager -- gerry.creager@xxxxxxxx
Texas Mesonet -- AATLT, Texas A&M University        
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843

  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: