I've been relatively quiet on these lists over the years but I thought I
would add my two cents to the discussion as well. I've been solely
responsible for our ldm here at LSC for at least 8 years.
If a replacement is just going to make improvements to the existing
"fetch it all" way of distributing data, please don't do it. Over the
past few years the ldm has been very reliable. I don't even think about
our ldm on a day to day basis any more. It was not like this many years
ago. I do not wish to go back to the days of fighting with our ldm to
stay up and keep a consistent data flow coming in.
I read over the wiki page a little this morning just to get a better
sense of the scope that we're talking about here.
I guess I'm thinking a bit beyond the ldm as it is now. Some background:
We purchased a NOAAport system several years ago because the routing
beyond our campus was so bad that we could not receive our IDD feeds
reliably. I would guess that we only use a very small portion of that
data feed on a regular basis. The problem is that we use many, many
other products on an irregular basis depending on weather events around
the country or world. This will also vary depending on the needs of our
courses as they progress through the semesters. Overall, we probably
use less than 20% of our data on a daily basis but likely our needs
would span to closer to 80% or 90% when evaluated on a product/category
view spread over a semester or calendar year. The end result is that we
end up ingesting a HUGE volume of data, processing it in multiple
different formats to satisfy the needs of various applications (ex:
gempak vs. mcidas), and then filing it on disk only to scour it out a
couple of days later when, most of the time, the majority was never used
at all.
I feel that the fetch on demand way of doing things with mcidas and the
IDV (ADDE) are the better approach. In their current state there is a
bit of room for improvement (IMO) but their approach to data access is
one that can scale better in the long run. With resolution increases in
imagery, radar and forecast grid products the bandwidth demands are
likely going to increase at a rate faster than our internet routes can
support if we continue with the "fetch it all" approach.
I've thought about the "ldm" becoming something minimal like a caching
proxy for ADDE requests that pulls in data when requested by our clients
regardless of the client. Not every site should need to run a full LDM
server just because they'd like to view data in one of the gempak
applications. (Of course this doesn't address many other issues such as
how we would handle backwards compatibility with things like the huge
investment of time we have made in gempak scripts.)
I would also like to see some kind of catalog or index that improves our
ability to find and access new products.
I'm sorry if I've rambled a bit around the topic with this.
Gilbert Sebenste wrote:
On Sat, 29 Mar 2008, Daryl Herzmann wrote:
I'll bite on this with my 2 cents!
I was sort of surprised by this announcement and very much concerned.
Without a doubt, LDM has been the most stable and most important piece of
middle ware software I use. Outside of the IDD, it is a key part of my
content generation and distribution system. It just works and it is
something I don't worry about failing, since it rarely/never does!
After a cursory look over the pages, it is not clear to me what is going
to happen with LDM. Is this 'replacement' going to be written in Java? Is
it going to maintain RPC compatibility with LDM? Aren't there other
projects doing some of the things mentioned, like Thredds Data Server(?),
etc?
Talking with other LDM users this weekend, they expressed similar
concerns. I would much rather see some of those features go into the LDM
codebase without such a drastic shift to different / unproven technology.
This new project looks very cool, but lets not replace LDM :)
+1 here. The current LDM works great, and I'm not sure reinventing the
wheel here is going to help. Unless it is going to work with older LDM's,
I'm quite content with how this generation of LDM works. And this from
someone who people call "Captain Upgrade" for being bleeding edge!
*******************************************************************************
Gilbert Sebenste ********
(My opinions only!) ******
Staff Meteorologist, Northern Illinois University ****
E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx ***
web: http://weather.admin.niu.edu **
*******************************************************************************
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
--
Mark Tucker
Meteorology Dept. Systems Administrator
Lyndon State College
http://apollo.lsc.vsc.edu
mark.tucker@xxxxxxxxxxxxxxx
(802)-626-6328
