[ldm-users] 20080402: Soliciting feedback on Next Generation LDM development

LDM-6 users,

Everyone here at the Unidata Program Center has been reading ** with
great interest ** the posts regarding our efforts aimed at developing a
Next Generation LDM.  We are _very_ pleased that several community
members have shared their thoughts on this development.  We sincerely
hope to hear from more of you on this important topic!

In order to provide some background on this development, we offer
the following short overview:

   Why develop a Next Generation LDM when the current one works so well?

   The LDM has grown to be a robust, reliable and portable base on which
   to build data distribution networks.  As design or implementation
   limitations were identified, innovative developments have been employed
   to keep the LDM viable.

   The current implementation of the LDM when coupled with cluster
   technology may be able to effectively relay all of the data desired
   by the expanding Unidata community for several more years.  However,
   input on the limitations in the current LDM design from you, the
   user community, has convinced us that new, alternate approaches to
   data distribution must be thoroughly investigated and pursued.

   The need for a more scalable technology is apparent from several

     - the volume of useful data appears to be growing faster than
       increases in network bandwidth

     - approaches to setting priorities for bandwidth by "shaping"
       its use, charging departments for bandwidth, or other institutional
       data policies may disrupt the model of essentially free bandwidth
       currently in use for the IDD

     - competition for network bandwidth from the explosive growth in
       video distribution may demand more dynamic routing than is now
       possible with current LDM technologies

   To avoid interruption in data delivery, future needs must be
   anticipated and reliable software must be developed well before the
   need is apparent.

   Our experience suggests the current LDM-6 may have advanced about as
   far as it can given the constraints of its architecture.

     Please be assured that the development of a Next Generation LDM
     does not imply that incremental improvements to the current LDM-6
     will cease!

   However, further major advances in the LDM may require a new
   protocol that is not tied to the client/server approach but instead
   uses peer-to-peer concepts.

   A new protocol and implementation could allow for the following

     - more dynamic creation and destruction of data-product streams

     - support for access to "one-time" data products (i.e.,
       data-products that are not continuously generated)

     - better load balancing of communication links

     - more adaptive and flexible dynamic routing of data-products with
       steady-state results that are relatively independent of the
       configuration of initial connections

     - a better interface for selecting data-products.  For example,
       reception of a data-product stream could be started by clicking
       on a hyperlink in a web page.

     - support for use on computing platforms beyond those being used
       in the community for data relay tasks (e.g., Windows)

   Naturally, minimizing disruption -- both to individual sites and
   to the flow of data -- is a major concern of any new implementation
   and deployment.  The LDM software is open source, so no one will be
   forced to upgrade to different and incompatible software.  In fact,
   there could be two competing distribution systems running in
   parallel for some time, during which the new system will have to
   prove itself capable of satisfying users of the current LDM as well
   as providing solutions to problems the current LDM cannot handle.

   Users interested in the history of the LDM/IDD and that would like
   to learn more about ongoing efforts at creating a Next Generation
   LDM should take a look at the (14 page) paper we presented at the
   AMS 2006 Annual Meeting:

   "The Unidata Internet Data Distribution (IDD) System: A Decade of
Again, we are seeking your input on this new, and exciting development!

In order to make it as easy as possible for you to share your thoughts,
we are providing several ways for you to comment:

- send email to the ldm-users@xxxxxxxxxxxxxxxx list

- post to the ldm-users forum:

  Unidata HomePage


     Participate in topical forums

       LDM Users Forum

- provide additions-to/comments-for the wiki that Steve Emmerson
  mentioned in his request for users stories/high-level usage


- send email to the Unidata User's Committee <usercomm@xxxxxxxxxxxxxxxx>

- send email to Unidata User Support <support@xxxxxxxxxxxxxxxx>

- contact any Unidata staff member directly

Please share your thoughts on the development of a Next Generation LDM
and what features you would like to see incorporated in its design!!


The Unidata Staff
* Unidata User Support                                  UCAR Unidata Program *
* (303) 497-8643                                               P.O. Box 3000 *
* support@xxxxxxxxxxxxxxxx                                 Boulder, CO 80307 *
* Unidata WWW Service                           http://www.unidata.ucar.edu/ *

  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: