[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[#RFZ-122547]: Re: Re: More data to send from CMA

Manuel,

> Note that Doug suggested that I comment out the REQUEST lines for CMA
> data from CMA's machine, so you could distinguish traffic. Please, let
> me know when I can uncomment those lines to start receiving data from CMA.

I think Doug suggested that ECMWF stop inserting CMA data into the
ECMWF product-queue.  He wrote

    Should we go ahead and stop the insertion of CMA data at ECMWF for now?
    Is it crucial to have this data in the archive as "test"?

As far as I'm concerned, ECMWF should request data from CMA.

> We were not only inserting ECMWF & CMA, but also MetOffice and JMA.
> These are the daily volumes by Data Provider that ECMWF was inserting in
> LDM:
> CMA:        27 GBytes
> ECMWF:     140 GBytes
> MetOffice:  22 GBytes
> JMA:         7 GBytes
> At the same time, ECMWF receives from NCAR on the same feed 8 GBytes of
> NCEP data. I don't know if 27 GBytes (CMA) could have such an adverse
> effect on the other 169 GBytes.

I know the LDM very well, but I don't know your data very well.  Do
the following product-identifier substrings correspond to the indicated
centers?

    CMA        "babj"
    ECMWF      "ecmwf"
    MetOffice  "egrr"
    JMA        "rjtd"

> Anyway, note that CMA should get all 169 GBytes inserted by ECMWF and
> the 8 GBytes inserted by NCAR.

Are you saying that CMA should receive 169 gigabytes per day from
ECMWF even though ECMWF is the source of only 140 gigabytes per day?
If true, then that seems like an odd configuration to me.  I would
have thought that CMA would request ECMWF-generated data from only
ECMWF, NCAR-generated data from only NCAR, etc..  Unless there is
good reason to do otherwise, requesting data from the source of 
the data and from another site that also receives the same data
doesn't make the system more robust: it's still vulnerable to an
outage at the data source and, given the nature of the Internet,
it's unlikely that data would sometimes only flow to one
downstream site and not the other.

The exception to this general rule is due to administrative
policies in the networks that comprise the Internet.  It is
possible for site C to be unable to contact site A even
though it can contact site B which can contact by site A.  This
is caused by network policies that require that certain
traffic be routed to certain gateways.  Have you, however,
seen this?  We have, but only rarely.

I would think that the natural topology for the TIGGE LDM
network would be a star network for each source of data:
each center that generates data would be at the hub of a 
metaphorical wheel and every center that wants the data would
be on the rim at the end of a spoke.  There would be a
separate star topology for every source of data.

This would require that each data source have sufficient
bandwidth to satisfy the aggregate flow from it to all its
data sinks.  Is this a problem?

Please tell me if my thinking is incorrect.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: RFZ-122547
Department: Support IDD TIGGE
Priority: Normal
Status: Open