[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[TIGGE #BGR-338705]: A problem when perform LDM tests

Hi Yangxin,

I apologize for taking so long to respond, but I have been out of the
office for the past few days.
> Regarding the memory mapping for LDM, now I understand that enough
> memroy should be reserved for the OS to operate efficiently, since
> LDM will ask the OS to allocate all the memory as LDM configured.
> Thank you for your clarifying explanation.

No worries.

> Last Friday, I went to CSTNET for the second time as they demand,
> to verify how some network devices restrict P2P data transfer thus
> further impact LDM to work properly with port 388.

Very good.

> My laptop computer was brought to connect directly to the outmost
> router of the CSTNET, therefore, the data transfer for my test will
> pass throuth all the network devices between CMA and CSTNET, I guess my 
> laptop is between the USA and the CSTNET.

If I read the first part of your sentence correctly, your laptop would
be between the CMA and CSTNET unless you mean the outermost router on
the downstream side of CSTNET.  If this reading is correct, it would mean
that your laptop was between CSTNET and CMA.  If you mean that the outermost
router is on the downstream side of CSTNET, then yes, your machine would
be between CSTNET and outside systems including those in the USA.

> Two tests have been performed. One has done with P2P restriction applied,
> the other case, no restriction for P2P. But the results for both tests are
> similar, all data reached downstream, data trasfer rate is OK, 20-30 Mbit/s,
> sometimes over 30Mbit/s. May I say that the network between that point and
> CMA is OK?

Yes, the tests you report seem to indicate that the connection between CMA and
downstreams past CSTNET (e.g., NCAR, ECMWF) should be OK.  If this is the
situation, it would indicate that something has changed since Mike Schmidt
proved that there was some sort of "packet shaping" on port 388 traffic by
switching the port to 8080 using 'balance'.

> So, maybe I should perform the next tests right at that point with your 
> server "yakov.unidata.ucar.edu <->". I will send babj products 
> and
> receive your products, ecmf, kwbc, etc.

I can setup yakov to request data from your machine.  Please resend the fully
qualified name of your machine and its IP address so I can setup this test.
For simplicity, the data I would send to you is that in the "operational"
IDD datastream named CONDUIT.  This datastream has about 3-5 GB/hour of data
in it, so it should be a good test.

> One of the problems is, when connecting to that router, I cannot have a real
> internet address as current production LDM Server at CMA, my address is
> something like "192.168.xxx.xxx".

Addresses like 192.168.xxx.xxx are internal addresses.  We will need to 
the actual external IP address of the machine (assigned by the router) in order
to be able to feed from your machine.

> Another one might be a problem or might not be a problem, that is, we are
? in a asynchronous manner to exchange data, I cannot have any idea about
> how your LDM respond and what is the transfer rate for your network in your
> test server when I will peroform the tests at CSTNET someday next week.

I don't know what "we are in a asynchronous manner to exchange data" means.

> Another possible way is a test that has nothing to with my tests for the
> second time at CSTNET. That is we use your LDM Server  the LDM Server
> "yakov.unidata.ucar.edu" and my LDM Server at CMA that I used for recent
> tests to do additional tests. This LDM Server here is right next to the
> real LDM Server. Of cours, I should ask CSTNET to stop the restriction on
> P2P for our IPs.

This sounds like a reasonable test, and yes, it would be a good idea to ask
the CSTNET people to stop restrictions on P2P for your IPs.

> CSTNET people also suggested that we might try the port 8080 in another
> way rather than use the "balance", but use LDM itself's ability of using
> different port. How do you think about this?

We could setup data transfer on a port other than 388, but success in a test
like this would only reinforce what we already know: there is some sort of
"packet shaping" being done on port 388 traffic.

> I'd like to konw your suggestions for the next steps.

I propose that we first setup a test of us feeding your machine the data in
the IDD CONDUIT datastream.  This will require:

- that I start ingesting CONDUIT on yakov.unidata.ucar.edu (easy)
- you setup the following feed requests on your machine:

request CONDUIT "[09]$"   yakov.unidata.ucar.edu
request CONDUIT "[18]$"   yakov.unidata.ucar.edu
request CONDUIT "[27]$"   yakov.unidata.ucar.edu
request CONDUIT "[36]$"   yakov.unidata.ucar.edu
request CONDUIT "[45]$"   yakov.unidata.ucar.edu

As soon as you add these request lines to your ~ldm/ldmd.conf file
and restart your LDM, I will be able to see the real name and
IP address of your machine and add the corresponding allow and
name <-> IP mapping in yakov's /etc/hosts file.  This will allow
your machine to feed from yakov as long as the IP address stays
the same.


Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
Unidata HomePage                       http://www.unidata.ucar.edu

Ticket Details
Ticket ID: BGR-338705
Department: Support IDD TIGGE
Priority: High
Status: Closed