[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20030503: 20030502: kickin my server



Ray,

Given that you are requesting a small amount of the total datastream,
you can consolidate many of your request lines. Also, you need to
be careful not to introduce uncecessary pattern checking. For example,
you are only interested in HDS products beginning with the
seqences beginning with "Y", whereas you patterns below will look for 
sequences ocurring anywhere within the WMO header. The added searching
will create overhead on your upstream. You can use the ~ldm/bin/regex program
(see LDM manpage) for validating regular expressions. For your ldmd.conf, 
you can use the following 2 (note, I wrapped the second request for
this email, but you don't have to!):

request NMC3|NNEXRAD|MCIDAS     ".*"    ldm.envsci.rutgers.edu

request WMO
        "^(W[FU]US|Y(PU.89|[TWEZ]..89|T..85) KWBE|YP..89 (KWBH|KWB.*/mAVN))"
        ldm.envsci.rutgers.edu

Note the use of "^" for specifying the line begiing with....
Also, WFUS and WUUS is combined to W[FU]US,
the KWBE entries are combined to the second grouping between the "|" delimiters,
and the MRF and AVN to the third grouping. 

Since the HDS data and DDPLUS data streams are subsets of the WMO alias,
you can use the single feedtype in that case.

If you experience slow connections, then the most likely problem would be
in grouping NNEXRAD with the large MCIDAS and FNEXRAD products. If this
occurs, then break out the NNEXRAD to its own request line.

Tom: Dana mentioned once before to me that he had several instances of long 
TIME_WAIT connections waiting to close. You can probably tune up your
time outs for closing the socket in those cases since otherwise, your
computer is more vulnerable to denial of service attacks (not related
to the LDM!!!) where someone could load your computer up with lots
of attempted connection failures.


Steve Chiswell
Unidata User Support





>From: "Ray Weber" <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200305021649.h42Gn87U002780

>I show only a few outbound, here is the ldmd.conf:
>Problem appears to be if I reboot, the old connections
>do not time out on Tom's end.
>
>The entries for Toms server;
>
>
>request         NMC3    ".*"    ldm.envsci.rutgers.edu
>request         NNEXRAD ".*"    ldm.envsci.rutgers.edu
>request         MCIDAS  ".*"    ldm.envsci.rutgers.edu
>request         HDS     "YP..89 KWBH (......)"  ldm.envsci.rutgers.edu
>request         HDS     "YP..89 KWB. (......) /mAVN"    ldm.envsci.rutgers.edu
>request HDS     "YPU.89 KWBE (......)"  ldm.envsci.rutgers.edu
>request HDS     "YT..89 KWBE (......)"  ldm.envsci.rutgers.edu
>request HDS     "YT..85 KWBE (......)"  ldm.envsci.rutgers.edu
>request HDS     "YW..89 KWBE (......)"  ldm.envsci.rutgers.edu
>request HDS     "YE..89 KWBE (......)"  ldm.envsci.rutgers.edu
>request HDS     "YZ..89 KWBE (......)"  ldm.envsci.rutgers.edu
>request HDS     "YP....KWB. (......) /mMRF"     ldm.envsci.rutgers.edu
>request DDPLUS  "WFUS.. .... (......)"  ldm.envsci.rutgers.edu
>request DDPLUS  "WUUS.. .... (......)"  ldm.envsci.rutgers.edu
>
>
>This started with the version 6 update since it opens a new connection for 
>each entry, and you don't have a reasonable timeout on old connections.
>I take just a few products from HDS, and thats why there are such selective
>entries. Don't need most of them for the summer, but you should correct
>the issue with connection timeouts. I show only 14 rpc.ldmd's running for
>his machine.
>
>Ray
>
>
>
>
>> -----Original Message-----
>> From: Unidata Support [mailto:address@hidden]
>> Sent: Friday, May 02, 2003 12:49 PM
>> To: Ray Weber; Tom Grzelak
>> Cc: address@hidden; address@hidden
>> Subject: 20030502: kickin my server
>> 
>> 
>> 
>> Ray,
>> 
>> If you would send a copy of your ldmd.conf file to address@hidden,
>> we may be able to help you with your configuration. If you have many request
>> entries, we will try to suggest a better consolidation.
>> 
>> Tom/Dana: It may help to send us a grep of your ldmd.log file with
>> "grep topo ldmd.conf | grep attbi.com"
>> so that we can see what feed connections are being established.
>> 
>> Steve Chiswell
>> 
>> 
>> 
>> 
>> 
>> 
>> >From: Tom Grzelak <address@hidden>
>> >Organization: UCAR/Unidata
>> >Keywords: 200305021427.h42ERs7U019833
>> 
>> >Ray,
>> >
>> >I have 35 connections right now from your machine.  I am not exactly sure 
>> >how you have your LDM set up, but this is getting to be a problem.  Each 
>> >time your ldm makes a separate connection on my machine, it spawns another 
>> >process.  I can't have all of these connections on my machine.  It is 
>> >killin' my load rates.  Every other group I feed data to only has 1-3 
>> >connections at any given time.  I know you make separate connections for 
>> >very specific products, but each time you do so, my server must run another
>  
>> >LDM process to support you.  It may help your bandwidth, but at my expense.
>> >
>> >Tom
>> >
>> >dana@web:/users/dana% netstat -a | grep attbi |grep EST
>> >web.ldm              weather.ne.client2.attbi.com.46772  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.48881  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.38058  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.57891  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.57901  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32820  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32834  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32836  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32838  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32839  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.816  5840      0 32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.817  5840      0 32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.822  5840      0 32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.831  5840      0 32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.839  5840      0 32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32946  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32948  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32949  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32956  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32802 62928      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32803  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32795  5840      0 
>> >32832
> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32797 63940      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32798  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32799  5840      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.32801 62928      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.37735 62928      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.45275 62928      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.45395 62928      0 
>> >32832
>> >0 ESTABLISHED
>> >web.ldm              weather.ne.client2.attbi.com.45412 62928      0 
>> >32832
>> >0 ESTABLISHED
>> >
>> >
>> >
>> >
>> >===============================================================
>> >Tom Grzelak                          address@hidden
>> >Senior Systems Administrator
>> >  ----> Center for Environmental Prediction
>> >  ----> Department of Environmental Sciences
>> >
>> >Rutgers University
>> >14 College Farm Road                    Voice: 732-932-4923
>> >New Brunswick, NJ 08901-8551            Fax: 732-932-8644
>> >                 http://www.envsci.rutgers.edu
>> >=============================================================== 
>> >
>> 
>> ****************************************************************************
>> Unidata User Support                                    UCAR Unidata Program
>> (303)497-8643                                                  P.O. Box 3000
>> address@hidden                                   Boulder, CO 80307
>> ----------------------------------------------------------------------------
>> Unidata WWW Service              http://my.unidata.ucar.edu/content/support 
>> ****************************************************************************
>> 
>> 
>
>