[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20020312: IDD data late (cont.)

>From: "Kevin Polston" <address@hidden>
>Organization: NOAA
>Keywords: 200203112125.g2BLPSa16982 IDD

Kevin,

>Well, I changed my ldmd.conf file to take out requesting all the N0R
>radar data. I set it up to add the radar sites for all of KS, MO, IA,
>NE, and OK with AMA and FWS as well. So that's like 16 radars. Well, I
>apparently had a typo in the file because I wasn't getting any radar
>data (I think it was actually an extra space).  I corrected that and am
>getting those sites now.

OK, good.

>But the big news is that the satellite data
>seemed to very timely and running without a problem! So that was good.
>So that means it probably was a "bottleneck".

I did a traceroute from papagayo to your machine and saw that it went
through 20 hops before getting to you.  This may well explain why data
can be slow in getting to you when it is not at papagayo.

>However....I do have two
>small problems still. The last time I checked the surface obs were
>still running late (more than 30 minutes behind) and there were still a
>few fields missing from various models (although it was significantly
>improved......with most of the data there as opposed to not there).  So
>I am going to watch and see how it goes with these 16 or so radar sites
>coming in and see if there is any degradation or not.

Missing fields from model runs (if they are there in the first place)
would indicate that they are being lost somewhere up the line or that
your decoding action is somehow not sending those products to the decoder.

>Question.....I saw on the webpage about setting up LDM to acquire the
>radar data that you can request regions for the individual sites--ie,
>south central, north central, etc. (here is the link....
>http://www.unidata.ucar.edu/packages/ldm/feedtypes/howto-nnexrad.html).
>Can I do that? I wasn't sure from reading the webpage if I could or not.

Yes.  You can specify that you want all of the products from all of the
radars in a region by specifying the product header for that region
in your ldmd.conf request line.  The problem for most sites is that
the radars they want line in more than one region, or the region
contains more radars than they can handle.

There is more flexibility in requesting the list of radars you want
and the set of products from each radar that you want.

>Also, I don't think we were on the same page here. You had
>mentioned a couple or so e-mails ago that you guys (or Chiz) was
>working on creating a national and/or regional VIL product.

No, I am quite sure that I didn't since we are not working on this.  Here
is our interchange on this:

you said:

  >Is there a plan to make a national
  >and regional composite VIL image?  I know that would be pretty slick.

I said:

  Chiz and I have been talking about this.

Talking about the possibility of doing something is not the same as
working on it.

>I asked how
>soon before that would be available. You said it depends on what you
>mean by "how soon".

This interchange was:

you said:

  >So then the chances are good of seeing the VIL product very soon.  :-)

I said:

  Depends on what "very" means.

>Then I asked again in my last e-mail and you
>started referring to the national 1km reflectivity composite that you
>said you are already sending out. I got the info on that...but I was
>wondering what about the VIL composite.

Again, Chiz and I have talked about creating other composite products.
Neither of us are working on them currently.

>So, mostly good news today. I will let you know how things go. I
>haven't tried yet messing with the nsswitch file and so forth like what
>you mentioned yesterday.  Would that help at all?

Please reread that email. The concept I was trying to get across was
the creation of other aliases for your upstream feed site(s).  Creating
of the alias that will be used can be as simple as adding aliases for
the machine in your /etc/hosts file.  To be that simple however, your
/etc/nsswitch.conf file has to specify that /etc/hosts is searched
first when trying to trying to lookup host information.

IF /etc/nsswitch.conf is setup to search for machine names by the
/etc/hosts file and then DNS, then the /etc/nsswitch.conf entry will
look like:

hosts:      files dns

IF it is setup to use DNS before looking in /etc/hosts, the
/etc/nsswitch.conf entry will look like:

hosts:      dns files

You want yours to look like:

hosts:      files dns

for the following to work.  Edit /etc/hosts (as root) and add:

129.93.52.150   papagayo.unl.edu        papagayo2.unl.edu

Be careful when editing /etc/hosts:  white spaces in lines are tabs, not
spaces!

>I mean, if there was
>a "bottleneck" in the data flow because of the radar data wouldn't that
>happen again even if I had an alias and another rpc.ldmd running?

What can happen when there is one request for a lot of data from a
source site is that the single rpc.ldmd on your side and the single
rpc.ldmd on the upstream side have to handle movement of all of the
data.  This means that each product to be sent is acted on one-by-one
from a list of products by each rpc.ldmd.  When you split the feed by
requesting from either different hosts or different aliases for the
same host, you are causing multiple connections to be made, and the
data then moves in the different channels in parallel.  The effect of
this is to allow products in one request stream to not be held up to
those in another.  This is especially important when the products in
one stream are large (e.g., NIMAGE) and the other are small (e.g.,
DDPLUS|IDS) FYI, this is how web browsers work.

If the only factor in data receipt slowness was the physical network
connection, then this technique would provide no benefit.  The reality
is that network throughput is a complex combination of factors.

>If
>it will help I will take your word for it even though I don't fully
>understand why it would help.

Perhaps the short explanation above will help to explain the process.

Tom