Re: [conduit] How's your GFS?

Carissa, 

> From: "Carissa Klemmer - NOAA Federal" <carissa.l.klemmer@xxxxxxxx>
> To: "Gilbert Sebenste" <gilbert@xxxxxxx>
> Cc: "Arthur A Person" <aap1@xxxxxxx>, "Bentley, Alicia M"
> <ambentley@xxxxxxxxxx>, "Mike Dross" <mdross@xxxxxxxxxxxxxxxxxx>, "Michael
> Schmidt" <mschmidt@xxxxxxxx>, "support-conduit@xxxxxxxxxxxxxxxx"
> <conduit@xxxxxxxxxxxxxxxx>, "_NCEP.List.pmb-dataflow"
> <ncep.list.pmb-dataflow@xxxxxxxx>, "Daes Support" <daessupport@xxxxxxxxxx>
> Sent: Wednesday, January 6, 2016 7:33:13 AM
> Subject: Re: [conduit] How's your GFS?

> Hi Art,

> Mike actually doesn't work with us anymore. What I can tell you is that we 
> have
> begun to formulate a plan to upgrade the way we serve data on our conduit
> servers. The plan will better balance the LDMs and we suspect should help with
> delays.

Excellent! I think we can make-do with what's there now until the re-config 
takes place. 
Thanks for all your efforts with conduit! 

Art 

> I don't have a great timeline for this upgrade yet, but we are talking months
> for sure. The best thing we can do is when you start experiencing problems let
> the team here know (dataflow email cc'ed) and we will look into it in real
> time.

> Carissa Klemmer
> NCEP Central Operations
> Dataflow Team Lead
> 301-683-3835

> On Tue, Jan 5, 2016 at 4:03 PM, Gilbert Sebenste < gilbert@xxxxxxx > wrote:

>> I know that LDM 6.12.15, now in beta, is supposed to make the feed * 
>> potentially
>> * better (faster) for large volumes of large files. It hasn’t been released
>> yet.

>> Gilbert

>> From: conduit-bounces@xxxxxxxxxxxxxxxx [mailto: 
>> conduit-bounces@xxxxxxxxxxxxxxxx
>> ] On Behalf Of Arthur A Person
>> Sent: Tuesday, January 05, 2016 2:45 PM
>> To: Michael Shedlock < michael.shedlock@xxxxxxxx >
>> Cc: Bentley, Alicia M < ambentley@xxxxxxxxxx >; Mike Dross <
>> mdross@xxxxxxxxxxxxxxxxxx >; Michael Schmidt < mschmidt@xxxxxxxx >;
>> support-conduit@xxxxxxxxxxxxxxxx < conduit@xxxxxxxxxxxxxxxx >;
>> _NCEP.List.pmb-dataflow < ncep.list.pmb-dataflow@xxxxxxxx >; Daes Support <
>> daessupport@xxxxxxxxxx >
>> Subject: Re: [conduit] [Ncep.list.pmb-dataflow] How's your GFS?

>> Mike,

>> Have there been any more discoveries regarding the conduit GFS 0.25 degree 
>> data
>> latency

>> issues? Overall, my impression is that latencies seem to be better than 
>> before
>> but not entirely

>> gone. For some reason, any latencies seem to be magnified when reaching Penn
>> State... I will

>> investigate this locally again to see if we can find a contributing cause.

>> Thanks... Art

>>> From: "Arthur A Person" < aap1@xxxxxxx >
>>> To: "Michael Shedlock" < michael.shedlock@xxxxxxxx >
>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Mike Dross" <
>>> mdross@xxxxxxxxxxxxxxxxxx >, "Michael Schmidt" < mschmidt@xxxxxxxx >, "
>>> support-conduit@xxxxxxxxxxxxxxxx " < conduit@xxxxxxxxxxxxxxxx >,
>>> "_NCEP.List.pmb-dataflow" < ncep.list.pmb-dataflow@xxxxxxxx >, "Daes 
>>> Support" <
>>> daessupport@xxxxxxxxxx >
>>> Sent: Monday, November 16, 2015 2:39:11 PM
>>> Subject: Re: [conduit] [Ncep.list.pmb-dataflow] How's your GFS?
>>> Mike,

>>>> From: "Michael Shedlock" < michael.shedlock@xxxxxxxx >
>>>> To: "Tyle, Kevin R" < ktyle@xxxxxxxxxx >, "Mike Dross" <
>>>> mdross@xxxxxxxxxxxxxxxxxx >, "Arthur A Person" < aap1@xxxxxxx >
>>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, poker@xxxxxxxxxxx , 
>>>> "Michael
>>>> Schmidt" < mschmidt@xxxxxxxx >, " support-conduit@xxxxxxxxxxxxxxxx " <
>>>> conduit@xxxxxxxxxxxxxxxx >, "_NCEP.List.pmb-dataflow" <
>>>> ncep.list.pmb-dataflow@xxxxxxxx >, "Daes Support" < daessupport@xxxxxxxxxx 
>>>> >
>>>> Sent: Monday, November 16, 2015 10:34:51 AM
>>>> Subject: Re: [conduit] [Ncep.list.pmb-dataflow] How's your GFS?
>>>> Kevin,

>>>> Thanks for the info. It looks like latencies from our conduit machines 
>>>> went up
>>>> early this morning ~11:00 UTC, but this time for all clients. And that's 
>>>> right,
>>>> this one was definitely an NCEP problem. NCEP had production issues this
>>>> morning, impacting the arrival of data onto our conduit machines, and 
>>>> affecting
>>>> the latency.

>>>> I'm still looking into all of this, though. And for everyone's record, the
>>>> latency for gfs.t12z.pgrb2.0p25.f096 over the weekend looked pretty good, 
>>>> all
>>>> way under a minute:
>>>>> 11/14
>>>>> Wisconsin: 0 seconds
>>>>> Unidata/UCAR: 1 second
>>>>> UIUC: 15 seconds
>>>>> PSU: 6 seconds

>>>>> 11/15:
>>>>> Wisconsin: 8 seconds
>>>>> Unidata/UCAR: 4 seconds
>>>>> UIUC: 22 seconds
>>>>> PSU: 2 seconds
>>> Are these latencies an average? On the graphs (
>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.meteo.psu.edu
>>> ) I see spikes

>>> to approx. 300-400 seconds over the past two days.

>>> Also, what's the difference between "conduit1", "conduit2" and "conduit3"? 
>>> There
>>> appears to be performance differences between them based on the

>>> Unidata graphic (e.g.
>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+flood.atmos.uiuc.edu
>>> ) with "conduit2" being the worst.

>>> Thanks... Art

>>>> Thank you to Art for sending the traceroute. Does anyone recall when this
>>>> latency problem started, or got worse, and how often it seems to happen?

>>>> Mike

>>>> On 11/16/2015 08:20 AM, Tyle, Kevin R wrote:

>>>>> More latency noted overnight, courtesy of Kyle Griffin @ UWisc-Madison:

>>>>> -----------------------------------------------------------------------------------

>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+flood.atmos.uiuc.edu

>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+cascade.atmos.albany.edu

>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.meteo.psu.edu

>>>>> And noted nicely downstream. This smells like an NCEP problem, as UWisc 
>>>>> and UIUC
>>>>> have the same (~1800 second) latency and others are >2000 (PSU, Albany).

>>>>> Comparing Albany and UWisc, the GFS files that are short (some are more 
>>>>> than 30%
>>>>> missing, one as much as 70% missing) are the same, but the file sizes are 
>>>>> not
>>>>> the same, implying downstream servers were getting some slightly 
>>>>> different sets
>>>>> of data from their upstream options.

>>>>> Just wanted to send this out in case either of you had a couple minutes 
>>>>> in your
>>>>> busy Monday mornings to check this out...might be getting to be an 
>>>>> annoying
>>>>> problem to try and chase...

>>>>> Kyle

>>>>> ----------------------------------------

>>>>> Kyle S. Griffin

>>>>> Department of Atmospheric and Oceanic Sciences

>>>>> University of Wisconsin - Madison

>>>>> Room 1407

>>>>> 1225 W Dayton St, Madison, WI 53706

>>>>> Email: ksgriffin2@xxxxxxxx

>>>>> _____________________________________________
>>>>> Kevin Tyle, Systems Administrator
>>>>> Dept. of Atmospheric & Environmental Sciences
>>>>> University at Albany
>>>>> Earth Science 235, 1400 Washington Avenue
>>>>> Albany, NY 12222
>>>>> Email: ktyle@xxxxxxxxxx
>>>>> Phone: 518-442-4578
>>>>> _____________________________________________

>>>>> From: conduit-bounces@xxxxxxxxxxxxxxxx <conduit-bounces@xxxxxxxxxxxxxxxx> 
>>>>> on
>>>>> behalf of Michael Shedlock <michael.shedlock@xxxxxxxx>
>>>>> Sent: Friday, November 13, 2015 2:53 PM
>>>>> To: Mike Dross; Arthur A Person
>>>>> Cc: Bentley, Alicia M; _NCEP.List.pmb-dataflow; Michael Schmidt;
>>>>> support-conduit@xxxxxxxxxxxxxxxx ; Daes Support
>>>>> Subject: Re: [conduit] [Ncep.list.pmb-dataflow] How's your GFS?

>>>>> All,

>>>>> NCEP is indeed on internet2, which I presume would apply here.

>>>>> A couple of noteworthy things.... I see some latency, but not for 
>>>>> everyone, and
>>>>> it doesn't seem to matter which conduit machine a client is connected to. 
>>>>> For
>>>>> example, with today's and yesterday's gfs.t12z.pgrb2.0p25.f096 (hour 96) 
>>>>> file
>>>>> here are the latencies per client that I see:
>>>>>> 11/12
>>>>>> Wisconsin: A few seconds
>>>>>> Unidata/UCAR: A few seconds
>>>>>> UIUC: 13 minutes
>>>>>> PSU: 27 minutes

>>>>>> 11/13:
>>>>>> Wisconsin: A few seconds
>>>>>> Unidata/UCAR: A few seconds
>>>>>> UIUC: 2.33 minutes
>>>>>> PSU: 2.75 minutes
>>>>> Another correlation is that UIUC and PSU (the ones with latency) are only 
>>>>> using
>>>>> one thread to connect to our conduit, whereas Wisc. and Unidata use 
>>>>> multiple
>>>>> threads.

>>>>> At the moment this sort of has the appearance of a bottleneck outside of 
>>>>> NCEP.
>>>>> It might also be useful to see traceroutes from UIUC and PSU to NCEP's 
>>>>> CONDUIT.
>>>>> I know I saw some traceroutes below. Can you try that and share with us?

>>>>> Mike Shedlock
>>>>> NCEP Central Operations
>>>>> Dataflow Team
>>>>> 301.683.3834

>>>>> On 11/13/2015 11:42 AM, Mike Dross wrote:

>>>>>> My $ 0.02 from having works with LDM since the mid 90's.

>>>>>> I assume NCEP is not on internet2? If so bandwidth shouldn't be an issue.
>>>>>> Regardless I would check the traceroutes to ensure a good path, high 
>>>>>> bandwidth,
>>>>>> low latency. Basic network topology check. I am sure you have done this.

>>>>>> An iperf test is a simple way to test the maximum throughput to see if 
>>>>>> bandwidth
>>>>>> is an issue. If that's not it, high latency or the way LDM is set up on 
>>>>>> the
>>>>>> upstream side is likely the culprit.

>>>>>> Mike

>>>>>> Sent from my iPad

>>>>>> On Nov 13, 2015, at 10:05 AM, Arthur A Person < aap1@xxxxxxx > wrote:

>>>>>>> Carissa,

>>>>>>> Yes, still issues. There was a period several weeks ago when throughput 
>>>>>>> was
>>>>>>> clean, but recently we've seen delays to varying degrees.

>>>>>>> Based on the Unidata latency chart from our reported statistics (
>>>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+iddrs2a.meteo.psu.edu
>>>>>>> ),

>>>>>>> we've seen delays during 0.25 degree gfs transmission that range from 
>>>>>>> 500
>>>>>>> seconds to 3500 seconds over the past couple of days.

>>>>>>> Also, comparison with charts from other schools seem to show better 
>>>>>>> reception
>>>>>>> when feeding from "conduit1" rather than "conduit2".

>>>>>>> Does this mean anything to you or is it purely coincidence or 
>>>>>>> incidental?

>>>>>>> Thanks for any insights you can provide.

>>>>>>> Art

>>>>>>>> From: "Carissa Klemmer - NOAA Federal" < carissa.l.klemmer@xxxxxxxx >
>>>>>>>> To: "Arthur A Person" < aap1@xxxxxxx >, "_NCEP.List.pmb-dataflow" <
>>>>>>>> ncep.list.pmb-dataflow@xxxxxxxx >
>>>>>>>> Cc: " support-conduit@xxxxxxxxxxxxxxxx " < conduit@xxxxxxxxxxxxxxxx >, 
>>>>>>>> "Pete
>>>>>>>> Pokrandt" < poker@xxxxxxxxxxxx >, "Michael Schmidt" < 
>>>>>>>> mschmidt@xxxxxxxx >,
>>>>>>>> "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Daes Support" <
>>>>>>>> daessupport@xxxxxxxxxx >
>>>>>>>> Sent: Friday, November 13, 2015 9:26:28 AM
>>>>>>>> Subject: Re: [conduit] How's your GFS?
>>>>>>>> Art,
>>>>>>>> I am going to add our team to this thread. Are you still seeing 
>>>>>>>> issues. Is so we
>>>>>>>> will take a look and see if we can tell if anything on our side is 
>>>>>>>> happening
>>>>>>>> around FH 96.

>>>>>>>> Carissa Klemmer
>>>>>>>> NCEP Central Operations
>>>>>>>> Dataflow Team Lead
>>>>>>>> 301-683-3835

>>>>>>>> On Thu, Nov 5, 2015 at 4:23 PM, Arthur A Person < aap1@xxxxxxx > wrote:
>>>>>>>>> Hi all...

>>>>>>>>> Conduit latencies have crept upward again for the past few weeks... 
>>>>>>>>> not
>>>>>>>>> unbearable, but still significant.

>>>>>>>>> At first it seemed to only affect us, but it looks like it's 
>>>>>>>>> affecting UIUC now
>>>>>>>>> also, but not so much Wisconsin.

>>>>>>>>> Inspecting our logs, we've noticed that there's no delay out to about 
>>>>>>>>> 90 hours
>>>>>>>>> of gfs transmission, but

>>>>>>>>> starting at 96 hours, the delays start to ramp up steadily. I'm not 
>>>>>>>>> sure how to
>>>>>>>>> explain that unless something

>>>>>>>>> else starts transmitting during that time that competes for 
>>>>>>>>> bandwidth. Also, I
>>>>>>>>> notice that sites receiving data

>>>>>>>>> from "conduit1" seem to be faring better than "conduit2". Is there any
>>>>>>>>> difference between these two

>>>>>>>>> originating systems or is that just coincidental? Anyone have 
>>>>>>>>> anything new to
>>>>>>>>> report on this issue?

>>>>>>>>> Thanks... Art

>>>>>>>>>> From: "Pete Pokrandt" < poker@xxxxxxxxxxxx >
>>>>>>>>>> To: "Carissa Klemmer - NOAA Federal" < carissa.l.klemmer@xxxxxxxx >,
>>>>>>>>>> mschmidt@xxxxxxxx
>>>>>>>>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Daes Support" <
>>>>>>>>>> daessupport@xxxxxxxxxx >, " support-conduit@xxxxxxxxxxxxxxxx " <
>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx >
>>>>>>>>>> Sent: Thursday, September 24, 2015 1:29:59 PM

>>>>>>>>>> Subject: Re: [conduit] How's your GFS?

>>>>>>>>>> Here are traceroutes from idd.aos.wisc.edu to conduit.ncep.noaa.gov 
>>>>>>>>>> and
>>>>>>>>>> ncepldm4.woc.noaa.gov taken at 17:17 UTC, right in the middle of the 
>>>>>>>>>> 18 UTC GFS
>>>>>>>>>> lag spike today.

>>>>>>>>>> [ldm@idd ~/etc]$ traceroute conduit.ncep.noaa.gov
>>>>>>>>>> traceroute to conduit.ncep.noaa.gov (140.90.101.42), 30 hops max, 60 
>>>>>>>>>> byte
>>>>>>>>>> packets
>>>>>>>>>> 1 r-cssc-b280c-1-core-vlan-510-primary.net.wisc.edu (144.92.130.3) 
>>>>>>>>>> 0.833 ms
>>>>>>>>>> 0.819 ms 0.855 ms
>>>>>>>>>> 2 internet2-ord-600w-100G.net.wisc.edu (144.92.254.229) 18.077 ms 
>>>>>>>>>> 18.095 ms
>>>>>>>>>> 18.067 ms
>>>>>>>>>> 3 et-7-0-0.115.rtr.wash.net.internet2.edu (198.71.45.57) 35.125 ms 
>>>>>>>>>> 35.278 ms
>>>>>>>>>> 35.261 ms
>>>>>>>>>> 4 198.71.45.228 (198.71.45.228) 35.378 ms 35.368 ms 35.335 ms
>>>>>>>>>> 5 ae0.clpk-core.maxgigapop.net (206.196.178.81) 36.401 ms 36.408 ms 
>>>>>>>>>> 36.284 ms
>>>>>>>>>> 6 noaa-rtr.maxgigapop.net (206.196.177.118) 36.523 ms 36.640 ms 
>>>>>>>>>> 36.411 ms
>>>>>>>>>> 7 140.90.111.36 (140.90.111.36) 68.769 ms 52.236 ms 52.210 ms
>>>>>>>>>> 8 140.90.76.69 (140.90.76.69) 36.602 ms 36.503 ms 36.827 ms
>>>>>>>>>> 9 * * *
>>>>>>>>>> 10 * * *
>>>>>>>>>> ...

>>>>>>>>>> [ldm@idd ~/etc]$ traceroute ncepldm4.woc.noaa.gov
>>>>>>>>>> traceroute to ncepldm4.woc.noaa.gov (140.172.17.205), 30 hops max, 
>>>>>>>>>> 60 byte
>>>>>>>>>> packets
>>>>>>>>>> 1 r-cssc-b280c-1-core-vlan-510-primary.net.wisc.edu (144.92.130.3) 
>>>>>>>>>> 0.838 ms
>>>>>>>>>> 0.847 ms 0.822 ms
>>>>>>>>>> 2 internet2-ord-600w-100G.net.wisc.edu (144.92.254.229) 18.513 ms 
>>>>>>>>>> 18.506 ms
>>>>>>>>>> 18.484 ms
>>>>>>>>>> 3 ae0.3454.core-l3.frgp.net (192.43.217.223) 40.245 ms 40.204 ms 
>>>>>>>>>> 40.123 ms
>>>>>>>>>> 4 noaa-i2.frgp.net (128.117.243.11) 43.617 ms 43.544 ms 43.699 ms
>>>>>>>>>> 5 2001-mlx8-eth-1-2.boulder.noaa.gov (140.172.2.18) 40.960 ms 40.951 
>>>>>>>>>> ms 41.058
>>>>>>>>>> ms
>>>>>>>>>> 6 mdf-rtr-6.boulder.noaa.gov (140.172.6.251) 46.516 ms 40.962 ms 
>>>>>>>>>> 40.876 ms
>>>>>>>>>> 7 * * *
>>>>>>>>>> 8 * * *
>>>>>>>>>> ...

>>>>>>>>>> --
>>>>>>>>>> Pete Pokrandt - Systems Programmer
>>>>>>>>>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>>>>>>>>>> 608-262-3086 - poker@xxxxxxxxxxxx

>>>>>>>>>> From: conduit-bounces@xxxxxxxxxxxxxxxx < 
>>>>>>>>>> conduit-bounces@xxxxxxxxxxxxxxxx > on
>>>>>>>>>> behalf of Carissa Klemmer - NOAA Federal < 
>>>>>>>>>> carissa.l.klemmer@xxxxxxxx >
>>>>>>>>>> Sent: Thursday, September 24, 2015 10:36 AM
>>>>>>>>>> To: mschmidt@xxxxxxxx
>>>>>>>>>> Cc: Bentley, Alicia M; Daes Support; support-conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>> Subject: Re: [conduit] How's your GFS?

>>>>>>>>>> Mike,

>>>>>>>>>> Can you provide what server you are coming from. I know your range, 
>>>>>>>>>> but I need
>>>>>>>>>> to provide to the helpdesk what is primary right now so they can 
>>>>>>>>>> trace back.

>>>>>>>>>> Carissa Klemmer
>>>>>>>>>> NCEP Central Operations
>>>>>>>>>> Dataflow Team Lead
>>>>>>>>>> 301-683-3835

>>>>>>>>>> On Thu, Sep 24, 2015 at 9:57 AM, Mike Schmidt < mschmidt@xxxxxxxx > 
>>>>>>>>>> wrote:
>>>>>>>>>>> Hi Carissa,

>>>>>>>>>>> We've seen the same jump in latencies;

>>>>>>>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+daffy.unidata.ucar.edu

>>>>>>>>>>> Here's our traceroute;

>>>>>>>>>>> # traceroute conduit.ncep.noaa.gov
>>>>>>>>>>> traceroute to conduit.ncep.noaa.gov (140.90.101.42), 30 hops max, 
>>>>>>>>>>> 60 byte
>>>>>>>>>>> packets
>>>>>>>>>>> 1 flra-n156.unidata.ucar.edu (128.117.156.253) 0.352 ms 0.344 ms 
>>>>>>>>>>> 0.325 ms
>>>>>>>>>>> 2 tcom-gs-1-n243-80.ucar.edu (128.117.243.85) 0.558 ms 0.584 ms 
>>>>>>>>>>> 0.662 ms
>>>>>>>>>>> 3 xe-0-1-2.873.core-l3.frgp.net (128.117.243.9) 1.138 ms 1.126 ms 
>>>>>>>>>>> 1.107 ms
>>>>>>>>>>> 4 v3454.rtr-chic.frgp.net (192.43.217.222) 23.227 ms 23.296 ms 
>>>>>>>>>>> 23.278 ms
>>>>>>>>>>> 5 et-7-0-0.115.rtr.wash.net.internet2.edu (198.71.45.57) 40.421 ms 
>>>>>>>>>>> 40.408 ms
>>>>>>>>>>> 40.340 ms
>>>>>>>>>>> 6 198.71.45.228 (198.71.45.228) 40.488 ms 40.649 ms 40.624 ms
>>>>>>>>>>> 7 ae0.clpk-core.maxgigapop.net (206.196.178.81) 41.545 ms 41.602 ms 
>>>>>>>>>>> 41.170 ms
>>>>>>>>>>> 8 noaa-rtr.maxgigapop.net (206.196.177.118) 41.796 ms 41.507 ms 
>>>>>>>>>>> 41.592 ms
>>>>>>>>>>> 9 140.90.111.36 (140.90.111.36) 41.419 ms 41.496 ms 41.623 ms
>>>>>>>>>>> 10 140.90.76.69 (140.90.76.69) 41.900 ms 41.728 ms 41.956 ms

>>>>>>>>>>> mike

>>>>>>>>>>> On Thu, Sep 24, 2015 at 7:49 AM, Carissa Klemmer - NOAA Federal <
>>>>>>>>>>> carissa.l.klemmer@xxxxxxxx > wrote:
>>>>>>>>>>>> Hi all,

>>>>>>>>>>>> I have opened a ticket with our helpdesk and included PSU 
>>>>>>>>>>>> traceroute. But can I
>>>>>>>>>>>> get a better handle on all the paths that are seeing latencies to
>>>>>>>>>>>> conduit.ncep.noaa.gov ? Is both PSU and WISC seeing spikes? Can I 
>>>>>>>>>>>> get a WISC
>>>>>>>>>>>> traceroute also please?

>>>>>>>>>>>> Thanks,

>>>>>>>>>>>> Carissa Klemmer
>>>>>>>>>>>> NCEP Central Operations
>>>>>>>>>>>> Dataflow Team Lead
>>>>>>>>>>>> 301-683-3835

>>>>>>>>>>>> On Thu, Sep 24, 2015 at 8:19 AM, Arthur A Person < aap1@xxxxxxx > 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Pete,

>>>>>>>>>>>>> I was thinking that too! If I only hadn't sent that email... :)

>>>>>>>>>>>>> Anyway, the delays aren't as bad as they were (at least here), 
>>>>>>>>>>>>> but are still
>>>>>>>>>>>>> indicative of a

>>>>>>>>>>>>> lurking problem. Almost seems as though some packet shaping is 
>>>>>>>>>>>>> going on, as Tom

>>>>>>>>>>>>> suggested previously. Maybe paths get overloaded and something 
>>>>>>>>>>>>> kicks in and
>>>>>>>>>>>>> meters-out

>>>>>>>>>>>>> usage??? Just speculating. I've asked our network folks here to 
>>>>>>>>>>>>> see if they can
>>>>>>>>>>>>> investigate

>>>>>>>>>>>>> our path to NCEP, but that may take awhile. Our traceroute from 
>>>>>>>>>>>>> this morning at
>>>>>>>>>>>>> 1113Z is:

>>>>>>>>>>>>> [ldm@iddrs1a ~]$ traceroute conduit.ncep.noaa.gov
>>>>>>>>>>>>> traceroute to conduit.ncep.noaa.gov (140.90.101.42), 30 hops max, 
>>>>>>>>>>>>> 60 byte
>>>>>>>>>>>>> packets
>>>>>>>>>>>>> 1 172.29.0.66 (172.29.0.66) 0.882 ms 192.5.158.1 (192.5.158.1) 
>>>>>>>>>>>>> 0.278 ms 0.264 ms
>>>>>>>>>>>>> 2 Blue1-ethernet3-1.gw.psu.edu (172.30.5.178) 0.220 ms
>>>>>>>>>>>>> White1-ethernet3-1.gw.psu.edu (172.30.5.177) 0.530 ms 0.526 ms
>>>>>>>>>>>>> 3 Windstream1-ethernet2-1.gw.psu.edu (172.30.5.106) 0.385 ms
>>>>>>>>>>>>> Telecom5-ethernet2-2.gw.psu.edu (172.30.5.102) 0.370 ms
>>>>>>>>>>>>> Windstream1-ethernet3-2.gw.psu.edu (172.30.5.114) 0.391 ms
>>>>>>>>>>>>> 4 Telecom5-ethernet2-1.gw.psu.edu (172.30.8.10) 0.391 ms 0.408 ms
>>>>>>>>>>>>> et-8-0-0.2364.rtr.chic.net.internet2.edu (64.57.30.2) 15.149 ms
>>>>>>>>>>>>> 5 et-7-0-0.115.rtr.wash.net.internet2.edu (198.71.45.57) 32.276 ms
>>>>>>>>>>>>> et-8-0-0.2364.rtr.chic.net.internet2.edu (64.57.30.2) 15.301 ms
>>>>>>>>>>>>> et-7-0-0.115.rtr.wash.net.internet2.edu (198.71.45.57) 32.594 ms
>>>>>>>>>>>>> 6 198.71.45.228 (198.71.45.228) 32.423 ms
>>>>>>>>>>>>> et-7-0-0.115.rtr.wash.net.internet2.edu (198.71.45.57) 32.431 ms 
>>>>>>>>>>>>> 198.71.45.228
>>>>>>>>>>>>> (198.71.45.228) 32.843 ms
>>>>>>>>>>>>> 7 198.71.45.228 (198.71.45.228) 32.853 ms 
>>>>>>>>>>>>> ae0.clpk-core.maxgigapop.net
>>>>>>>>>>>>> (206.196.178.81) 33.407 ms 33.401 ms
>>>>>>>>>>>>> 8 ae0.clpk-core.maxgigapop.net (206.196.178.81) 33.858 ms
>>>>>>>>>>>>> noaa-rtr.maxgigapop.net (206.196.177.118) 33.483 ms
>>>>>>>>>>>>> ae0.clpk-core.maxgigapop.net (206.196.178.81) 33.515 ms
>>>>>>>>>>>>> 9 140.90.111.36 (140.90.111.36) 33.574 ms 33.545 ms 
>>>>>>>>>>>>> noaa-rtr.maxgigapop.net
>>>>>>>>>>>>> (206.196.177.118) 33.907 ms
>>>>>>>>>>>>> 10 140.90.76.69 (140.90.76.69) 34.220 ms 34.012 ms 33.901 ms

>>>>>>>>>>>>> The above was taken while we were running about 1000 seconds 
>>>>>>>>>>>>> behind. A recent
>>>>>>>>>>>>> change here

>>>>>>>>>>>>> (9/16/2015) was to direct our first hop to Chicago instead of 
>>>>>>>>>>>>> Pittsburgh (3rox)
>>>>>>>>>>>>> which is now a 100 Gbit

>>>>>>>>>>>>> link. Tests to UCAR at that time were showing 1.38 Gbps 
>>>>>>>>>>>>> throughput with reduced
>>>>>>>>>>>>> latencies. Since

>>>>>>>>>>>>> our delays now are not as bad as they were previously, perhaps 
>>>>>>>>>>>>> this has helped.
>>>>>>>>>>>>> However, there may

>>>>>>>>>>>>> still be a choke point further down the line at maxgigapop or 
>>>>>>>>>>>>> internal to NCEP
>>>>>>>>>>>>> itself. perfsonar monitoring

>>>>>>>>>>>>> to NCEP would be useful... does anyone have any tests to NCEP 
>>>>>>>>>>>>> currently running?
>>>>>>>>>>>>> Can we identify

>>>>>>>>>>>>> any common path segments in the trace route above?

>>>>>>>>>>>>> Art

>>>>>>>>>>>>>> From: "Pete Pokrandt" < poker@xxxxxxxxxxxx >
>>>>>>>>>>>>>> To: "Arthur A Person" < aap1@xxxxxxx >, " 
>>>>>>>>>>>>>> support-conduit@xxxxxxxxxxxxxxxx " <
>>>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx >
>>>>>>>>>>>>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Daes Support" 
>>>>>>>>>>>>>> <
>>>>>>>>>>>>>> daessupport@xxxxxxxxxx >, "Carissa Klemmer - NOAA Federal" <
>>>>>>>>>>>>>> carissa.l.klemmer@xxxxxxxx >, "Tyle, Kevin R" < ktyle@xxxxxxxxxx 
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> Sent: Thursday, September 24, 2015 1:25:31 AM

>>>>>>>>>>>>>> Subject: Re: [conduit] How's your GFS?

>>>>>>>>>>>>>> Art,

>>>>>>>>>>>>>> Looks like you spoke too soon. Big lags ~1000 secs started up 
>>>>>>>>>>>>>> again with today's
>>>>>>>>>>>>>> 12 UTC cycle. Very mysterious..

>>>>>>>>>>>>>> They are showing up on our feed and consequently downstream from 
>>>>>>>>>>>>>> us at Albny.

>>>>>>>>>>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

>>>>>>>>>>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+install.atmos.albany.edu

>>>>>>>>>>>>>> <sigh..>

>>>>>>>>>>>>>> Pete

>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Pete Pokrandt - Systems Programmer
>>>>>>>>>>>>>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>>>>>>>>>>>>>> 608-262-3086 - poker@xxxxxxxxxxxx

>>>>>>>>>>>>>> From: Arthur A Person < aap1@xxxxxxx >
>>>>>>>>>>>>>> Sent: Monday, September 21, 2015 10:14 AM
>>>>>>>>>>>>>> To: support-conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>>>>>> Cc: Bentley, Alicia M; Daes Support; Carissa Klemmer - NOAA 
>>>>>>>>>>>>>> Federal; Pete
>>>>>>>>>>>>>> Pokrandt; Tyle, Kevin R
>>>>>>>>>>>>>> Subject: Re: [conduit] How's your GFS?

>>>>>>>>>>>>>> Folks,

>>>>>>>>>>>>>> Looks like something changed late on Friday in the network paths 
>>>>>>>>>>>>>> affecting Penn
>>>>>>>>>>>>>> State and the other universities feeding

>>>>>>>>>>>>>> CONDUIT from NCEP... delays have dropped to a crisp 30 seconds 
>>>>>>>>>>>>>> or less. Does
>>>>>>>>>>>>>> anyone know if a problem was found/fixed?

>>>>>>>>>>>>>> I know some issues were addressed at Penn State with some issues 
>>>>>>>>>>>>>> still being
>>>>>>>>>>>>>> worked-on. Back in the first week of

>>>>>>>>>>>>>> September the feeds were good and then degraded... just want to 
>>>>>>>>>>>>>> make sure that
>>>>>>>>>>>>>> doesn't happen again before I re-enable

>>>>>>>>>>>>>> ingest of the gfs 0.25 degree data.

>>>>>>>>>>>>>> Art

>>>>>>>>>>>>>>> From: "Arthur A Person" < aap1@xxxxxxx >
>>>>>>>>>>>>>>> To: " support-conduit@xxxxxxxxxxxxxxxx " < 
>>>>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx >
>>>>>>>>>>>>>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Daes 
>>>>>>>>>>>>>>> Support" <
>>>>>>>>>>>>>>> daessupport@xxxxxxxxxx >, "Carissa Klemmer - NOAA Federal" <
>>>>>>>>>>>>>>> carissa.l.klemmer@xxxxxxxx >
>>>>>>>>>>>>>>> Sent: Wednesday, September 9, 2015 4:27:26 PM
>>>>>>>>>>>>>>> Subject: Re: [conduit] How's your GFS?
>>>>>>>>>>>>>>> Just a heads up...

>>>>>>>>>>>>>>> I've reconfigured our IDD relay to distribute conduit without 
>>>>>>>>>>>>>>> the gfs 0.25
>>>>>>>>>>>>>>> degree data until we get our latencies under control. We've got 
>>>>>>>>>>>>>>> some issues
>>>>>>>>>>>>>>> internal to Penn State creating problems on top of any external 
>>>>>>>>>>>>>>> issues and our
>>>>>>>>>>>>>>> conduit feed is useless the way it is at the moment. By 
>>>>>>>>>>>>>>> reverting to conduit
>>>>>>>>>>>>>>> without gfs 0.25, hopefully we'll maintain a useful stream. As 
>>>>>>>>>>>>>>> soon as the
>>>>>>>>>>>>>>> latencies are addressed, I will reintroduce the gfs 0.25.

>>>>>>>>>>>>>>> Art

>>>>>>>>>>>>>>>> From: "Arthur A Person" < aap1@xxxxxxx >
>>>>>>>>>>>>>>>> To: "Carissa Klemmer - NOAA Federal" < 
>>>>>>>>>>>>>>>> carissa.l.klemmer@xxxxxxxx >
>>>>>>>>>>>>>>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Daes 
>>>>>>>>>>>>>>>> Support" <
>>>>>>>>>>>>>>>> daessupport@xxxxxxxxxx >, " support-conduit@xxxxxxxxxxxxxxxx " 
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx >
>>>>>>>>>>>>>>>> Sent: Wednesday, September 9, 2015 8:00:19 AM
>>>>>>>>>>>>>>>> Subject: Re: [conduit] How's your GFS?
>>>>>>>>>>>>>>>> All,

>>>>>>>>>>>>>>>>> From: "Carissa Klemmer - NOAA Federal" < 
>>>>>>>>>>>>>>>>> carissa.l.klemmer@xxxxxxxx >
>>>>>>>>>>>>>>>>> To: "Arthur A Person" < aap1@xxxxxxx >
>>>>>>>>>>>>>>>>> Cc: "Tyle, Kevin R" < ktyle@xxxxxxxxxx >, "Bentley, Alicia M" 
>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>> ambentley@xxxxxxxxxx >, "Daes Support" < 
>>>>>>>>>>>>>>>>> daessupport@xxxxxxxxxx >, "
>>>>>>>>>>>>>>>>> support-conduit@xxxxxxxxxxxxxxxx " < conduit@xxxxxxxxxxxxxxxx 
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> Sent: Tuesday, September 8, 2015 10:19:06 PM
>>>>>>>>>>>>>>>>> Subject: Re: [conduit] How's your GFS?
>>>>>>>>>>>>>>>>> All,

>>>>>>>>>>>>>>>>> NCEP is not making any active changes to our networks that 
>>>>>>>>>>>>>>>>> should affect your
>>>>>>>>>>>>>>>>> latencies, especially not over a weekend. I am not aware of 
>>>>>>>>>>>>>>>>> any changes that
>>>>>>>>>>>>>>>>> occurred over the holiday that would have impacted these 
>>>>>>>>>>>>>>>>> networks. This is
>>>>>>>>>>>>>>>>> likely downstream of NCEP control which is why you see the 
>>>>>>>>>>>>>>>>> latencies come and
>>>>>>>>>>>>>>>>> go.

>>>>>>>>>>>>>>>> Okay... I guess my interpretation was wrong, then. My 
>>>>>>>>>>>>>>>> apologies. There does seem
>>>>>>>>>>>>>>>> to be a problem pretty close to NCEP, however, since the 
>>>>>>>>>>>>>>>> latencies seem to come
>>>>>>>>>>>>>>>> and go at all top-tier sites... although not all sites are the 
>>>>>>>>>>>>>>>> same (ours seems
>>>>>>>>>>>>>>>> to be the highest). Maybe we're pushing the long-haul 
>>>>>>>>>>>>>>>> connectivity to the limit
>>>>>>>>>>>>>>>> and multiple choke points are showing up? Time to get our 
>>>>>>>>>>>>>>>> networking folks more
>>>>>>>>>>>>>>>> involved...

>>>>>>>>>>>>>>>> Art

>>>>>>>>>>>>>>>>> Carissa Klemmer
>>>>>>>>>>>>>>>>> NCEP Central Operations
>>>>>>>>>>>>>>>>> Production Management Branch Dataflow Team
>>>>>>>>>>>>>>>>> 301-683-3835

>>>>>>>>>>>>>>>>> On Tue, Sep 8, 2015 at 1:21 PM, Arthur A Person < 
>>>>>>>>>>>>>>>>> aap1@xxxxxxx > wrote:
>>>>>>>>>>>>>>>>>> We appear to have had gfs reception problems with 0Z and 6Z 
>>>>>>>>>>>>>>>>>> runs last night.
>>>>>>>>>>>>>>>>>> After implementation

>>>>>>>>>>>>>>>>>> of the 0.25 degree gfs, CONDUIT latencies were very large 
>>>>>>>>>>>>>>>>>> across all sites
>>>>>>>>>>>>>>>>>> during 0.25 degree data

>>>>>>>>>>>>>>>>>> transmission, but a week-or-so ago dropped to negligible 
>>>>>>>>>>>>>>>>>> levels. Over the
>>>>>>>>>>>>>>>>>> weekend they jumped

>>>>>>>>>>>>>>>>>> back up again. I interpret this to mean NCEP is tinkering 
>>>>>>>>>>>>>>>>>> with network paths
>>>>>>>>>>>>>>>>>> trying to find an effective

>>>>>>>>>>>>>>>>>> way to get these huge bursts of data out to the downstream 
>>>>>>>>>>>>>>>>>> sites. The gfs data
>>>>>>>>>>>>>>>>>> loss last night may have

>>>>>>>>>>>>>>>>>> been from the large latencies or from other unrelated 
>>>>>>>>>>>>>>>>>> delivery problems...
>>>>>>>>>>>>>>>>>> dunno...

>>>>>>>>>>>>>>>>>> Art

>>>>>>>>>>>>>>>>>>> From: "Tyle, Kevin R" < ktyle@xxxxxxxxxx >
>>>>>>>>>>>>>>>>>>> To: "Pete Pokrandt" < poker@xxxxxxxxxxxx >
>>>>>>>>>>>>>>>>>>> Cc: "Bentley, Alicia M" < ambentley@xxxxxxxxxx >, "Daes 
>>>>>>>>>>>>>>>>>>> Support" <
>>>>>>>>>>>>>>>>>>> daessupport@xxxxxxxxxx >, conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>>>>> Sent: Tuesday, September 8, 2015 1:00:10 PM
>>>>>>>>>>>>>>>>>>> Subject: Re: [conduit] How's your GFS?
>>>>>>>>>>>>>>>>>>> Hi Pete, et al.:

>>>>>>>>>>>>>>>>>>> We here at UAlbany continue to get spotty reception of the 
>>>>>>>>>>>>>>>>>>> GFS since 00Z today …
>>>>>>>>>>>>>>>>>>> anyone else having issues? We feed from Madison and State 
>>>>>>>>>>>>>>>>>>> College.

>>>>>>>>>>>>>>>>>>> Earlier thread below:

>>>>>>>>>>>>>>>>>>> -------------------------

>>>>>>>>>>>>>>>>>>> Yeah, I’m not surprised that the addition of the ¼ deg GFS 
>>>>>>>>>>>>>>>>>>> is causing the need
>>>>>>>>>>>>>>>>>>> for a bigger queue (and likely burlier machine). That’s the 
>>>>>>>>>>>>>>>>>>> main reason I have
>>>>>>>>>>>>>>>>>>> resisted requesting it.

>>>>>>>>>>>>>>>>>>> I’ll fix the issue that makes ldmstats show 
>>>>>>>>>>>>>>>>>>> “install.atmos…” instead of
>>>>>>>>>>>>>>>>>>> “cascade.atmos…”

>>>>>>>>>>>>>>>>>>> Something else must be at play since the ¼ GFS has been 
>>>>>>>>>>>>>>>>>>> flowing for several
>>>>>>>>>>>>>>>>>>> weeks now without incident, likely tied to the increased 
>>>>>>>>>>>>>>>>>>> latency you starting
>>>>>>>>>>>>>>>>>>> seeing.

>>>>>>>>>>>>>>>>>>> Looks like we only got the GFS through 60 hours today with 
>>>>>>>>>>>>>>>>>>> the 12Z run, so
>>>>>>>>>>>>>>>>>>> something definitely appears to be amiss … I’ll cc: the 
>>>>>>>>>>>>>>>>>>> conduit list to see if
>>>>>>>>>>>>>>>>>>> anyone else is noticing problems.

>>>>>>>>>>>>>>>>>>> _____________________________________________
>>>>>>>>>>>>>>>>>>> Kevin Tyle, Systems Administrator
>>>>>>>>>>>>>>>>>>> Dept. of Atmospheric & Environmental Sciences
>>>>>>>>>>>>>>>>>>> University at Albany
>>>>>>>>>>>>>>>>>>> Earth Science 235, 1400 Washington Avenue
>>>>>>>>>>>>>>>>>>> Albany, NY 12222
>>>>>>>>>>>>>>>>>>> Email: ktyle@xxxxxxxxxx
>>>>>>>>>>>>>>>>>>> Phone: 518-442-4578
>>>>>>>>>>>>>>>>>>> _____________________________________________

>>>>>>>>>>>>>>>>>>> From: Pete Pokrandt [mailto: poker@xxxxxxxxxxxx ]
>>>>>>>>>>>>>>>>>>> Sent: Tuesday, September 08, 2015 12:17 PM
>>>>>>>>>>>>>>>>>>> To: Tyle, Kevin R < ktyle@xxxxxxxxxx >
>>>>>>>>>>>>>>>>>>> Cc: Daes Support < daessupport@xxxxxxxxxx >
>>>>>>>>>>>>>>>>>>> Subject: Re: How's your GFS?

>>>>>>>>>>>>>>>>>>> My GFS appears to be complete, but I do see that 
>>>>>>>>>>>>>>>>>>> something's going on with our
>>>>>>>>>>>>>>>>>>> feed - the latencies jumped way up somewhere over the 
>>>>>>>>>>>>>>>>>>> weekend:

>>>>>>>>>>>>>>>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

>>>>>>>>>>>>>>>>>>> You're seeing the same, and increased yet from your feed 
>>>>>>>>>>>>>>>>>>> from Penn State (at
>>>>>>>>>>>>>>>>>>> least to the machine ' install.atmos.albany.edu - I don't 
>>>>>>>>>>>>>>>>>>> see any stats
>>>>>>>>>>>>>>>>>>> reported from cascade, which is what it looks like you are 
>>>>>>>>>>>>>>>>>>> feeding from me on)

>>>>>>>>>>>>>>>>>>> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+install.atmos.albany.edu

>>>>>>>>>>>>>>>>>>> I think I need to buy more memory and keep a larger queue 
>>>>>>>>>>>>>>>>>>> on idd.aos.wisc.edu
>>>>>>>>>>>>>>>>>>> with the 0.25 deg GFS coming in. There are times where my 
>>>>>>>>>>>>>>>>>>> queue only holds
>>>>>>>>>>>>>>>>>>> about 20 minutes of data, which is likely contributing to 
>>>>>>>>>>>>>>>>>>> your incomplete GFS
>>>>>>>>>>>>>>>>>>> files..

>>>>>>>>>>>>>>>>>>> Here's what my 0.5 deg (the gblav2.* files) and the 0.25 
>>>>>>>>>>>>>>>>>>> deg (gblav0p25 out to
>>>>>>>>>>>>>>>>>>> 87 h) look like for the 00 ant 06 UTC runs today

>>>>>>>>>>>>>>>>>>> 0.5 deg:
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 60953687 Sep 7 22:25 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F000
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 66996066 Sep 7 22:28 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F003
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 67902041 Sep 7 22:30 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F006
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 67961293 Sep 7 22:32 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F009
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68081826 Sep 7 22:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F012
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68710398 Sep 7 22:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F015
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69664268 Sep 7 22:36 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F018
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69177180 Sep 7 22:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F021
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69816235 Sep 7 22:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F024
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69010253 Sep 7 22:39 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F027
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69786985 Sep 7 22:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F030
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68876266 Sep 7 22:41 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F033
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69376601 Sep 7 22:42 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F036
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69029846 Sep 7 22:43 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F039
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69142392 Sep 7 22:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F042
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68990399 Sep 7 22:45 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F045
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69343366 Sep 7 22:46 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F048
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69150894 Sep 7 22:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F051
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69504675 Sep 7 22:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F054
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69196832 Sep 7 22:48 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F057
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69335487 Sep 7 22:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F060
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69261676 Sep 7 22:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F063
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69166068 Sep 7 22:51 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F066
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69054105 Sep 7 22:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F069
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68895264 Sep 7 22:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F072
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69202038 Sep 7 22:56 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F075
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69339334 Sep 7 22:56 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F078
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69181930 Sep 7 22:57 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F081
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69674148 Sep 7 22:58 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F084
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69383769 Sep 7 22:58 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F087
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69645526 Sep 7 22:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F090
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69119323 Sep 7 23:00 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F093
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69363296 Sep 7 23:01 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F096
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69030287 Sep 7 23:03 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F099
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69819322 Sep 7 23:03 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F102
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69498561 Sep 7 23:04 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F105
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69690447 Sep 7 23:05 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F108
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69274213 Sep 7 23:06 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F111
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70089206 Sep 7 23:07 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F114
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70007688 Sep 7 23:08 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F117
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70237308 Sep 7 23:08 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F120
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69849708 Sep 7 23:09 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F123
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69883550 Sep 7 23:11 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F126
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69586365 Sep 7 23:11 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F129
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70110782 Sep 7 23:12 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F132
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69430545 Sep 7 23:13 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F135
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69461630 Sep 7 23:14 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F138
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69264487 Sep 7 23:15 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F141
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69553206 Sep 7 23:16 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F144
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68924371 Sep 7 23:17 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F147
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69191965 Sep 7 23:17 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F150
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68639462 Sep 7 23:19 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F153
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69035706 Sep 7 23:22 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F156
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68831618 Sep 7 23:25 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F159
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69428952 Sep 7 23:27 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F162
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69514672 Sep 7 23:28 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F165
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69614097 Sep 7 23:29 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F168
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69404524 Sep 7 23:29 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F171
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69534566 Sep 7 23:30 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F174
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69528455 Sep 7 23:31 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F177
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69747643 Sep 7 23:31 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F180
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69397125 Sep 7 23:32 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F183
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69973323 Sep 7 23:32 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F186
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69070113 Sep 7 23:33 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F189
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69586837 Sep 7 23:34 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F192
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69202267 Sep 7 23:34 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F195
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69169373 Sep 7 23:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F198
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68193948 Sep 7 23:36 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F201
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 67963148 Sep 7 23:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F204
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 67689203 Sep 7 23:39 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F207
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68079977 Sep 7 23:41 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F210
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68931672 Sep 7 23:43 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F213
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68749459 Sep 7 23:46 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F216
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68739072 Sep 7 23:46 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F219
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68789427 Sep 7 23:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F222
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68031035 Sep 7 23:48 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F225
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68735199 Sep 7 23:48 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F228
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 65347330 Sep 7 23:49 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F231
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 65891902 Sep 7 23:49 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F234
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 65383729 Sep 7 23:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F237
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 66299227 Sep 7 23:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F240
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64525715 Sep 7 23:52 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F252
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64515690 Sep 7 23:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F264
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 63803271 Sep 7 23:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F276
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 63261621 Sep 7 23:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F288
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64171542 Sep 7 23:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F300
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64308576 Sep 7 23:56 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F312
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64334459 Sep 7 23:58 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F324
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64189700 Sep 7 23:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F336
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 63829248 Sep 7 23:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F348
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64655803 Sep 8 00:00 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F360
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64436657 Sep 8 00:07 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F372
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64546095 Sep 8 00:12 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090800_F384
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 61169101 Sep 8 04:26 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F000
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 67422108 Sep 8 04:28 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F003
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68374534 Sep 8 04:31 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F006
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68543418 Sep 8 04:33 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F009
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69298218 Sep 8 04:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F012
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69188133 Sep 8 04:36 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F015
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69917655 Sep 8 04:37 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F018
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69558566 Sep 8 04:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F021
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69659459 Sep 8 04:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F024
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69288102 Sep 8 04:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F027
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68686968 Sep 8 04:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F030
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68640234 Sep 8 04:42 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F033
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69544506 Sep 8 04:42 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F036
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68462036 Sep 8 04:43 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F039
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69287354 Sep 8 04:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F042
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69228412 Sep 8 04:45 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F045
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69444769 Sep 8 04:46 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F048
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69089036 Sep 8 04:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F051
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69542812 Sep 8 04:48 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F054
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69377775 Sep 8 04:49 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F057
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69324867 Sep 8 04:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F060
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69313464 Sep 8 04:51 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F063
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69820155 Sep 8 04:52 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F066
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69484687 Sep 8 04:52 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F069
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69581997 Sep 8 04:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F072
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69189693 Sep 8 04:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F075
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69751906 Sep 8 04:55 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F078
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69558875 Sep 8 04:56 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F081
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69903084 Sep 8 04:58 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F084
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69627748 Sep 8 04:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F087
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69678696 Sep 8 04:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F090
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69497446 Sep 8 05:00 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F093
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69735442 Sep 8 05:01 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F096
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69767861 Sep 8 05:02 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F099
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70169785 Sep 8 05:03 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F102
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69625644 Sep 8 05:04 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F105
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69954293 Sep 8 05:05 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F108
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69996186 Sep 8 05:06 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F111
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70297897 Sep 8 05:06 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F114
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70037957 Sep 8 05:08 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F117
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69968183 Sep 8 05:08 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F120
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69564905 Sep 8 05:10 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F123
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69725865 Sep 8 05:11 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F126
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69349475 Sep 8 05:11 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F129
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69625604 Sep 8 05:12 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F132
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69392152 Sep 8 05:15 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F135
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69551134 Sep 8 05:18 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F138
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69108820 Sep 8 05:19 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F141
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69469618 Sep 8 05:19 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F144
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 68774645 Sep 8 05:20 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F147
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69135260 Sep 8 05:20 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F150
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69009857 Sep 8 05:21 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F153
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69647753 Sep 8 05:21 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F156
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69604259 Sep 8 05:22 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F159
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69851358 Sep 8 05:22 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F162
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69621423 Sep 8 05:23 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F165
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69987289 Sep 8 05:24 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F168
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70009168 Sep 8 05:24 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F171
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70272431 Sep 8 05:25 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F174
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69951044 Sep 8 05:26 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F177
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70294466 Sep 8 05:28 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F180
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69693077 Sep 8 05:31 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F183
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70277595 Sep 8 05:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F186
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70161497 Sep 8 05:36 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F189
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70075264 Sep 8 05:37 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F192
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69929971 Sep 8 05:37 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F195
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69879151 Sep 8 05:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F198
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69726455 Sep 8 05:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F201
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70186834 Sep 8 05:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F204
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69735649 Sep 8 05:39 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F207
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70062469 Sep 8 05:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F210
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69475211 Sep 8 05:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F213
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69688060 Sep 8 05:41 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F216
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69169089 Sep 8 05:42 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F219
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69623322 Sep 8 05:42 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F222
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69434126 Sep 8 05:43 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F225
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69447710 Sep 8 05:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F228
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69232930 Sep 8 05:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F231
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69688395 Sep 8 05:45 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F234
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 69476983 Sep 8 05:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F237
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 70027781 Sep 8 05:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F240
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64748968 Sep 8 05:52 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F252
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64729059 Sep 8 05:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F264
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64211460 Sep 8 05:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F276
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64117374 Sep 8 05:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F288
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64123032 Sep 8 05:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F300
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64714736 Sep 8 05:56 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F312
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 65052210 Sep 8 05:56 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F324
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 65123631 Sep 8 05:57 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F336
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64903451 Sep 8 05:58 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F348
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64423290 Sep 8 05:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F360
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 64365594 Sep 8 05:59 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F372
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 63855749 Sep 8 06:07 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav2.15090806_F384

>>>>>>>>>>>>>>>>>>> 0.25 deg

>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 180752345 Sep 7 22:25 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_Fanl
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 197882387 Sep 7 22:26 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F000
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 217304897 Sep 7 22:27 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F003
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 221447144 Sep 7 22:30 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F006
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 221383770 Sep 7 22:32 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F009
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 222748480 Sep 7 22:34 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F012
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224209489 Sep 7 22:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F015
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226360332 Sep 7 22:36 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F018
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225185199 Sep 7 22:37 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F021
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226720828 Sep 7 22:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F024
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224211990 Sep 7 22:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F027
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226623368 Sep 7 22:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F030
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224601041 Sep 7 22:41 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F033
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225696377 Sep 7 22:41 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F036
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224803488 Sep 7 22:43 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F039
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225463303 Sep 7 22:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F042
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224172234 Sep 7 22:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F045
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225750651 Sep 7 22:45 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F048
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224513834 Sep 7 22:46 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F051
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225871134 Sep 7 22:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F054
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224871484 Sep 7 22:48 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F057
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225954437 Sep 7 22:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F060
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225600052 Sep 7 22:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F063
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225672348 Sep 7 22:51 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F066
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225064451 Sep 7 22:52 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F069
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225318101 Sep 7 22:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F072
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225303961 Sep 7 22:57 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F075
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226805528 Sep 7 23:03 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F078
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226187062 Sep 7 23:08 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F081
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 227313364 Sep 7 23:09 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F084
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226221831 Sep 7 23:15 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090800_F087
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 197951753 Sep 8 04:25 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F000
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 181438882 Sep 8 04:26 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_Fanl
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 218273142 Sep 8 04:28 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F003
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 222180270 Sep 8 04:30 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F006
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 222627637 Sep 8 04:32 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F009
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225440960 Sep 8 04:34 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F012
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224877734 Sep 8 04:35 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F015
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226700650 Sep 8 04:36 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F018
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225325799 Sep 8 04:37 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F021
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226163438 Sep 8 04:38 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F024
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225234793 Sep 8 04:39 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F027
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 224315172 Sep 8 04:40 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F030
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 223485303 Sep 8 04:41 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F033
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226101395 Sep 8 04:42 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F036
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 222880336 Sep 8 04:43 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F039
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225276943 Sep 8 04:44 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F042
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225167793 Sep 8 04:45 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F045
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225771493 Sep 8 04:46 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F048
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225066649 Sep 8 04:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F051
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225905191 Sep 8 04:47 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F054
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225706912 Sep 8 04:48 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F057
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225891555 Sep 8 04:49 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F060
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225723607 Sep 8 04:50 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F063
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 227329359 Sep 8 04:51 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F066
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226381130 Sep 8 04:52 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F069
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 227000926 Sep 8 04:53 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F072
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 225483067 Sep 8 04:54 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F075
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 227295269 Sep 8 04:55 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F078
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226316715 Sep 8 04:55 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F081
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 227632093 Sep 8 04:57 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F084
>>>>>>>>>>>>>>>>>>> -rw-r--r--. 1 ldm ldm 226447758 Sep 8 05:01 
>>>>>>>>>>>>>>>>>>> /data/grib2/gblav0p25.15090806_F087

>>>>>>>>>>>>>>>>>>> Pete

>>>>>>>>>>>>>>>>>>> On 09/08/2015 08:43 AM, Tyle, Kevin R wrote:

>>>>>>>>>>>>>>>>>>>> Hi Pete,

>>>>>>>>>>>>>>>>>>>> We’ve had incomplete GFS the last two runs (00 and 06 UTC 
>>>>>>>>>>>>>>>>>>>> today) … how did
>>>>>>>>>>>>>>>>>>>> things look on your end?

>>>>>>>>>>>>>>>>>>>> Thanks,

>>>>>>>>>>>>>>>>>>>> Kevin

>>>>>>>>>>>>>>>>>>>> _____________________________________________
>>>>>>>>>>>>>>>>>>>> Kevin Tyle, Systems Administrator
>>>>>>>>>>>>>>>>>>>> Dept. of Atmospheric & Environmental Sciences
>>>>>>>>>>>>>>>>>>>> University at Albany
>>>>>>>>>>>>>>>>>>>> Earth Science 235, 1400 Washington Avenue
>>>>>>>>>>>>>>>>>>>> Albany, NY 12222
>>>>>>>>>>>>>>>>>>>> Email: ktyle@xxxxxxxxxx
>>>>>>>>>>>>>>>>>>>> Phone: 518-442-4578
>>>>>>>>>>>>>>>>>>>> _____________________________________________

>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Pete Pokrandt - Systems Programmer
>>>>>>>>>>>>>>>>>>> UW-Madison Dept of Atmospheric and Oceanic Sciences
>>>>>>>>>>>>>>>>>>> 608-262-3086 - poker@xxxxxxxxxxxx

>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>> conduit mailing list
>>>>>>>>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>>>>> For list information or to unsubscribe, visit:
>>>>>>>>>>>>>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>>>>>>>>>>>>> --

>>>>>>>>>>>>>>>>>> Arthur A. Person
>>>>>>>>>>>>>>>>>> Research Assistant, System Administrator
>>>>>>>>>>>>>>>>>> Penn State Department of Meteorology
>>>>>>>>>>>>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> conduit mailing list
>>>>>>>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>>>> For list information or to unsubscribe, visit:
>>>>>>>>>>>>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>>>>>>>>>>> --

>>>>>>>>>>>>>>>> Arthur A . Person
>>>>>>>>>>>>>>>> Research Assistant, System Administrator
>>>>>>>>>>>>>>>> Penn State Department of Meteorology
>>>>>>>>>>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> conduit mailing list
>>>>>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>> For list information or to unsubscribe, visit:
>>>>>>>>>>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>>>>>>>>>> --

>>>>>>>>>>>>>>> Arthur A. Person
>>>>>>>>>>>>>>> Research Assistant, System Administrator
>>>>>>>>>>>>>>> Penn State Department of Meteorology
>>>>>>>>>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>>>>>>>>> --

>>>>>>>>>>>>>> Arthur A. Person
>>>>>>>>>>>>>> Research Assistant, System Administrator
>>>>>>>>>>>>>> Penn State Department of Meteorology
>>>>>>>>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>>>>>>>> --

>>>>>>>>>>>>> Arthur A. Person
>>>>>>>>>>>>> Research Assistant, System Administrator
>>>>>>>>>>>>> Penn State Department of Meteorology
>>>>>>>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> conduit mailing list
>>>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>>>> For list information or to unsubscribe, visit:
>>>>>>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>>>>> _______________________________________________
>>>>>>>>>> conduit mailing list
>>>>>>>>>> conduit@xxxxxxxxxxxxxxxx
>>>>>>>>>> For list information or to unsubscribe, visit:
>>>>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>>>>> --

>>>>>>>>> Arthur A. Person
>>>>>>>>> Research Assistant, System Administrator
>>>>>>>>> Penn State Department of Meteorology
>>>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>> --

>>>>>>> Arthur A. Person
>>>>>>> Research Assistant, System Administrator
>>>>>>> Penn State Department of Meteorology
>>>>>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>>>>>> _______________________________________________
>>>>>>> conduit mailing list
>>>>>>> conduit@xxxxxxxxxxxxxxxx
>>>>>>> For list information or to unsubscribe, visit:
>>>>>>> http://www.unidata.ucar.edu/mailing_lists/

>>>>>> _______________________________________________
>>>>>> Ncep.list.pmb-dataflow mailing list
>>>>>> Ncep.list.pmb-dataflow@xxxxxxxxxxxxxxxxxxxx
>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.pmb-dataflow

>>>> _______________________________________________
>>>> conduit mailing list
>>>> conduit@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>> --

>>> Arthur A. Person
>>> Research Assistant, System Administrator
>>> Penn State Department of Meteorology
>>> email: aap1@xxxxxxx , phone: 814-863-1563

>>> _______________________________________________
>>> conduit mailing list
>>> conduit@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe, visit:
>>> http://www.unidata.ucar.edu/mailing_lists/
>> --

>> Arthur A. Person
>> Research Assistant, System Administrator
>> Penn State Department of Meteorology
>> email: aap1@xxxxxxx , phone: 814-863-1563

>> _______________________________________________
>> Ncep.list.pmb-dataflow mailing list
>> Ncep.list.pmb-dataflow@xxxxxxxxxxxxxxxxxxxx
>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.pmb-dataflow

> --
> Carissa Klemmer
> NCEP Central Operations
> Dataflow Team Lead
> 301-683-3835

-- 
Arthur A. Person 
Research Assistant, System Administrator 
Penn State Department of Meteorology 
email: aap1@xxxxxxx, phone: 814-863-1563 
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the conduit archives: