Re: [conduit] [Ncep.list.pmb-dataflow] How's your GFS?

If your have 'mtr' Âon the NCEP network run it to the LDM downstream feeds to 
see if there are any spikes. Basic ping / ldmping test can rule in or network 
latency.Â
Also if they can or you can set up iperf (super simple) and have the other site 
run a bandwidth test you can test the throughput to rule that in/out as an 
issue.Â
If th network checks clear, I would look closer at the LDM config, especially 
the queues. Unidata can speak more to that and also splitting the downstream 
feeds using regex patterns. Â
- MikeÂ

Sent from Outlook




On Fri, Nov 13, 2015 at 11:53 AM -0800, "Michael Shedlock" 
<michael.shedlock@xxxxxxxx> wrote:











  
  
    All,

    

    NCEP is indeed on internet2, which I presume would apply here. 

    

    A couple of noteworthy things.... I see some latency, but not for
    everyone, and it doesn't seem to matter which conduit machine a
    client is connected to. For example, with today's and yesterday's
    gfs.t12z.pgrb2.0p25.f096 (hour 96) file here are the latencies per
    client that I see:

    11/12

      Wisconsin: A few seconds

      Unidata/UCAR: A few seconds

      UIUC: 13 minutes

      PSU: 27 minutes

      

      11/13:

      Wisconsin: A few seconds

      Unidata/UCAR: A few seconds

      UIUC: 2.33 minutes

      PSU: 2.75 minutes

    
    

    Another correlation is that UIUC and PSU (the ones with latency) are
    only using one thread to connect to our conduit, whereas Wisc. and
    Unidata use multiple threads.

    

    At the moment this sort of has the appearance of a bottleneck
    outside of NCEP. It might also be useful to see traceroutes from
    UIUC and PSU to NCEP's CONDUIT. I know I saw some traceroutes
    below. Can you try that and share with us?

    

    

    Mike Shedlock

    NCEP Central Operations

    Dataflow Team

    301.683.3834

    

    

    On 11/13/2015 11:42 AM, Mike Dross
      wrote:

    
    
      
      My $ 0.02 from having works with LDM since the mid 90's.Â
      

      
      I assume NCEP is not on internet2?
        ÂIf so bandwidth shouldn't be an issue. Regardless I would check
        the traceroutes to ensure a good path, high bandwidth, low
        latency. ÂBasic network topology check. I am sure you have done
        this.
      

      
      An iperf test is a simple way to test
        the maximum throughput to see if bandwidth is an issue. If
        that's not it, high latency or the way LDM is set up on the
        upstream side is likely the culprit.Â
      

      
      MikeÂ
      

      
      

        Sent from my iPad
      

        On Nov 13, 2015, at 10:05 AM, Arthur A Person <aap1@xxxxxxx>
        wrote:

        

      
      
        
          
            Carissa,
            

            
            Yes, still issues. ÂThere was a period several weeks
              ago when throughput was clean, but recently we've seen
              delays to varying degrees.
            Based on the Unidata latency chart from our reported
              statistics 
(http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+iddrs2a.meteo.psu.edu),
             we've seen delays during 0.25 degree gfs transmission
              that range fromÂ500 seconds
                to 3500 seconds overÂthe past couple of days.
            Also, comparison with
                charts from other schools seem to show better receptionÂwhenÂ
feeding from "conduit1" rather
                than "conduit2".
            Does this mean anything
                to you or is it purely coincidence or incidental?
            

            
            Thanks for any insights you can provide.
            

            
            Â Â Â Â Â Â Â Â Â Â Art
            

            
            
            
              From: "Carissa
                Klemmer - NOAA Federal" <carissa.l.klemmer@xxxxxxxx>

                To: "Arthur A Person" <aap1@xxxxxxx>,
                "_NCEP.List.pmb-dataflow" <ncep.list.pmb-dataflow@xxxxxxxx>

                Cc: "support-conduit@xxxxxxxxxxxxxxxx"
                <conduit@xxxxxxxxxxxxxxxx>,
                "Pete Pokrandt" <poker@xxxxxxxxxxxx>,
                "Michael Schmidt" <mschmidt@xxxxxxxx>,
                "Bentley, Alicia M" <ambentley@xxxxxxxxxx>,
                "Daes Support" <daessupport@xxxxxxxxxx>

                Sent: Friday, November 13, 2015 9:26:28 AM

                Subject: Re: [conduit] How's your GFS?

              
            
            
              
                
                  Art, 

                    I am going to add our team to this thread. Are you
                    still seeing issues. Is so we will take a look and
                    see if we can tell if anything on our side is
                    happening around FH 96.Â
                  

                    
                      
                        
                          Carissa
                                Klemmer
                              

                            NCEP
                                Central Operations
                              

                            Dataflow
                                Team Lead

                            301-683-3835
                        
                      
                    
                    

                    On Thu, Nov 5, 2015 at 4:23
                      PM, Arthur A Person <aap1@xxxxxxx>
                      wrote:

                      
                        
                          
                            Hi all...

                            
                            

                            Conduit latencies have crept upward
                              again for the past few weeks... not
                              unbearable, but still significant.

                            
                            At first it seemed to only affect us,
                              but it looks like it's affecting UIUC now
                              also, but not so much Wisconsin.

                            
                            Inspecting our logs, we've noticed that
                              there's no delay out to about 90 hours of
                              gfs transmission, but

                            
                            starting at 96 hours, the delays start
                              to ramp up steadily. I'm not sure how to
                              explain that unless something

                            
                            else starts transmitting during that
                              time that competes for bandwidth. Also, I
                              notice that sites receiving data

                            
                            from "conduit1" seem to be faring
                              better than "conduit2". Is there any
                              difference between these two

                            
                            originating systems or is that just
                              coincidental? Anyone have anything new to
                              report on this issue?

                            
                            

                            Thanks... Art

                            
                            

                            
                            
                              From: "Pete Pokrandt"
                                <poker@xxxxxxxxxxxx>

                                To: "Carissa Klemmer - NOAA
                                Federal" <carissa.l.klemmer@xxxxxxxx>,
                                mschmidt@xxxxxxxx

                                  Cc: "Bentley, Alicia M" 
<ambentley@xxxxxxxxxx>,
                                  "Daes Support" <daessupport@xxxxxxxxxx>,
                                  "support-conduit@xxxxxxxxxxxxxxxx"
                                  <conduit@xxxxxxxxxxxxxxxx>

                                Sent: Thursday, September
                                24, 2015 1:29:59 PM
                                
                                  

                                    Subject: Re: [conduit] How's
                                    your GFS?

                                  
                                
                              
                            
                            
                              
                                
                                  
                                    
                                      

Here are traceroutes from idd.aos.wisc.edu
                                        to conduit.ncep.noaa.gov
                                        and ncepldm4.woc.noaa.gov
                                        taken at 17:17 UTC, right in the
                                        middle of the 18 UTC GFS lag
                                        spike today.
                                      



                                      
                                      



                                      
                                      

[ldm@idd ~/etc]$ traceroute conduit.ncep.noaa.gov

                                        traceroute to conduit.ncep.noaa.gov
                                        (140.90.101.42), 30 hops max, 60
                                        byte packets

                                        Â1Â 
r-cssc-b280c-1-core-vlan-510-primary.net.wisc.edu
                                        (144.92.130.3) 0.833 ms 0.819
                                        ms 0.855 ms

                                        Â2Â internet2-ord-600w-100G.net.wisc.edu
                                        (144.92.254.229)Â 18.077 msÂ
                                        18.095 ms 18.067 ms

                                        Â3Â 
et-7-0-0.115.rtr.wash.net.internet2.edu
                                        (198.71.45.57)Â 35.125 msÂ
                                        35.278 ms 35.261 ms

                                        Â4Â 198.71.45.228
                                        (198.71.45.228)Â 35.378 msÂ
                                        35.368 ms 35.335 ms

                                        Â5Â ae0.clpk-core.maxgigapop.net
                                        (206.196.178.81)Â 36.401 msÂ
                                        36.408 ms 36.284 ms

                                        Â6Â noaa-rtr.maxgigapop.net
                                        (206.196.177.118)Â 36.523 msÂ
                                        36.640 ms 36.411 ms

                                        Â7Â 140.90.111.36
                                        (140.90.111.36)Â 68.769 msÂ
                                        52.236 ms 52.210 ms

                                        Â8Â 140.90.76.69 (140.90.76.69)Â
                                        36.602 ms 36.503 ms 36.827 ms

                                        Â9Â * * *

                                        10Â * * *

                                        ...
                                      



                                      
                                      



                                      
                                      

[ldm@idd ~/etc]$ traceroute ncepldm4.woc.noaa.gov

                                        traceroute to ncepldm4.woc.noaa.gov
                                        (140.172.17.205), 30 hops max,
                                        60 byte packets

                                        Â1Â 
r-cssc-b280c-1-core-vlan-510-primary.net.wisc.edu
                                        (144.92.130.3) 0.838 ms 0.847
                                        ms 0.822 ms

                                        Â2Â internet2-ord-600w-100G.net.wisc.edu
                                        (144.92.254.229)Â 18.513 msÂ
                                        18.506 ms 18.484 ms

                                        Â3Â ae0.3454.core-l3.frgp.net
                                        (192.43.217.223)Â 40.245 msÂ
                                        40.204 ms 40.123 ms

                                        Â4Â noaa-i2.frgp.net
                                        (128.117.243.11)Â 43.617 msÂ
                                        43.544 ms 43.699 ms

                                        Â5Â 2001-mlx8-eth-1-2.boulder.noaa.gov
                                        (140.172.2.18)Â 40.960 msÂ
                                        40.951 ms 41.058 ms

                                        Â6Â mdf-rtr-6.boulder.noaa.gov
                                        (140.172.6.251)Â 46.516 msÂ
                                        40.962 ms 40.876 ms

                                        Â7Â * * *

                                        Â8Â * * *

                                        ...

                                      
                                      



                                      
                                      
                                        
                                          

--

                                            Pete Pokrandt - Systems
                                            Programmer

                                            UW-Madison Dept of
                                            Atmospheric and Oceanic
                                            Sciences

                                            608-262-3086Â - poker@xxxxxxxxxxxx

                                            

                                          
                                        
                                      
                                      

                                      

                                      
                                        
                                        From:
                                            conduit-bounces@xxxxxxxxxxxxxxxx
                                            <conduit-bounces@xxxxxxxxxxxxxxxx>
                                            on behalf of Carissa Klemmer
                                            - NOAA Federal 
<carissa.l.klemmer@xxxxxxxx>

                                            Sent: Thursday,
                                            September 24, 2015 10:36 AM

                                            To: mschmidt@xxxxxxxx

                                            Cc: Bentley, Alicia
                                            M; Daes Support; 
support-conduit@xxxxxxxxxxxxxxxx

                                            Subject: Re:
                                            [conduit] How's your GFS?
                                          Â
                                        
                                        
                                          
                                            Mike,
                                              

                                              

                                            
                                            Can
                                              you provide what server
                                              you are coming from. I
                                              know your range, but I
                                              need to provide to the
                                              helpdesk what is primary
                                              right now so they can
                                              trace back.
                                              

                                            
                                          
                                          

                                            
                                              
                                                
                                                  Carissa
                                                        Klemmer
                                                      

                                                    NCEP
                                                        Central
                                                        Operations
                                                      

                                                    Dataflow
                                                        Team Lead

                                                    301-683-3835
                                                
                                              
                                            
                                            

                                            On
                                              Thu, Sep 24, 2015 at 9:57
                                              AM, Mike Schmidt 
                                                <mschmidt@xxxxxxxx>
                                              wrote:

                                              
                                                
                                                  
                                                    Hi Carissa,

                                                      

                                                      We've seen the
                                                      same jump in
                                                      latencies;

                                                      

                                                      
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+daffy.unidata.ucar.edu

                                                      

                                                    
                                                    Here's our
                                                    traceroute;

                                                    

                                                    # traceroute 
conduit.ncep.noaa.gov

                                                      traceroute to 
conduit.ncep.noaa.gov
                                                      (140.90.101.42),
                                                      30 hops max, 60
                                                      byte packets

                                                    Â1Â 
flra-n156.unidata.ucar.edu
                                                    (128.117.156.253)Â
                                                    0.352 ms 0.344 msÂ
                                                    0.325 ms

                                                    Â2Â 
tcom-gs-1-n243-80.ucar.edu
                                                    (128.117.243.85)Â
                                                    0.558 ms 0.584 msÂ
                                                    0.662 ms

                                                    Â3Â 
xe-0-1-2.873.core-l3.frgp.net
                                                    (128.117.243.9)Â
                                                    1.138 ms 1.126 msÂ
                                                    1.107 ms

                                                    Â4Â v3454.rtr-chic.frgp.net
                                                    (192.43.217.222)Â
                                                    23.227 ms 23.296
                                                    ms 23.278 ms

                                                    Â5Â 
et-7-0-0.115.rtr.wash.net.internet2.edu
                                                    (198.71.45.57)Â
                                                    40.421 ms 40.408
                                                    ms 40.340 ms

                                                    Â6Â 198.71.45.228
                                                    (198.71.45.228)Â
                                                    40.488 ms 40.649
                                                    ms 40.624 ms

                                                    Â7Â 
ae0.clpk-core.maxgigapop.net
                                                    (206.196.178.81)Â
                                                    41.545 ms 41.602
                                                    ms 41.170 ms

                                                    Â8Â noaa-rtr.maxgigapop.net
                                                    (206.196.177.118)Â
                                                    41.796 ms 41.507
                                                    ms 41.592 ms

                                                    Â9Â 140.90.111.36
                                                    (140.90.111.36)Â
                                                    41.419 ms 41.496
                                                    ms 41.623 ms

                                                    10Â 140.90.76.69
                                                    (140.90.76.69)Â
                                                    41.900 ms 41.728
                                                    ms 41.956 ms

                                                        

                                                      
                                                  mike

                                                      

                                                      
                                                    
                                                
                                                  
                                                    

                                                      On
                                                        Thu, Sep 24,
                                                        2015 at 7:49 AM,
                                                        Carissa Klemmer
                                                        - NOAA Federal
                                                        
<carissa.l.klemmer@xxxxxxxx>
                                                        wrote:

                                                        
                                                          
                                                          Hi
                                                          all,

                                                          

                                                          
                                                          I
                                                          have opened a
                                                          ticket with
                                                          our helpdesk
                                                          and included
                                                          PSU
                                                          traceroute.
                                                          But can I get
                                                          a better
                                                          handle on all
                                                          the paths that
                                                          are seeing
                                                          latencies to
                                                          conduit.ncep.noaa.gov?
                                                          Is both PSU
                                                          and WISC
                                                          seeing spikes?
                                                          Can I get a
                                                          WISC
                                                          traceroute
                                                          also please?
                                                          

                                                          

                                                          
                                                          Thanks,

                                                          
                                                          
                                                          

                                                          
                                                          
                                                          
                                                          Carissa
                                                          Klemmer
                                                          

                                                          NCEP
                                                          Central
                                                          Operations
                                                          

                                                          Dataflow
                                                          Team Lead

                                                          301-683-3835

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          

                                                          On
                                                          Thu, Sep 24,
                                                          2015 at 8:19
                                                          AM, Arthur A
                                                          Person 
                                                          <aap1@xxxxxxx>
                                                          wrote:

                                                          
                                                          
                                                          
                                                          Pete,
                                                          

                                                          
                                                          I was
                                                          thinking that
                                                          too! If I
                                                          only hadn't
                                                          sent that
                                                          email... Â :)
                                                          

                                                          
                                                          Anyway,
                                                          the delays
                                                          aren't as bad
                                                          as they were
                                                          (at least
                                                          here), but are
                                                          still
                                                          indicative of
                                                          a
                                                          lurking
                                                          problem.Â
                                                          Almost seems
                                                          as though some
                                                          packet shaping
                                                          is going on,
                                                          as Tom
                                                          suggested
                                                          previously.Â
                                                          Maybe paths
                                                          get overloaded
                                                          and something
                                                          kicks in and
                                                          meters-out
                                                          usage???Â
                                                          JustÂspeculating.Â
                                                          I've asked our
                                                          network folks
                                                          here to see if
                                                          they can
                                                          investigate
                                                          our
                                                          pathÂto NCEP,
                                                          but that may
                                                          take awhile.Â
                                                          Our traceroute
                                                          from this
                                                          morning at
                                                          1113Z is:
                                                          

                                                          
                                                          [ldm@iddrs1a
                                                          ~]$ traceroute
                                                          
conduit.ncep.noaa.gov

                                                          traceroute to
                                                          conduit.ncep.noaa.gov
                                                          (140.90.101.42),
                                                          30 hops max,
                                                          60 byte
                                                          packets

                                                          1 172.29.0.66
                                                          (172.29.0.66)
                                                          0.882 ms
                                                          192.5.158.1
                                                          (192.5.158.1)
                                                          0.278 ms 0.264
                                                          ms

                                                          2 
Blue1-ethernet3-1.gw.psu.edu
                                                          (172.30.5.178)
                                                          0.220 ms
                                                          
White1-ethernet3-1.gw.psu.edu
                                                          (172.30.5.177)
                                                          0.530 ms 0.526
                                                          ms

                                                          3 
Windstream1-ethernet2-1.gw.psu.edu
                                                          (172.30.5.106)
                                                          0.385 ms
                                                          
Telecom5-ethernet2-2.gw.psu.edu
                                                          (172.30.5.102)
                                                          0.370 ms
                                                          
Windstream1-ethernet3-2.gw.psu.edu
                                                          (172.30.5.114)
                                                          0.391 ms

                                                          4 
Telecom5-ethernet2-1.gw.psu.edu
                                                          (172.30.8.10)
                                                          0.391 ms 0.408
                                                          ms
                                                          
et-8-0-0.2364.rtr.chic.net.internet2.edu
                                                          (64.57.30.2)
                                                          15.149 ms

                                                          5 
et-7-0-0.115.rtr.wash.net.internet2.edu
                                                          (198.71.45.57)
                                                          32.276 ms
                                                          
et-8-0-0.2364.rtr.chic.net.internet2.edu
                                                          (64.57.30.2)
                                                          15.301 ms
                                                          
et-7-0-0.115.rtr.wash.net.internet2.edu
                                                          (198.71.45.57)
                                                          32.594 ms

                                                          6
                                                          198.71.45.228
                                                          (198.71.45.228)
                                                          32.423 ms 
et-7-0-0.115.rtr.wash.net.internet2.edu (198.71.45.57) 32.431 ms
                                                          198.71.45.228
                                                          (198.71.45.228)
                                                          32.843 ms

                                                          7
                                                          198.71.45.228
                                                          (198.71.45.228)
                                                          32.853 ms 
ae0.clpk-core.maxgigapop.net (206.196.178.81) 33.407 ms 33.401 ms

                                                          8 
ae0.clpk-core.maxgigapop.net
                                                          (206.196.178.81)
                                                          33.858 ms
                                                          
noaa-rtr.maxgigapop.net
                                                          (206.196.177.118)
                                                          33.483 ms
                                                          
ae0.clpk-core.maxgigapop.net
                                                          (206.196.178.81)
                                                          33.515 ms

                                                          9
                                                          140.90.111.36
                                                          (140.90.111.36)
                                                          33.574 ms
                                                          33.545 ms 
noaa-rtr.maxgigapop.net (206.196.177.118) 33.907 ms

                                                          10
                                                          140.90.76.69
                                                          (140.90.76.69)
                                                          34.220 ms
                                                          34.012 ms
                                                          33.901 ms

                                                          
                                                          

                                                          
                                                          The above
                                                          was taken
                                                          while we were
                                                          running about
                                                          1000 seconds
                                                          behind. A
                                                          recent change
                                                          here
                                                          (9/16/2015)Âwas to
                                                          direct our
                                                          first hop to
                                                          Chicago
                                                          instead ofÂPittsburgh
                                                          (3rox) which
                                                          is now a 100
                                                          Gbit
                                                          link.Â
                                                          Tests to UCAR
                                                          at that time
                                                          were showing
                                                          1.38 Gbps
                                                          throughput
                                                          with reduced
                                                          latencies.Â
                                                          Since
                                                          our
                                                          delays now are
                                                          notÂas
                                                          bad as they
                                                          were
                                                          previously,
                                                          perhaps this
                                                          has helped.Â
                                                          However, there
                                                          may
                                                          still be
                                                          a choke
                                                          pointÂfurther
                                                          down the line
                                                          at maxgigapop
                                                          or internal to
                                                          NCEP itself.
                                                          Âperfsonar
                                                          monitoring
                                                          to NCEP
                                                          would be
                                                          useful... does
                                                          anyone have
                                                          any tests to
                                                          NCEP
                                                          currentlyÂrunning?Â
                                                          Can we
                                                          identify
                                                          any
                                                          common path
                                                          segments in
                                                          the trace
                                                          route above?
                                                          

                                                          
                                                          Â Â Â Â
                                                          Â Â Â Â Â Â Â
                                                          ÂArt
                                                          

                                                          
                                                          
                                                          
                                                          
                                                          From: "Pete
                                                          Pokrandt" 
<poker@xxxxxxxxxxxx>

                                                          To: "Arthur
                                                          A Person" 
<aap1@xxxxxxx>,
                                                          
"support-conduit@xxxxxxxxxxxxxxxx"
                                                          
<conduit@xxxxxxxxxxxxxxxx>

                                                          Cc: "Bentley,
                                                          Alicia M" 
<ambentley@xxxxxxxxxx>,
                                                          "Daes Support"
                                                          
<daessupport@xxxxxxxxxx>,
                                                          "Carissa
                                                          Klemmer - NOAA
                                                          Federal" 
<carissa.l.klemmer@xxxxxxxx>,

                                                          "Tyle, Kevin
                                                          R" <ktyle@xxxxxxxxxx>

                                                          Sent: Thursday,
                                                          September 24,
                                                          2015 1:25:31
                                                          AM
                                                          
                                                          

                                                          Subject: Re:
                                                          [conduit]
                                                          How's your
                                                          GFS?

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          

Art,
                                                          



                                                          
                                                          

Looks like
                                                          you spoke too
                                                          soon. Big lags
                                                          ~1000 secs
                                                          started up
                                                          again with
                                                          today's 12 UTC
                                                          cycle. Very
                                                          mysterious..
                                                          



                                                          
                                                          

They are
                                                          showing up on
                                                          our feed and
                                                          consequently
                                                          downstream
                                                          from us at
                                                          Albny.
                                                          



                                                          
                                                          

http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

                                                          
                                                          

http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+install.atmos.albany.edu

                                                          
                                                          



                                                          
                                                          

<sigh..>
                                                          



                                                          
                                                          

Pete
                                                          



                                                          
                                                          



                                                          
                                                          
                                                          
                                                          

--

                                                          Pete Pokrandt
                                                          - Systems
                                                          Programmer

                                                          UW-Madison
                                                          Dept of
                                                          Atmospheric
                                                          and Oceanic
                                                          Sciences

                                                          608-262-3086Â -
                                                          poker@xxxxxxxxxxxx

                                                          

                                                          
                                                          
                                                          
                                                          

                                                          

                                                          
                                                          
                                                          From:
                                                          Arthur A
                                                          Person <aap1@xxxxxxx>

                                                          Sent:
                                                          Monday,
                                                          September 21,
                                                          2015 10:14 AM

                                                          To: 
support-conduit@xxxxxxxxxxxxxxxx

                                                          Cc:
                                                          Bentley,
                                                          Alicia M; Daes
                                                          Support;
                                                          Carissa
                                                          Klemmer - NOAA
                                                          Federal; Pete
                                                          Pokrandt;
                                                          Tyle, Kevin R

                                                          Subject:
                                                          Re: [conduit]
                                                          How's your
                                                          GFS?
                                                          Â
                                                          
                                                          
                                                          
                                                          Folks,
                                                          

                                                          
                                                          
                                                          
                                                          Looks like
                                                          something
                                                          changed late
                                                          onÂFriday
                                                          in the network
                                                          paths
                                                          affecting Penn
                                                          State and the
                                                          other
                                                          universities
                                                          feeding
                                                          
                                                          CONDUIT from
                                                          NCEP... delays
                                                          have dropped
                                                          to a crisp 30
                                                          seconds or
                                                          less. Does
                                                          anyone know if
                                                          a problem was
                                                          found/fixed?
                                                          
                                                          I know some
                                                          issues were
                                                          addressed at
                                                          Penn State
                                                          with some
                                                          issues still
                                                          being
                                                          worked-on.Â
                                                          Back in the
                                                          first week of
                                                          
                                                          September the
                                                          feeds were
                                                          good and then
                                                          degraded...
                                                          just want to
                                                          make sure that
                                                          doesn't happen
                                                          again before I
                                                          re-enable
                                                          
                                                          ingest of the
                                                          gfs 0.25
                                                          degree data.
                                                          
                                                          

                                                          
                                                          
                                                          Â Â Â Â Â Â Â
                                                          Â Â Â Â Â Art
                                                          
                                                          

                                                          
                                                          
                                                          
                                                          
                                                          From: "Arthur
                                                          A Person" 
<aap1@xxxxxxx>

                                                          To: 
"support-conduit@xxxxxxxxxxxxxxxx"
                                                          
<conduit@xxxxxxxxxxxxxxxx>

                                                          Cc: "Bentley,
                                                          Alicia M" 
<ambentley@xxxxxxxxxx>,
                                                          "Daes Support"
                                                          
<daessupport@xxxxxxxxxx>,
                                                          "Carissa
                                                          Klemmer - NOAA
                                                          Federal" 
<carissa.l.klemmer@xxxxxxxx>

                                                          Sent: Wednesday,
                                                          September 9,
                                                          2015 4:27:26
                                                          PM

                                                          Subject: Re:
                                                          [conduit]
                                                          How's your
                                                          GFS?

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          Just a
                                                          heads up...
                                                          

                                                          I've
                                                          reconfigured
                                                          our IDD relay
                                                          to distribute
                                                          conduit
                                                          without the
                                                          gfs 0.25
                                                          degree data
                                                          until we get
                                                          our latencies
                                                          under
                                                          control.Â
                                                          We've got some
                                                          issues
                                                          internal to
                                                          Penn State
                                                          creating
                                                          problems on
                                                          top of any
                                                          external
                                                          issues and our
                                                          conduit feed
                                                          is useless the
                                                          way it is at
                                                          the moment.Â
                                                          By reverting
                                                          to conduit
                                                          without gfs
                                                          0.25,
                                                          hopefully
                                                          we'll maintain
                                                          a useful
                                                          stream. As
                                                          soon as the
                                                          latencies are
                                                          addressed, I
                                                          will
                                                          reintroduce
                                                          the gfs 0.25.
                                                          

                                                          Â Â Â Â Â
                                                          Â Â Â Â ÂArt
                                                          

                                                          
                                                          
                                                          
                                                          From: "Arthur
                                                          A Person" 
<aap1@xxxxxxx>

                                                          To: "Carissa
                                                          Klemmer - NOAA
                                                          Federal" 
<carissa.l.klemmer@xxxxxxxx>

                                                          Cc: "Bentley,
                                                          Alicia M" 
<ambentley@xxxxxxxxxx>,
                                                          "Daes Support"
                                                          
<daessupport@xxxxxxxxxx>,
                                                          
"support-conduit@xxxxxxxxxxxxxxxx"
                                                          
<conduit@xxxxxxxxxxxxxxxx>

                                                          Sent: Wednesday,
                                                          September 9,
                                                          2015 8:00:19
                                                          AM

                                                          Subject: Re:
                                                          [conduit]
                                                          How's your
                                                          GFS?

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          All,
                                                          

                                                          
                                                          
                                                          
                                                          From: "Carissa
                                                          Klemmer - NOAA
                                                          Federal" 
<carissa.l.klemmer@xxxxxxxx>

                                                          To: "Arthur
                                                          A Person" 
<aap1@xxxxxxx>

                                                          Cc: "Tyle,
                                                          Kevin R" 
<ktyle@xxxxxxxxxx>,
                                                          "Bentley,
                                                          Alicia M" 
<ambentley@xxxxxxxxxx>,
                                                          "Daes Support"
                                                          
<daessupport@xxxxxxxxxx>,

                                                          
"support-conduit@xxxxxxxxxxxxxxxx"
                                                          
<conduit@xxxxxxxxxxxxxxxx>

                                                          Sent: Tuesday,
                                                          September 8,
                                                          2015 10:19:06
                                                          PM

                                                          Subject: Re:
                                                          [conduit]
                                                          How's your
                                                          GFS?

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          All,Â
                                                          

                                                          
                                                          NCEP
                                                          is not making
                                                          any active
                                                          changes to our
                                                          networks that
                                                          should affect
                                                          your
                                                          latencies,
                                                          especially not
                                                          over a
                                                          weekend. I am
                                                          not aware of
                                                          any changes
                                                          that occurred
                                                          over the
                                                          holiday that
                                                          would have
                                                          impacted these
                                                          networks. This
                                                          is likely
                                                          downstream of
                                                          NCEP control
                                                          which is why
                                                          you see the
                                                          latencies come
                                                          and go.Â
                                                          
                                                          
                                                          Okay... I
                                                          guess my
                                                          interpretation
                                                          was wrong,
                                                          then. My
                                                          apologies.Â
                                                          There does
                                                          seem to be a
                                                          problem pretty
                                                          close to NCEP,
                                                          however, since
                                                          the latencies
                                                          seem to come
                                                          and go at all
                                                          top-tier
                                                          sites...
                                                          although not
                                                          all sites are
                                                          the same (ours
                                                          seems to be
                                                          the highest).Â
                                                          Maybe we're
                                                          pushing the
                                                          long-haul
                                                          connectivity
                                                          to the limit
                                                          and multiple
                                                          choke points
                                                          are showing
                                                          up? Time to
                                                          get our
                                                          networking
                                                          folks more
                                                          involved...
                                                          

                                                          Â Â Â Â Â
                                                          Â Â Â Â Â Â Â
                                                          Â ÂArt
                                                          
                                                          
                                                          

                                                          
                                                          

                                                          
                                                          
                                                          

                                                          
                                                          Carissa
                                                          Klemmer
                                                          

                                                          NCEP
                                                          Central
                                                          Operations
                                                          

                                                          Production
                                                          Management
                                                          Branch
                                                          Dataflow Team
                                                          

                                                          301-683-3835

                                                          
                                                          
                                                          

                                                          On
                                                          Tue, Sep 8,
                                                          2015 at 1:21
                                                          PM, Arthur A
                                                          Person 
                                                          <aap1@xxxxxxx>
                                                          wrote:

                                                          
                                                          
                                                          
                                                          We appear
                                                          to have had
                                                          gfs reception
                                                          problems with
                                                          0Z and 6Z runs
                                                          last night.Â
                                                          After
                                                          implementation
                                                          of the
                                                          0.25 degree
                                                          gfs,
                                                          CONDUITÂlatencies
                                                          were very
                                                          large across
                                                          all sites
                                                          during 0.25
                                                          degree data
                                                          transmission,Âbut
                                                          a week-or-so
                                                          ago dropped to
                                                          negligible
                                                          levels. Over
                                                          the weekend
                                                          they jumped
                                                          back up
                                                          again. I
                                                          interpret this
                                                          to mean NCEP
                                                          is tinkering
                                                          with network
                                                          paths trying
                                                          to find an
                                                          effective
                                                          way to
                                                          getÂthese
                                                          hugeÂbursts
                                                          of data out to
                                                          the downstream
                                                          sites. The
                                                          gfs data loss
                                                          last night may
                                                          have
                                                          been from
                                                          theÂlarge
                                                          latencies or
                                                          from other
                                                          unrelated
                                                          delivery
                                                          problems...
                                                          dunno...
                                                          

                                                          Â Â Â Â Â
                                                          Â Â Â Â Â Â
                                                          Art
                                                          

                                                          
                                                          
                                                          
                                                          From: "Tyle,
                                                          Kevin R" 
<ktyle@xxxxxxxxxx>

                                                          To: "Pete
                                                          Pokrandt" 
<poker@xxxxxxxxxxxx>

                                                          Cc: "Bentley,
                                                          Alicia M" 
<ambentley@xxxxxxxxxx>,
                                                          "Daes Support"
                                                          
<daessupport@xxxxxxxxxx>,
                                                          
conduit@xxxxxxxxxxxxxxxx

                                                          Sent: Tuesday,
                                                          September 8,
                                                          2015 1:00:10
                                                          PM

                                                          Subject: Re:
                                                          [conduit]
                                                          How's your
                                                          GFS?

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          

Hi
                                                          Pete, et al.:
                                                          

Â
                                                          

We
                                                          here at
                                                          UAlbany
                                                          continue to
                                                          get spotty
                                                          reception of
                                                          the GFS since
                                                          00Z today â
                                                          anyone else
                                                          having
                                                          issues? We
                                                          feed from
                                                          Madison and
                                                          State College.
                                                          

Â
                                                          

Earlier
                                                          thread below:
                                                          

Â
                                                          

-------------------------
                                                          

Yeah,
                                                          Iâm not
                                                          surprised that
                                                          the addition
                                                          of the  deg
                                                          GFS is causing
                                                          the need for a
                                                          bigger queue
                                                          (and likely
                                                          burlier
                                                          machine).Â
                                                          Thatâs the
                                                          main reason I
                                                          have resisted
                                                          requesting it.
                                                          

Â
                                                          

Iâll
                                                          fix the issue
                                                          that makes
                                                          ldmstats show
                                                          âinstall.atmosââ
                                                          instead of
                                                          âcascade.atmosââ
                                                          

Â
                                                          

Something
                                                          else must be
                                                          at play since
                                                          the  GFS has
                                                          been flowing
                                                          for several
                                                          weeks now
                                                          without
                                                          incident,
                                                          likely tied to
                                                          the increased
                                                          latency you
                                                          starting
                                                          seeing.
                                                          

Â
                                                          

Looks
                                                          like we only
                                                          got the GFS
                                                          through 60
                                                          hours today
                                                          with the 12Z
                                                          run, so
                                                          something
                                                          definitely
                                                          appears to be
                                                          amiss â Iâll
                                                          cc: the
                                                          conduit list
                                                          to see if
                                                          anyone else is
                                                          noticing
                                                          problems.
                                                          

Â
                                                          
                                                          

_____________________________________________

                                                          Kevin Tyle,
                                                          Systems
                                                          Administrator
                                                          

                                                          Dept. of
                                                          Atmospheric
                                                          &
                                                          Environmental
                                                          SciencesÂÂ 

                                                          University at
                                                          Albany

                                                          Earth Science
                                                          235, 1400
                                                          Washington
                                                          AvenueÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
ÂÂÂÂÂÂÂ
                                                          

                                                          Albany, NY
                                                          12222

                                                          Email: 
ktyle@xxxxxxxxxx

                                                          Phone: 518-442-4578ÂÂÂ
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
                                                          

_____________________________________________
                                                          
                                                          

Â
                                                          
                                                          
                                                          

From:
                                                          Pete Pokrandt
                                                          
[mailto:poker@xxxxxxxxxxxx]
                                                          

                                                          Sent:
                                                          Tuesday,
                                                          September 08,
                                                          2015 12:17 PM

                                                          To:
                                                          Tyle, Kevin R
                                                          <ktyle@xxxxxxxxxx>

                                                          Cc:
                                                          Daes Support
                                                          
<daessupport@xxxxxxxxxx>

                                                          Subject:
                                                          Re: How's your
                                                          GFS?
                                                          
                                                          
                                                          
                                                          

My
                                                          GFS appears to
                                                          be complete,
                                                          but I do see
                                                          that
                                                          something's
                                                          going on with
                                                          our feed - the
                                                          latencies
                                                          jumped way up
                                                          somewhere over
                                                          the weekend:

                                                          

                                                          
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

                                                          

                                                          You're seeing
                                                          the same, and
                                                          increased yet
                                                          from your feed
                                                          from Penn
                                                          State (at
                                                          least to the
                                                          machine 
'install.atmos.albany.edu
                                                          - I don't see
                                                          any stats
                                                          reported from
                                                          cascade, which
                                                          is what it
                                                          looks like you
                                                          are feeding
                                                          from me on)

                                                          

                                                          
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+install.atmos.albany.edu

                                                          

                                                          I think I need
                                                          to buy more
                                                          memory and
                                                          keep a larger
                                                          queue on 
idd.aos.wisc.edu with the 0.25 deg GFS coming in. There are times
                                                          where my queue
                                                          only holds
                                                          about 20
                                                          minutes of
                                                          data, which is
                                                          likely
                                                          contributing
                                                          to your
                                                          incomplete GFS
                                                          files..

                                                          

                                                          Here's what my
                                                          0.5 deg (the
                                                          gblav2.*
                                                          files) and the
                                                          0.25 deg
                                                          (gblav0p25 out
                                                          to 87 h) look
                                                          like for the
                                                          00 ant 06 UTC
                                                          runs today

                                                          

                                                          

                                                          0.5 deg:

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          60953687 SepÂ
                                                          7 22:25
                                                          
/data/grib2/gblav2.15090800_F000

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          66996066 SepÂ
                                                          7 22:28
                                                          
/data/grib2/gblav2.15090800_F003

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          67902041 SepÂ
                                                          7 22:30
                                                          
/data/grib2/gblav2.15090800_F006

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          67961293 SepÂ
                                                          7 22:32
                                                          
/data/grib2/gblav2.15090800_F009

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68081826 SepÂ
                                                          7 22:35
                                                          
/data/grib2/gblav2.15090800_F012

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68710398 SepÂ
                                                          7 22:35
                                                          
/data/grib2/gblav2.15090800_F015

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69664268 SepÂ
                                                          7 22:36
                                                          
/data/grib2/gblav2.15090800_F018

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69177180 SepÂ
                                                          7 22:38
                                                          
/data/grib2/gblav2.15090800_F021

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69816235 SepÂ
                                                          7 22:38
                                                          
/data/grib2/gblav2.15090800_F024

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69010253 SepÂ
                                                          7 22:39
                                                          
/data/grib2/gblav2.15090800_F027

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69786985 SepÂ
                                                          7 22:40
                                                          
/data/grib2/gblav2.15090800_F030

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68876266 SepÂ
                                                          7 22:41
                                                          
/data/grib2/gblav2.15090800_F033

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69376601 SepÂ
                                                          7 22:42
                                                          
/data/grib2/gblav2.15090800_F036

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69029846 SepÂ
                                                          7 22:43
                                                          
/data/grib2/gblav2.15090800_F039

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69142392 SepÂ
                                                          7 22:44
                                                          
/data/grib2/gblav2.15090800_F042

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68990399 SepÂ
                                                          7 22:45
                                                          
/data/grib2/gblav2.15090800_F045

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69343366 SepÂ
                                                          7 22:46
                                                          
/data/grib2/gblav2.15090800_F048

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69150894 SepÂ
                                                          7 22:47
                                                          
/data/grib2/gblav2.15090800_F051

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69504675 SepÂ
                                                          7 22:47
                                                          
/data/grib2/gblav2.15090800_F054

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69196832 SepÂ
                                                          7 22:48
                                                          
/data/grib2/gblav2.15090800_F057

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69335487 SepÂ
                                                          7 22:50
                                                          
/data/grib2/gblav2.15090800_F060

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69261676 SepÂ
                                                          7 22:50
                                                          
/data/grib2/gblav2.15090800_F063

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69166068 SepÂ
                                                          7 22:51
                                                          
/data/grib2/gblav2.15090800_F066

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69054105 SepÂ
                                                          7 22:53
                                                          
/data/grib2/gblav2.15090800_F069

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68895264 SepÂ
                                                          7 22:54
                                                          
/data/grib2/gblav2.15090800_F072

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69202038 SepÂ
                                                          7 22:56
                                                          
/data/grib2/gblav2.15090800_F075

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69339334 SepÂ
                                                          7 22:56
                                                          
/data/grib2/gblav2.15090800_F078

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69181930 SepÂ
                                                          7 22:57
                                                          
/data/grib2/gblav2.15090800_F081

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69674148 SepÂ
                                                          7 22:58
                                                          
/data/grib2/gblav2.15090800_F084

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69383769 SepÂ
                                                          7 22:58
                                                          
/data/grib2/gblav2.15090800_F087

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69645526 SepÂ
                                                          7 22:59
                                                          
/data/grib2/gblav2.15090800_F090

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69119323 SepÂ
                                                          7 23:00
                                                          
/data/grib2/gblav2.15090800_F093

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69363296 SepÂ
                                                          7 23:01
                                                          
/data/grib2/gblav2.15090800_F096

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69030287 SepÂ
                                                          7 23:03
                                                          
/data/grib2/gblav2.15090800_F099

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69819322 SepÂ
                                                          7 23:03
                                                          
/data/grib2/gblav2.15090800_F102

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69498561 SepÂ
                                                          7 23:04
                                                          
/data/grib2/gblav2.15090800_F105

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69690447 SepÂ
                                                          7 23:05
                                                          
/data/grib2/gblav2.15090800_F108

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69274213 SepÂ
                                                          7 23:06
                                                          
/data/grib2/gblav2.15090800_F111

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70089206 SepÂ
                                                          7 23:07
                                                          
/data/grib2/gblav2.15090800_F114

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70007688 SepÂ
                                                          7 23:08
                                                          
/data/grib2/gblav2.15090800_F117

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70237308 SepÂ
                                                          7 23:08
                                                          
/data/grib2/gblav2.15090800_F120

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69849708 SepÂ
                                                          7 23:09
                                                          
/data/grib2/gblav2.15090800_F123

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69883550 SepÂ
                                                          7 23:11
                                                          
/data/grib2/gblav2.15090800_F126

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69586365 SepÂ
                                                          7 23:11
                                                          
/data/grib2/gblav2.15090800_F129

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70110782 SepÂ
                                                          7 23:12
                                                          
/data/grib2/gblav2.15090800_F132

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69430545 SepÂ
                                                          7 23:13
                                                          
/data/grib2/gblav2.15090800_F135

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69461630 SepÂ
                                                          7 23:14
                                                          
/data/grib2/gblav2.15090800_F138

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69264487 SepÂ
                                                          7 23:15
                                                          
/data/grib2/gblav2.15090800_F141

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69553206 SepÂ
                                                          7 23:16
                                                          
/data/grib2/gblav2.15090800_F144

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68924371 SepÂ
                                                          7 23:17
                                                          
/data/grib2/gblav2.15090800_F147

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69191965 SepÂ
                                                          7 23:17
                                                          
/data/grib2/gblav2.15090800_F150

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68639462 SepÂ
                                                          7 23:19
                                                          
/data/grib2/gblav2.15090800_F153

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69035706 SepÂ
                                                          7 23:22
                                                          
/data/grib2/gblav2.15090800_F156

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68831618 SepÂ
                                                          7 23:25
                                                          
/data/grib2/gblav2.15090800_F159

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69428952 SepÂ
                                                          7 23:27
                                                          
/data/grib2/gblav2.15090800_F162

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69514672 SepÂ
                                                          7 23:28
                                                          
/data/grib2/gblav2.15090800_F165

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69614097 SepÂ
                                                          7 23:29
                                                          
/data/grib2/gblav2.15090800_F168

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69404524 SepÂ
                                                          7 23:29
                                                          
/data/grib2/gblav2.15090800_F171

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69534566 SepÂ
                                                          7 23:30
                                                          
/data/grib2/gblav2.15090800_F174

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69528455 SepÂ
                                                          7 23:31
                                                          
/data/grib2/gblav2.15090800_F177

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69747643 SepÂ
                                                          7 23:31
                                                          
/data/grib2/gblav2.15090800_F180

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69397125 SepÂ
                                                          7 23:32
                                                          
/data/grib2/gblav2.15090800_F183

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69973323 SepÂ
                                                          7 23:32
                                                          
/data/grib2/gblav2.15090800_F186

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69070113 SepÂ
                                                          7 23:33
                                                          
/data/grib2/gblav2.15090800_F189

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69586837 SepÂ
                                                          7 23:34
                                                          
/data/grib2/gblav2.15090800_F192

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69202267 SepÂ
                                                          7 23:34
                                                          
/data/grib2/gblav2.15090800_F195

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69169373 SepÂ
                                                          7 23:35
                                                          
/data/grib2/gblav2.15090800_F198

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68193948 SepÂ
                                                          7 23:36
                                                          
/data/grib2/gblav2.15090800_F201

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          67963148 SepÂ
                                                          7 23:38
                                                          
/data/grib2/gblav2.15090800_F204

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          67689203 SepÂ
                                                          7 23:39
                                                          
/data/grib2/gblav2.15090800_F207

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68079977 SepÂ
                                                          7 23:41
                                                          
/data/grib2/gblav2.15090800_F210

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68931672 SepÂ
                                                          7 23:43
                                                          
/data/grib2/gblav2.15090800_F213

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68749459 SepÂ
                                                          7 23:46
                                                          
/data/grib2/gblav2.15090800_F216

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68739072 SepÂ
                                                          7 23:46
                                                          
/data/grib2/gblav2.15090800_F219

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68789427 SepÂ
                                                          7 23:47
                                                          
/data/grib2/gblav2.15090800_F222

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68031035 SepÂ
                                                          7 23:48
                                                          
/data/grib2/gblav2.15090800_F225

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68735199 SepÂ
                                                          7 23:48
                                                          
/data/grib2/gblav2.15090800_F228

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          65347330 SepÂ
                                                          7 23:49
                                                          
/data/grib2/gblav2.15090800_F231

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          65891902 SepÂ
                                                          7 23:49
                                                          
/data/grib2/gblav2.15090800_F234

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          65383729 SepÂ
                                                          7 23:50
                                                          
/data/grib2/gblav2.15090800_F237

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          66299227 SepÂ
                                                          7 23:50
                                                          
/data/grib2/gblav2.15090800_F240

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64525715 SepÂ
                                                          7 23:52
                                                          
/data/grib2/gblav2.15090800_F252

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64515690 SepÂ
                                                          7 23:53
                                                          
/data/grib2/gblav2.15090800_F264

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          63803271 SepÂ
                                                          7 23:53
                                                          
/data/grib2/gblav2.15090800_F276

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          63261621 SepÂ
                                                          7 23:54
                                                          
/data/grib2/gblav2.15090800_F288

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64171542 SepÂ
                                                          7 23:54
                                                          
/data/grib2/gblav2.15090800_F300

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64308576 SepÂ
                                                          7 23:56
                                                          
/data/grib2/gblav2.15090800_F312

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64334459 SepÂ
                                                          7 23:58
                                                          
/data/grib2/gblav2.15090800_F324

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64189700 SepÂ
                                                          7 23:59
                                                          
/data/grib2/gblav2.15090800_F336

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          63829248 SepÂ
                                                          7 23:59
                                                          
/data/grib2/gblav2.15090800_F348

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64655803 SepÂ
                                                          8 00:00
                                                          
/data/grib2/gblav2.15090800_F360

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64436657 SepÂ
                                                          8 00:07
                                                          
/data/grib2/gblav2.15090800_F372

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64546095 SepÂ
                                                          8 00:12
                                                          
/data/grib2/gblav2.15090800_F384

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          61169101 SepÂ
                                                          8 04:26
                                                          
/data/grib2/gblav2.15090806_F000

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          67422108 SepÂ
                                                          8 04:28
                                                          
/data/grib2/gblav2.15090806_F003

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68374534 SepÂ
                                                          8 04:31
                                                          
/data/grib2/gblav2.15090806_F006

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68543418 SepÂ
                                                          8 04:33
                                                          
/data/grib2/gblav2.15090806_F009

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69298218 SepÂ
                                                          8 04:35
                                                          
/data/grib2/gblav2.15090806_F012

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69188133 SepÂ
                                                          8 04:36
                                                          
/data/grib2/gblav2.15090806_F015

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69917655 SepÂ
                                                          8 04:37
                                                          
/data/grib2/gblav2.15090806_F018

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69558566 SepÂ
                                                          8 04:38
                                                          
/data/grib2/gblav2.15090806_F021

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69659459 SepÂ
                                                          8 04:38
                                                          
/data/grib2/gblav2.15090806_F024

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69288102 SepÂ
                                                          8 04:40
                                                          
/data/grib2/gblav2.15090806_F027

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68686968 SepÂ
                                                          8 04:40
                                                          
/data/grib2/gblav2.15090806_F030

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68640234 SepÂ
                                                          8 04:42
                                                          
/data/grib2/gblav2.15090806_F033

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69544506 SepÂ
                                                          8 04:42
                                                          
/data/grib2/gblav2.15090806_F036

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68462036 SepÂ
                                                          8 04:43
                                                          
/data/grib2/gblav2.15090806_F039

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69287354 SepÂ
                                                          8 04:44
                                                          
/data/grib2/gblav2.15090806_F042

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69228412 SepÂ
                                                          8 04:45
                                                          
/data/grib2/gblav2.15090806_F045

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69444769 SepÂ
                                                          8 04:46
                                                          
/data/grib2/gblav2.15090806_F048

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69089036 SepÂ
                                                          8 04:47
                                                          
/data/grib2/gblav2.15090806_F051

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69542812 SepÂ
                                                          8 04:48
                                                          
/data/grib2/gblav2.15090806_F054

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69377775 SepÂ
                                                          8 04:49
                                                          
/data/grib2/gblav2.15090806_F057

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69324867 SepÂ
                                                          8 04:50
                                                          
/data/grib2/gblav2.15090806_F060

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69313464 SepÂ
                                                          8 04:51
                                                          
/data/grib2/gblav2.15090806_F063

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69820155 SepÂ
                                                          8 04:52
                                                          
/data/grib2/gblav2.15090806_F066

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69484687 SepÂ
                                                          8 04:52
                                                          
/data/grib2/gblav2.15090806_F069

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69581997 SepÂ
                                                          8 04:53
                                                          
/data/grib2/gblav2.15090806_F072

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69189693 SepÂ
                                                          8 04:54
                                                          
/data/grib2/gblav2.15090806_F075

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69751906 SepÂ
                                                          8 04:55
                                                          
/data/grib2/gblav2.15090806_F078

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69558875 SepÂ
                                                          8 04:56
                                                          
/data/grib2/gblav2.15090806_F081

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69903084 SepÂ
                                                          8 04:58
                                                          
/data/grib2/gblav2.15090806_F084

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69627748 SepÂ
                                                          8 04:59
                                                          
/data/grib2/gblav2.15090806_F087

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69678696 SepÂ
                                                          8 04:59
                                                          
/data/grib2/gblav2.15090806_F090

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69497446 SepÂ
                                                          8 05:00
                                                          
/data/grib2/gblav2.15090806_F093

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69735442 SepÂ
                                                          8 05:01
                                                          
/data/grib2/gblav2.15090806_F096

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69767861 SepÂ
                                                          8 05:02
                                                          
/data/grib2/gblav2.15090806_F099

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70169785 SepÂ
                                                          8 05:03
                                                          
/data/grib2/gblav2.15090806_F102

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69625644 SepÂ
                                                          8 05:04
                                                          
/data/grib2/gblav2.15090806_F105

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69954293 SepÂ
                                                          8 05:05
                                                          
/data/grib2/gblav2.15090806_F108

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69996186 SepÂ
                                                          8 05:06
                                                          
/data/grib2/gblav2.15090806_F111

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70297897 SepÂ
                                                          8 05:06
                                                          
/data/grib2/gblav2.15090806_F114

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70037957 SepÂ
                                                          8 05:08
                                                          
/data/grib2/gblav2.15090806_F117

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69968183 SepÂ
                                                          8 05:08
                                                          
/data/grib2/gblav2.15090806_F120

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69564905 SepÂ
                                                          8 05:10
                                                          
/data/grib2/gblav2.15090806_F123

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69725865 SepÂ
                                                          8 05:11
                                                          
/data/grib2/gblav2.15090806_F126

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69349475 SepÂ
                                                          8 05:11
                                                          
/data/grib2/gblav2.15090806_F129

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69625604 SepÂ
                                                          8 05:12
                                                          
/data/grib2/gblav2.15090806_F132

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69392152 SepÂ
                                                          8 05:15
                                                          
/data/grib2/gblav2.15090806_F135

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69551134 SepÂ
                                                          8 05:18
                                                          
/data/grib2/gblav2.15090806_F138

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69108820 SepÂ
                                                          8 05:19
                                                          
/data/grib2/gblav2.15090806_F141

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69469618 SepÂ
                                                          8 05:19
                                                          
/data/grib2/gblav2.15090806_F144

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          68774645 SepÂ
                                                          8 05:20
                                                          
/data/grib2/gblav2.15090806_F147

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69135260 SepÂ
                                                          8 05:20
                                                          
/data/grib2/gblav2.15090806_F150

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69009857 SepÂ
                                                          8 05:21
                                                          
/data/grib2/gblav2.15090806_F153

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69647753 SepÂ
                                                          8 05:21
                                                          
/data/grib2/gblav2.15090806_F156

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69604259 SepÂ
                                                          8 05:22
                                                          
/data/grib2/gblav2.15090806_F159

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69851358 SepÂ
                                                          8 05:22
                                                          
/data/grib2/gblav2.15090806_F162

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69621423 SepÂ
                                                          8 05:23
                                                          
/data/grib2/gblav2.15090806_F165

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69987289 SepÂ
                                                          8 05:24
                                                          
/data/grib2/gblav2.15090806_F168

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70009168 SepÂ
                                                          8 05:24
                                                          
/data/grib2/gblav2.15090806_F171

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70272431 SepÂ
                                                          8 05:25
                                                          
/data/grib2/gblav2.15090806_F174

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69951044 SepÂ
                                                          8 05:26
                                                          
/data/grib2/gblav2.15090806_F177

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70294466 SepÂ
                                                          8 05:28
                                                          
/data/grib2/gblav2.15090806_F180

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69693077 SepÂ
                                                          8 05:31
                                                          
/data/grib2/gblav2.15090806_F183

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70277595 SepÂ
                                                          8 05:35
                                                          
/data/grib2/gblav2.15090806_F186

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70161497 SepÂ
                                                          8 05:36
                                                          
/data/grib2/gblav2.15090806_F189

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70075264 SepÂ
                                                          8 05:37
                                                          
/data/grib2/gblav2.15090806_F192

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69929971 SepÂ
                                                          8 05:37
                                                          
/data/grib2/gblav2.15090806_F195

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69879151 SepÂ
                                                          8 05:38
                                                          
/data/grib2/gblav2.15090806_F198

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69726455 SepÂ
                                                          8 05:38
                                                          
/data/grib2/gblav2.15090806_F201

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70186834 SepÂ
                                                          8 05:38
                                                          
/data/grib2/gblav2.15090806_F204

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69735649 SepÂ
                                                          8 05:39
                                                          
/data/grib2/gblav2.15090806_F207

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70062469 SepÂ
                                                          8 05:40
                                                          
/data/grib2/gblav2.15090806_F210

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69475211 SepÂ
                                                          8 05:40
                                                          
/data/grib2/gblav2.15090806_F213

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69688060 SepÂ
                                                          8 05:41
                                                          
/data/grib2/gblav2.15090806_F216

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69169089 SepÂ
                                                          8 05:42
                                                          
/data/grib2/gblav2.15090806_F219

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69623322 SepÂ
                                                          8 05:42
                                                          
/data/grib2/gblav2.15090806_F222

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69434126 SepÂ
                                                          8 05:43
                                                          
/data/grib2/gblav2.15090806_F225

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69447710 SepÂ
                                                          8 05:44
                                                          
/data/grib2/gblav2.15090806_F228

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69232930 SepÂ
                                                          8 05:44
                                                          
/data/grib2/gblav2.15090806_F231

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69688395 SepÂ
                                                          8 05:45
                                                          
/data/grib2/gblav2.15090806_F234

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          69476983 SepÂ
                                                          8 05:47
                                                          
/data/grib2/gblav2.15090806_F237

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          70027781 SepÂ
                                                          8 05:50
                                                          
/data/grib2/gblav2.15090806_F240

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64748968 SepÂ
                                                          8 05:52
                                                          
/data/grib2/gblav2.15090806_F252

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64729059 SepÂ
                                                          8 05:53
                                                          
/data/grib2/gblav2.15090806_F264

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64211460 SepÂ
                                                          8 05:53
                                                          
/data/grib2/gblav2.15090806_F276

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64117374 SepÂ
                                                          8 05:54
                                                          
/data/grib2/gblav2.15090806_F288

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64123032 SepÂ
                                                          8 05:54
                                                          
/data/grib2/gblav2.15090806_F300

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64714736 SepÂ
                                                          8 05:56
                                                          
/data/grib2/gblav2.15090806_F312

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          65052210 SepÂ
                                                          8 05:56
                                                          
/data/grib2/gblav2.15090806_F324

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          65123631 SepÂ
                                                          8 05:57
                                                          
/data/grib2/gblav2.15090806_F336

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64903451 SepÂ
                                                          8 05:58
                                                          
/data/grib2/gblav2.15090806_F348

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64423290 SepÂ
                                                          8 05:59
                                                          
/data/grib2/gblav2.15090806_F360

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          64365594 SepÂ
                                                          8 05:59
                                                          
/data/grib2/gblav2.15090806_F372

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          63855749 SepÂ
                                                          8 06:07
                                                          
/data/grib2/gblav2.15090806_F384

                                                          

                                                          0.25 deg

                                                          

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          180752345 SepÂ
                                                          7 22:25
                                                          
/data/grib2/gblav0p25.15090800_Fanl

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          197882387 SepÂ
                                                          7 22:26
                                                          
/data/grib2/gblav0p25.15090800_F000

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          217304897 SepÂ
                                                          7 22:27
                                                          
/data/grib2/gblav0p25.15090800_F003

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          221447144 SepÂ
                                                          7 22:30
                                                          
/data/grib2/gblav0p25.15090800_F006

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          221383770 SepÂ
                                                          7 22:32
                                                          
/data/grib2/gblav0p25.15090800_F009

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          222748480 SepÂ
                                                          7 22:34
                                                          
/data/grib2/gblav0p25.15090800_F012

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224209489 SepÂ
                                                          7 22:35
                                                          
/data/grib2/gblav0p25.15090800_F015

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226360332 SepÂ
                                                          7 22:36
                                                          
/data/grib2/gblav0p25.15090800_F018

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225185199 SepÂ
                                                          7 22:37
                                                          
/data/grib2/gblav0p25.15090800_F021

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226720828 SepÂ
                                                          7 22:38
                                                          
/data/grib2/gblav0p25.15090800_F024

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224211990 SepÂ
                                                          7 22:38
                                                          
/data/grib2/gblav0p25.15090800_F027

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226623368 SepÂ
                                                          7 22:40
                                                          
/data/grib2/gblav0p25.15090800_F030

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224601041 SepÂ
                                                          7 22:41
                                                          
/data/grib2/gblav0p25.15090800_F033

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225696377 SepÂ
                                                          7 22:41
                                                          
/data/grib2/gblav0p25.15090800_F036

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224803488 SepÂ
                                                          7 22:43
                                                          
/data/grib2/gblav0p25.15090800_F039

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225463303 SepÂ
                                                          7 22:44
                                                          
/data/grib2/gblav0p25.15090800_F042

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224172234 SepÂ
                                                          7 22:44
                                                          
/data/grib2/gblav0p25.15090800_F045

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225750651 SepÂ
                                                          7 22:45
                                                          
/data/grib2/gblav0p25.15090800_F048

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224513834 SepÂ
                                                          7 22:46
                                                          
/data/grib2/gblav0p25.15090800_F051

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225871134 SepÂ
                                                          7 22:47
                                                          
/data/grib2/gblav0p25.15090800_F054

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224871484 SepÂ
                                                          7 22:48
                                                          
/data/grib2/gblav0p25.15090800_F057

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225954437 SepÂ
                                                          7 22:50
                                                          
/data/grib2/gblav0p25.15090800_F060

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225600052 SepÂ
                                                          7 22:50
                                                          
/data/grib2/gblav0p25.15090800_F063

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225672348 SepÂ
                                                          7 22:51
                                                          
/data/grib2/gblav0p25.15090800_F066

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225064451 SepÂ
                                                          7 22:52
                                                          
/data/grib2/gblav0p25.15090800_F069

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225318101 SepÂ
                                                          7 22:54
                                                          
/data/grib2/gblav0p25.15090800_F072

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225303961 SepÂ
                                                          7 22:57
                                                          
/data/grib2/gblav0p25.15090800_F075

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226805528 SepÂ
                                                          7 23:03
                                                          
/data/grib2/gblav0p25.15090800_F078

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226187062 SepÂ
                                                          7 23:08
                                                          
/data/grib2/gblav0p25.15090800_F081

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          227313364 SepÂ
                                                          7 23:09
                                                          
/data/grib2/gblav0p25.15090800_F084

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226221831 SepÂ
                                                          7 23:15
                                                          
/data/grib2/gblav0p25.15090800_F087

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          197951753 SepÂ
                                                          8 04:25
                                                          
/data/grib2/gblav0p25.15090806_F000

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          181438882 SepÂ
                                                          8 04:26
                                                          
/data/grib2/gblav0p25.15090806_Fanl

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          218273142 SepÂ
                                                          8 04:28
                                                          
/data/grib2/gblav0p25.15090806_F003

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          222180270 SepÂ
                                                          8 04:30
                                                          
/data/grib2/gblav0p25.15090806_F006

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          222627637 SepÂ
                                                          8 04:32
                                                          
/data/grib2/gblav0p25.15090806_F009

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225440960 SepÂ
                                                          8 04:34
                                                          
/data/grib2/gblav0p25.15090806_F012

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224877734 SepÂ
                                                          8 04:35
                                                          
/data/grib2/gblav0p25.15090806_F015

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226700650 SepÂ
                                                          8 04:36
                                                          
/data/grib2/gblav0p25.15090806_F018

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225325799 SepÂ
                                                          8 04:37
                                                          
/data/grib2/gblav0p25.15090806_F021

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226163438 SepÂ
                                                          8 04:38
                                                          
/data/grib2/gblav0p25.15090806_F024

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225234793 SepÂ
                                                          8 04:39
                                                          
/data/grib2/gblav0p25.15090806_F027

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          224315172 SepÂ
                                                          8 04:40
                                                          
/data/grib2/gblav0p25.15090806_F030

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          223485303 SepÂ
                                                          8 04:41
                                                          
/data/grib2/gblav0p25.15090806_F033

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226101395 SepÂ
                                                          8 04:42
                                                          
/data/grib2/gblav0p25.15090806_F036

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          222880336 SepÂ
                                                          8 04:43
                                                          
/data/grib2/gblav0p25.15090806_F039

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225276943 SepÂ
                                                          8 04:44
                                                          
/data/grib2/gblav0p25.15090806_F042

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225167793 SepÂ
                                                          8 04:45
                                                          
/data/grib2/gblav0p25.15090806_F045

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225771493 SepÂ
                                                          8 04:46
                                                          
/data/grib2/gblav0p25.15090806_F048

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225066649 SepÂ
                                                          8 04:47
                                                          
/data/grib2/gblav0p25.15090806_F051

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225905191 SepÂ
                                                          8 04:47
                                                          
/data/grib2/gblav0p25.15090806_F054

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225706912 SepÂ
                                                          8 04:48
                                                          
/data/grib2/gblav0p25.15090806_F057

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225891555 SepÂ
                                                          8 04:49
                                                          
/data/grib2/gblav0p25.15090806_F060

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225723607 SepÂ
                                                          8 04:50
                                                          
/data/grib2/gblav0p25.15090806_F063

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          227329359 SepÂ
                                                          8 04:51
                                                          
/data/grib2/gblav0p25.15090806_F066

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226381130 SepÂ
                                                          8 04:52
                                                          
/data/grib2/gblav0p25.15090806_F069

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          227000926 SepÂ
                                                          8 04:53
                                                          
/data/grib2/gblav0p25.15090806_F072

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          225483067 SepÂ
                                                          8 04:54
                                                          
/data/grib2/gblav0p25.15090806_F075

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          227295269 SepÂ
                                                          8 04:55
                                                          
/data/grib2/gblav0p25.15090806_F078

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226316715 SepÂ
                                                          8 04:55
                                                          
/data/grib2/gblav0p25.15090806_F081

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          227632093 SepÂ
                                                          8 04:57
                                                          
/data/grib2/gblav0p25.15090806_F084

                                                          -rw-r--r--. 1
                                                          ldm ldm
                                                          226447758 SepÂ
                                                          8 05:01
                                                          
/data/grib2/gblav0p25.15090806_F087

                                                          

                                                          

                                                          

                                                          Pete

                                                          

                                                          

                                                          

                                                          

                                                          On 09/08/2015
                                                          08:43 AM,
                                                          Tyle, Kevin R
                                                          wrote:
                                                          
                                                          
                                                          

Hi
                                                          Pete,
                                                          

Â
                                                          

Weâve
                                                          had incomplete
                                                          GFS the last
                                                          two runs (00
                                                          and 06 UTC
                                                          today) â how
                                                          did things
                                                          look on your
                                                          end?
                                                          

Â
                                                          

Thanks,
                                                          

Â
                                                          

Kevin
                                                          

Â
                                                          
                                                          

_____________________________________________

                                                          Kevin Tyle,
                                                          Systems
                                                          Administrator
                                                          

                                                          Dept. of
                                                          Atmospheric
                                                          &
                                                          Environmental
                                                          SciencesÂÂ 

                                                          University at
                                                          Albany

                                                          Earth Science
                                                          235, 1400
                                                          Washington
                                                          AvenueÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
ÂÂÂÂÂÂÂ
                                                          

                                                          Albany, NY
                                                          12222

                                                          Email: 
ktyle@xxxxxxxxxx

                                                          Phone: 518-442-4578ÂÂÂ
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
                                                          

_____________________________________________
                                                          
                                                          
                                                          



                                                          

                                                          

                                                          
                                                          -- 
                                                          Pete Pokrandt - 
Systems Programmer
                                                          UW-Madison Dept of 
Atmospheric and Oceanic Sciences
                                                          608-262-3086Â - 
poker@xxxxxxxxxxxx

                                                          
                                                          

                                                          
                                                          
_______________________________________________

                                                          conduit
                                                          mailing list

                                                          
conduit@xxxxxxxxxxxxxxxx

                                                          For list
                                                          information or
                                                          to
                                                          unsubscribe,
                                                          visit: 
http://www.unidata.ucar.edu/mailing_lists/

                                                          
                                                          
                                                          

                                                          -- 

                                                          
                                                          Arthur A.
                                                          Person

                                                          Research
                                                          Assistant,
                                                          System
                                                          Administrator

                                                          Penn State
                                                          Department of
                                                          Meteorology

                                                          email: Âaap1@xxxxxxx, 
phone: Â814-863-1563

                                                          
                                                          
                                                          
                                                          

_______________________________________________

                                                          conduit
                                                          mailing list

                                                          
conduit@xxxxxxxxxxxxxxxx

                                                          For list
                                                          information or
                                                          to
                                                          unsubscribe,
                                                          visit: 
http://www.unidata.ucar.edu/mailing_lists/

                                                          
                                                          
                                                          
                                                          

                                                          
                                                          
                                                          

                                                          -- 

                                                          
                                                          Arthur A.
                                                          Person

                                                          Research
                                                          Assistant,
                                                          System
                                                          Administrator

                                                          Penn State
                                                          Department of
                                                          Meteorology

                                                          email: Âaap1@xxxxxxx, 
phone: Â814-863-1563

                                                          
                                                          
                                                          

_______________________________________________

                                                          conduit
                                                          mailing list

                                                          
conduit@xxxxxxxxxxxxxxxx

                                                          For list
                                                          information or
                                                          to
                                                          unsubscribe,
                                                          visit: 
http://www.unidata.ucar.edu/mailing_lists/

                                                          
                                                          
                                                          

                                                          -- 

                                                          
                                                          Arthur A.
                                                          Person

                                                          Research
                                                          Assistant,
                                                          System
                                                          Administrator

                                                          Penn State
                                                          Department of
                                                          Meteorology

                                                          email: Âaap1@xxxxxxx, 
phone: Â814-863-1563

                                                          
                                                          
                                                          

                                                          
                                                          
                                                          

                                                          -- 

                                                          
                                                          Arthur A.
                                                          Person

                                                          Research
                                                          Assistant,
                                                          System
                                                          Administrator

                                                          Penn State
                                                          Department of
                                                          Meteorology

                                                          email: Âaap1@xxxxxxx, 
phone: Â814-863-1563

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          

                                                          
                                                          
                                                          

                                                          -- 

                                                          
                                                          Arthur A.
                                                          Person

                                                          Research
                                                          Assistant,
                                                          System
                                                          Administrator

                                                          Penn State
                                                          Department of
                                                          Meteorology

                                                          email: Âaap1@xxxxxxx, 
phone: Â814-863-1563

                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          
                                                          

                                                          
                                                          
                                                          
                                                          

_______________________________________________

                                                          conduit
                                                          mailing list

                                                          
conduit@xxxxxxxxxxxxxxxx

                                                          For list
                                                          information or
                                                          to
                                                          unsubscribe,
                                                          visit: 
http://www.unidata.ucar.edu/mailing_lists/ 

                                                        
                                                      
                                                      

                                                    
                                                  
                                                
                                              
                                            
                                            

                                          
                                        
                                      
                                    
                                    

_______________________________________________

                                    conduit mailing list

                                    conduit@xxxxxxxxxxxxxxxx

                                    For list information or to
                                    unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/

                                  
                                
                                

                                -- 

                                
                                Arthur A. Person

                                  Research Assistant, System
                                  Administrator

                                  Penn State Department of Meteorology

                                  email: Âaap1@xxxxxxx,
                                  phone: Â814-863-1563

                                
                              
                            
                          
                        
                      
                    
                  
                
                

              
            
            

            -- 

            
            Arthur A. Person

              Research Assistant, System Administrator

              Penn State Department of Meteorology

              email: Âaap1@xxxxxxx, phone:
              Â814-863-1563

            
          
        
      
      
        _______________________________________________

          conduit mailing list

          conduit@xxxxxxxxxxxxxxxx

          For list information or to unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/
          
      
      

      
      

      _______________________________________________
Ncep.list.pmb-dataflow mailing list
Ncep.list.pmb-dataflow@xxxxxxxxxxxxxxxxxxxx
https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.pmb-dataflow