Re: [conduit] Large CONDUIT latencies to UW-Madison starting the last day or two.

  • To: "'Carissa Klemmer - NOAA Federal'" <carissa.l.klemmer@xxxxxxxx>
  • Subject: Re: [conduit] Large CONDUIT latencies to UW-Madison starting the last day or two.
  • From: "Patrick L. Francis" <wxprofessor@xxxxxxxxx>
  • Date: Mon, 22 Feb 2016 11:11:19 -0500
So from the looks of your screen grab that is 7 minutes to download the 0.25 
GFS from our FTP server. Do you have ideas of how slow that is compared to 
normal from Colorado? I will be passing this information on to our IT folks.

Carissa Klemmer 
NCEP Central Operations 
Dataflow Team Lead

Hey Carissa :)


I left the fetch script run on our colo box this morning so we could get a good 
idea of download times. Here is a screen cap of that directory structure… 
Essentially when I fetch model files, I drop them on a ramdisk and encode them 
on the fly, and then leave them on the ramdisk to ensure faster processing. 
Here is what the 06z run looks like so far (obviously non-operational :) )


So we can see times vary from 6 minutes to it looks like 26 minutes to download 
one GFS run from a 100gbit connection directly on the Hurricane Electric 
backbone 8 hops away from ncep :)  Here is the traceroute / MTR from that box 
earlier this morning:


What leads me to believe that packetshaping is the likely culprit, is that I 
can jump on my amazon ec2 box at the same time, and download the same file in 
about 2 seconds from the same source :)


Here is a sample of the routing from the EC2 spinup to the ncep server.. recall 
the first 20 or so hops are just internal to amazon


The difference between the Hurricane Electric / colo routing to the amazon 
routing appears to be the handoff / routing from Internet2 to gigapop to… so whatever is “triggering” the different routes is what is 
initiating the delay :)   Or it could just be the dreaded gopher! :)


Happy Monday!







Patrick L. Francis

Vice President of Research & Development


Aeris Weather










  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the conduit archives: