Re: [conduit] Large CONDUIT latencies to UW-Madison idd.aos.wisc.edu starting the last day or two.

  • To: "'Carissa Klemmer - NOAA Federal'" <carissa.l.klemmer@xxxxxxxx>
  • Subject: Re: [conduit] Large CONDUIT latencies to UW-Madison idd.aos.wisc.edu starting the last day or two.
  • From: "Patrick L. Francis" <wxprofessor@xxxxxxxxx>
  • Date: Mon, 22 Feb 2016 11:11:19 -0500
So from the looks of your screen grab that is 7 minutes to download the 0.25 
GFS from our FTP server. Do you have ideas of how slow that is compared to 
normal from Colorado? I will be passing this information on to our IT folks.




Carissa Klemmer 
NCEP Central Operations 
Dataflow Team Lead



Hey Carissa :)

 

I left the fetch script run on our colo box this morning so we could get a good 
idea of download times. Here is a screen cap of that directory structureâ 
Essentially when I fetch model files, I drop them on a ramdisk and encode them 
on the fly, and then leave them on the ramdisk to ensure faster processing. 
Here is what the 06z run looks like so far (obviously non-operational :) )

 

http://drmalachi.org/files/ncep/he-gfsdl.2016.02.22.png

 

So we can see times vary from 6 minutes to it looks like 26 minutes to download 
one GFS run from a 100gbit connection directly on the Hurricane Electric 
backbone 8 hops away from ncep :)  Here is the traceroute / MTR from that box 
earlier this morning:

 

http://drmalachi.org/files/ncep/he-ncep.2016.02.22.jpg

 

What leads me to believe that packetshaping is the likely culprit, is that I 
can jump on my amazon ec2 box at the same time, and download the same file in 
about 2 seconds from the same source :)

 

Here is a sample of the routing from the EC2 spinup to the ncep server.. recall 
the first 20 or so hops are just internal to amazon

 

http://drmalachi.org/files/ncep/ec2-ncep.png

 

The difference between the Hurricane Electric / colo routing to the amazon 
routing appears to be the handoff / routing from Internet2 to gigapop to 
140.90.111.36â so whatever is âtriggeringâ the different routes is what is 
initiating the delay :)   Or it could just be the dreaded gopher! :)

 

Happy Monday!

 

Cheers,

 

--patrick

 

-------------------------------------------------------

Patrick L. Francis

Vice President of Research & Development

 

Aeris Weather

 

 <http://aerisweather.com/> http://aerisweather.com/

 <http://modelweather.com/> http://modelweather.com/

 

 <http://facebook.com/wxprofessor/> http://facebook.com/wxprofessor/