Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [conduit] Large CONDUIT latencies to UW-Madison idd.aos.wisc.edu starting the last day or two.

  • To: "'Carissa Klemmer - NOAA Federal'" <carissa.l.klemmer@xxxxxxxx>
  • Subject: Re: [conduit] Large CONDUIT latencies to UW-Madison idd.aos.wisc.edu starting the last day or two.
  • From: "Patrick L. Francis" <wxprofessor@xxxxxxxxx>
  • Date: Mon, 22 Feb 2016 11:11:19 -0500
So from the looks of your screen grab that is 7 minutes to download the 0.25 
GFS from our FTP server. Do you have ideas of how slow that is compared to 
normal from Colorado? I will be passing this information on to our IT folks.




Carissa Klemmer 
NCEP Central Operations 
Dataflow Team Lead



Hey Carissa :)

 

I left the fetch script run on our colo box this morning so we could get a good 
idea of download times. Here is a screen cap of that directory structure… 
Essentially when I fetch model files, I drop them on a ramdisk and encode them 
on the fly, and then leave them on the ramdisk to ensure faster processing. 
Here is what the 06z run looks like so far (obviously non-operational :) )

 

http://drmalachi.org/files/ncep/he-gfsdl.2016.02.22.png

 

So we can see times vary from 6 minutes to it looks like 26 minutes to download 
one GFS run from a 100gbit connection directly on the Hurricane Electric 
backbone 8 hops away from ncep :)  Here is the traceroute / MTR from that box 
earlier this morning:

 

http://drmalachi.org/files/ncep/he-ncep.2016.02.22.jpg

 

What leads me to believe that packetshaping is the likely culprit, is that I 
can jump on my amazon ec2 box at the same time, and download the same file in 
about 2 seconds from the same source :)

 

Here is a sample of the routing from the EC2 spinup to the ncep server.. recall 
the first 20 or so hops are just internal to amazon

 

http://drmalachi.org/files/ncep/ec2-ncep.png

 

The difference between the Hurricane Electric / colo routing to the amazon 
routing appears to be the handoff / routing from Internet2 to gigapop to 
140.90.111.36… so whatever is “triggering” the different routes is what is 
initiating the delay :)   Or it could just be the dreaded gopher! :)

 

Happy Monday!

 

Cheers,

 

--patrick

 

-------------------------------------------------------

Patrick L. Francis

Vice President of Research & Development

 

Aeris Weather

 

 <http://aerisweather.com/> http://aerisweather.com/

 <http://modelweather.com/> http://modelweather.com/

 

 <http://facebook.com/wxprofessor/> http://facebook.com/wxprofessor/

 

 

 

 

  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the conduit archives: