So from the looks of your screen grab that is 7 minutes to download the 0.25
GFS from our FTP server. Do you have ideas of how slow that is compared to
normal from Colorado? I will be passing this information on to our IT folks.
Carissa Klemmer
NCEP Central Operations
Dataflow Team Lead
Hey Carissa :)
I left the fetch script run on our colo box this morning so we could get a good
idea of download times. Here is a screen cap of that directory structure…
Essentially when I fetch model files, I drop them on a ramdisk and encode them
on the fly, and then leave them on the ramdisk to ensure faster processing.
Here is what the 06z run looks like so far (obviously non-operational :) )
http://drmalachi.org/files/ncep/he-gfsdl.2016.02.22.png
So we can see times vary from 6 minutes to it looks like 26 minutes to download
one GFS run from a 100gbit connection directly on the Hurricane Electric
backbone 8 hops away from ncep :) Here is the traceroute / MTR from that box
earlier this morning:
http://drmalachi.org/files/ncep/he-ncep.2016.02.22.jpg
What leads me to believe that packetshaping is the likely culprit, is that I
can jump on my amazon ec2 box at the same time, and download the same file in
about 2 seconds from the same source :)
Here is a sample of the routing from the EC2 spinup to the ncep server.. recall
the first 20 or so hops are just internal to amazon
http://drmalachi.org/files/ncep/ec2-ncep.png
The difference between the Hurricane Electric / colo routing to the amazon
routing appears to be the handoff / routing from Internet2 to gigapop to
140.90.111.36… so whatever is “triggering” the different routes is what is
initiating the delay :) Or it could just be the dreaded gopher! :)
Happy Monday!
Cheers,
--patrick
-------------------------------------------------------
Patrick L. Francis
Vice President of Research & Development
Aeris Weather
<http://aerisweather.com/> http://aerisweather.com/
<http://modelweather.com/> http://modelweather.com/
<http://facebook.com/wxprofessor/> http://facebook.com/wxprofessor/
