Re: [python-users] Trying to save case data for replay or later visualization

  • To: Bryan Guarente <guarente@xxxxxxxx>
  • Subject: Re: [python-users] Trying to save case data for replay or later visualization
  • From: Ryan May <rmay@xxxxxxxx>
  • Date: Wed, 9 Oct 2019 21:29:53 -0600

So this *should* be as easy as:

from siphon.catalog import TDSCatalog
cat = TDSCatalog('

However, because I was foolish, if you try to do this with a big file (i.e.
more than your memory), you will run out of memory because it tries to
download all the bytes first and then writes them to the file. As a
workaround, you can use this function (from here:

import requests
import shutil

def download_file(url):
    local_filename = url.split('/')[-1]
    with requests.get(url, stream=True) as r:
        with open(local_filename, 'wb') as f:
            shutil.copyfileobj(r.raw, f)

    return local_filename

and do:


This should be fixed in the upcoming Siphon 0.9 release.

Hope this helps,


On Wed, Oct 9, 2019 at 3:02 PM Bryan Guarente <guarente@xxxxxxxx> wrote:

> Ryan,
> I am looking for a way to grab all of today's model data as grb files.  We
> had talked about grabbing the "latest" catalog and checking it every time a
> new model updates to download more data.  I thought siphon or even trying
> python-awips would work to get some of this done.  Pointing to an EDEX
> server seemed smart to me, but I am sure you have different ideas.
> Thanks.
> Bryan
> --
> Bryan Guarente
> Meteorologist/Instructional Designer
> The COMET Program
> Boulder, CO
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> python-users mailing list
> python-users@xxxxxxxxxxxxxxxx
> For list information, to unsubscribe, or change your membership options,
> visit:

Ryan May, Ph.D.
Software Engineer
Boulder, CO
  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the python-users archives: