Hey Patrick,
Good to see you back! To answer your question easily, let's take a look at
the College of DuPage's dish stats:
https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?noaaport3.cod.edu
As you can see, their max peak hourly was about 30 GB in one hour; their
average is around 20 GB/hr.
NOAAAport is a 120 mbsec feed now, and on December 1, 2021, as super-res
level 3 radar gets sent across, and during major events, you better believe
that number will bounce up further!
Gilbert
On Fri, Oct 1, 2021 at 3:31 PM Patrick L. Francis <wxprofessor@xxxxxxxxx>
wrote:
>
> Karen,
>
> There's a name I haven't seen for awhile, great to see your name again!1
>
> My queues are typically for the processing of real-time data. I don’t
> generally keep an hour of data in the queue because I have redundancy and
> in general if data coming across is more than 20-30 minutes old I’m going
> to ignore it anyway. I know that most data is archived elsewhere and I’m
> not interested in keeping an archive— just wanting to process what I get in
> real-time.
>
> Here's wher it get interesting... or perhaps this is just an old man that
> gets excited about silly things... :) I like both having the dish for the
> speediness of the delivery, but also to backup friends down the road.. When
> I was a professor it was backing up others way back in the day that there
> were only a few of us out there.. like gerry at A&M had the massive feeds
> going... Well now I'm not a professor anymore, but I like backing up other
> weather peeps in the commercial sector too :)
>
> So, if I have one of my noaaport relays well here.. let me elaborate:
>
> for those that may not remember I take the feed from the dish to the
> novra, then split that multicast feed from the novra to two (minimum)
> noaaport relays.. the job of the noaaport relays is to balance the feed to
> all of my production servers (where the actual work on the data is done)
> ... well, I also have friends out there that depend on timely data, and as
> an old unidata member, I like to share, and help others too, who in turn
> help me :) So those that have a downstream backup feed from me, may have
> an issue with their setup and need to reboot their system, or start up
> another and need that data they missed... here is where it gets critical...
>
> If someone is making a product, and had a hard time getting wherever they
> needed to go to boot up their ingest feed, and I am one of their backups,
> or if it's a university setup that backs up from me that is running
> research, or whatever.... how much data should be kept in the queu e for
> them to fire up?
>
> In the beginning years ago 500m was fine... but by a few years ago we
> needed several gig to cover an hour, and we have since had goesR added and
> more model products added...
>
> what I do on each of my boxes is build the ramdisk to keep the queue in
> because it's much much faster (less io and such) at xferring data from the
> queue to be processed on the fly triggered from receipt, but also because
> someone that might be downstream from me as a backup might need to pull
> some data too, though I setup my backup feeds from the relays and not the
> production servers...
>
> anyway, I'm just one of the weird guys that thinks about these things, and
> years ago more people would chiume in and chat about things I guess lol :)
>
> so that's my 2 cents, with your 2 cents, so does that mean that together
> we make less sense? :)
>
> thanks for chiming in karen.. good to see your name again :)
>
> cheers,
>
> --patrick
>
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
--
----
Gilbert Sebenste
Consulting Meteorologist
AllisonHouse, LLC
