My 2 cents.
My queues are typically for the processing of real-time data. I don’t
generally keep an hour of data in the queue because I have redundancy and
in general if data coming across is more than 20-30 minutes old I’m going
to ignore it anyway. I know that most data is archived elsewhere and I’m
not interested in keeping an archive— just wanting to process what I get in
real-time.
I played around with keeping the queue in Ramdisk, but I didn’t see that it
gained me much. The queue is in mapped memory so the ramdisk really only
seemed to help when when stopping LDM because that’s when things takes
longer depending on queue size.
I usually set offset time to 10 minutes so when I start things up I don’t
try to backfill much.
Note: I’ve been running for quite a while with no problems. But, feel free
to correct anything I’ve got wrong.
On Fri, Oct 1, 2021 at 3:02 PM Patrick L. Francis <wxprofessor@xxxxxxxxx>
wrote:
>
> So nobody had anything to say on the subject of queue size? :) (thanks
> to everyone that wrote privately providing feeds until the new dish(es) are
> up :)
>
> Essentially where I was going with the question, is I know that I'm old..
> but I seem to recall somewhere along the line that the idea behind the ldm
> queue size, is that it should be able to hold somewhere around 1 hour's (or
> more) of data... well that value has changed a lot over the years, as the
> volume of noaaport has changed too!
>
> So I was just wondering if there were any new thoughts on that subject...
> I plan to do some analysis, but was just thinking... how much do we need
> now, compared to what we needed 10, or 15 years ago? and how cool it was,
> and does it matter? etc... :)
>
> Just hoping to start a discussion!
>
> cheers,
>
> --patrick
>
> wxprofessor @ gmail . com
>
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
--
*"Outside of a dog, a book is a man's best friend. Inside of a dog, it's
too dark to read."*
*--Groucho Marx*
-------------------------------------------
Karen.Cooper@xxxxxxxx
Phone#: 405-325-6456
Cell: 405-834-8559
CIWRO/OU
National Severe Storms Laboratory
