[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[NOAAPORT #ZNX-891732]: 20181004: hardware recommendation for ingesting NOAAPort data?
- Subject: [NOAAPORT #ZNX-891732]: 20181004: hardware recommendation for ingesting NOAAPort data?
- Date: Thu, 04 Oct 2018 17:31:55 -0600
Hi Amenda,
I drove your message into our inquiry tracking system so that others
could chime in. I hope you don't mind.
re:
> Do you have hardware recommendations for running the Unidata
> LDM/noaaportIngester to acquire all data on NOAAPort?
>
> We're getting ready to replace some old hardware that is dropping packets
> and want to make sure the replacement doesn't!
Our only guidelines are that the machine running the NOAAPort ingest
have:
- at least 2 Ethernet adapters
One is for the direct connection of the UDP output from the DVBS2
receiver being used (e.g., Novra S300N).
The other is to be used for connecting the machine to the net.
- sufficient RAM
We like to make our LDM queues large enough to hold at least 1 hour
of data received, and more is better. Having a large store of received
products will help the LDM detect AND reject duplicate products. It
is our observation that there are a LOT of duplicated products sent in
the NOAAPort SBN, and we prefer to send as few of the duplicated products
to those receiving LDM/IDD feeds populated by NOAAPort products as
possible.
- be fast enough to keep up with the processing of the NOAAPort stream
of products
This is the least crucial part of our "requirements" since the burden
of ingesting, LDM/IDD product creation and IDD relay is quite modest.
In fact, for years we would use old equipment that was headed for
the scrap heap so that we would be assured that our ingest code would
run at any/all end-user sites.
Perhaps giving you an overview of the two Linux based NOAAPort ingest
machines we run here would be helpful:
chico.unidata.ucar.edu:
CenOS 6.10
8 processors - Intel(R) Xeon(R) CPU E5430 @ 2.66GHz\
32 GB RAM
200 GB disk - 7200 RPM HDD
4 Gbps Ethernet
leno.unidata.ucar.edu - Intel NUC
CenOS 6.10
4 processors - Intel(R) Core(TM) i5-4250U CPU @ 1.30GHz
16 GB RAM
450 GB disk - SSD + HDD
2 Gbps Ethernet
Some questions for you:
- I'm curious about your comment that your machine is dropping packets
Are you sure that this the machine itself, or could missed/dropped
packets be a result of how the NOAAPort stream is being sent to
your machine?
Are you seeing dropped packets in /sbin/ifconfig listings?
- what are the specs for the machine(s) you are currently using?
- what sort of Carrier to Noise (C/N also referred to as EsNo) values
are you seeing on your Novra S300Ns?
A couple of years ago when I was trying to tweak our NOAAPort ingestion,
I had the opportunity to visit the Skaggs building and become informed about
the NOAAPort ingest setup there. It was my observation that our ingest
quality (as measured by (C/N)) was much better than yours, and we had
correspondingly less numbers of Gaps (missing packets in a sequence). At
that time, our C/N values were typically in the mid to high 15s with
occasional
values as high as the low to mid 16s. After the switch to the new satellite,
our C/N improved to the very high 15s to as high as 18 - 18.3. With the
higher C/Ns, we saw a decrease in Gaps and corresponding improvement in
ingest data quality.
Cheers,
Tom
--
****************************************************************************
Unidata User Support UCAR Unidata Program
(303) 497-8642 P.O. Box 3000
address@hidden Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage http://www.unidata.ucar.edu
****************************************************************************
Ticket Details
===================
Ticket ID: ZNX-891732
Department: Support NOAAPORT
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.