[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[IDD #XYT-473297]: IP address change SUNY Oswego
- Subject: [IDD #XYT-473297]: IP address change SUNY Oswego
- Date: Mon, 19 Aug 2013 09:58:55 -0600
Steven,
Before I tackle this problem, I would like to try something. The LDM host
idd.cise-nsf.gov no longer exists, so would you please replace that hostname
with host idd.meteo.psu.edu in all REQUEST-s for data. Then, restart your LDM
and let me know if the problem persists or not.
> *Tom or any other Unidata support person,*
>
> *Our machine is named blizzard.oswego.edu with a CNAME of
> met-020157.oswego.edu.*
>
> ==============================================================
> #dig blizzard.oswego.edu
>
> ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.4 <<>> blizzard.oswego.edu
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38877
> ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2
>
> ;; QUESTION SECTION:
> ;blizzard.oswego.edu. IN A
>
> ;; ANSWER SECTION:
> blizzard.oswego.edu. 3600 IN CNAME met-020157.oswego.edu.
> met-020157.oswego.edu. 3600 IN A 129.3.79.241
>
> ;; AUTHORITY SECTION:
> oswego.edu. 3600 IN NS monitor.oswego.edu.
> oswego.edu. 3600 IN NS ns2.oswego.edu.
>
> ;; ADDITIONAL SECTION:
> ns2.oswego.edu. 3600 IN A 129.3.245.33
> monitor.oswego.edu. 3600 IN A 129.3.1.100
>
> ;; Query time: 0 msec
> ;; SERVER: 129.3.22.15#53(129.3.22.15)
> ;; WHEN: Sun Aug 18 15:58:24 2013
> ;; MSG SIZE rcvd: 150
> ==============================================================
>
> *However we currently have a data ingesting issue*:
>
> *After many LDM restarts over last week, this weekend and this morning
> the data initially streams in but minutes later stops even though the
> data manager (LDM) stays running. The following two downstream sites
> are used by our LDM to request data-products .**
> ** idd.unidata.ucar.edu **
> ** idd.cise-nsf.gov**
> **
> **Here is some info which may help:*
>
> [ldm@blizzard ~]$ netstat -A inet -t -n
> Active Internet connections (w/o servers)
> Proto Recv-Q Send-Q Local Address Foreign
> Address State
> tcp 0 1 129.3.79.241:45944 192.12.209.62:388 SYN_SENT
> tcp 0 1 129.3.79.241:45947 192.12.209.62:388 SYN_SENT
> tcp 0 1 129.3.79.241:45945 192.12.209.62:388 SYN_SENT
> tcp 0 0 129.3.79.241:41855 128.117.140.3:388 ESTABLISHED
> tcp 0 0 129.3.79.241:45964 128.117.140.3:388 ESTABLISHED
> tcp 0 32 129.3.79.241:41857 128.117.140.3:388 ESTABLISHED
> //=========== Typed the command seconds later ================
> [ldm@blizzard ~]$ netstat -A inet -t -n
> Active Internet connections (w/o servers)
> Proto Recv-Q Send-Q Local Address Foreign
> Address State
> tcp 0 1 129.3.79.241:40309 192.12.209.62:111 SYN_SENT
> tcp 0 1 129.3.79.241:40308 192.12.209.62:111 SYN_SENT
> tcp 0 0 129.3.79.241:41855 128.117.140.3:388 ESTABLISHED
> tcp 0 0 129.3.79.241:45964 128.117.140.3:388 ESTABLISHED
> tcp 0 1 129.3.79.241:40310 192.12.209.62:111 SYN_SENT
> tcp 0 32 129.3.79.241:41857 128.117.140.3:388 ESTABLISHED
> ?
>
> *LDM running for some time with no retrieval of data. Contents of the
> ldmd.log file.*
>
> [ldm@blizzard logs]$ tail -20 ldmd.log
> Aug 18 16:05:57 blizzard idd.cise-nsf.gov[17899] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:05:57 blizzard idd.cise-nsf.gov[17897] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:06:27 blizzard idd.cise-nsf.gov[17896] NOTE: LDM-6 desired
> product-class: 20130818190627.552 TS_ENDT {{CONDUIT, ".*"}}
> Aug 18 16:06:27 blizzard idd.cise-nsf.gov[17899] NOTE: LDM-6 desired
> product-class: 20130818190627.552 TS_ENDT {{NEXRAD3, ".*"}}
> Aug 18 16:06:27 blizzard idd.cise-nsf.gov[17897] NOTE: LDM-6 desired
> product-class: 20130818190627.553 TS_ENDT {{FSL2|UNIDATA, ".*"}}
> Aug 18 16:07:09 blizzard idd.cise-nsf.gov[17896] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:07:09 blizzard idd.cise-nsf.gov[17897] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:07:09 blizzard idd.cise-nsf.gov[17899] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:07:39 blizzard idd.cise-nsf.gov[17897] NOTE: LDM-6 desired
> product-class: 20130818190739.555 TS_ENDT {{FSL2|UNIDATA, ".*"}}
> Aug 18 16:07:39 blizzard idd.cise-nsf.gov[17896] NOTE: LDM-6 desired
> product-class: 20130818190739.555 TS_ENDT {{CONDUIT, ".*"}}
> Aug 18 16:07:39 blizzard idd.cise-nsf.gov[17899] NOTE: LDM-6 desired
> product-class: 20130818190739.555 TS_ENDT {{NEXRAD3, ".*"}}
> Aug 18 16:08:21 blizzard idd.cise-nsf.gov[17897] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:08:21 blizzard idd.cise-nsf.gov[17899] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:08:21 blizzard idd.cise-nsf.gov[17896] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:08:51 blizzard idd.cise-nsf.gov[17899] NOTE: LDM-6 desired
> product-class: 20130818190851.555 TS_ENDT {{NEXRAD3, ".*"}}
> Aug 18 16:08:51 blizzard idd.cise-nsf.gov[17897] NOTE: LDM-6 desired
> product-class: 20130818190851.555 TS_ENDT {{FSL2|UNIDATA, ".*"}}
> Aug 18 16:08:51 blizzard idd.cise-nsf.gov[17896] NOTE: LDM-6 desired
> product-class: 20130818190851.555 TS_ENDT {{CONDUIT, ".*"}}
> Aug 18 16:09:33 blizzard idd.cise-nsf.gov[17897] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:09:33 blizzard idd.cise-nsf.gov[17899] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:09:33 blizzard idd.cise-nsf.gov[17896] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
>
>
> ?
> *Restarted ldm*, beginning of new log file
>
> [ldm@blizzard logs]$ cat ldmd.log
> Aug 18 16:12:31 blizzard rpc.ldmd[26013] NOTE: Starting Up (version: 6.8.1;
> built: Jul 7 2010 16:37:40)
> Aug 18 16:12:31 blizzard rpc.ldmd[26013] NOTE: Using local address 0.0.0.0:388
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26017] NOTE: Starting
> Up(6.8.1): idd.unidata.ucar.edu:388 20130818191231.025 TS_ENDT {{CONDUIT,
> ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26018] NOTE: Starting Up(6.8.1):
> idd.cise-nsf.gov:388 20130818191231.025 TS_ENDT {{CONDUIT, ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26018] NOTE: Previous
> product-information file ".230ab4f86f44477775a32010f7898708.info" doesn't
> exist
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26019] NOTE: Starting Up(6.8.1):
> idd.cise-nsf.gov:388 20130818191231.025 TS_ENDT {{FSL2|UNIDATA, ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26019] NOTE: Previous
> product-information file ".2a46e46fb5a8e4fb424ff572c3a8b098.info" doesn't
> exist
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26018] NOTE: LDM-6 desired
> product-class: 20130818191231.025 TS_ENDT {{CONDUIT, ".*"}}
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26017] NOTE: LDM-6 desired
> product-class: 20130818191231.025 TS_ENDT {{CONDUIT, ".*"},{NONE,
> "SIG=a6a40e2c438bf1b813b8aa3a5494ebb7"}}
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26020] NOTE: Starting
> Up(6.8.1): idd.unidata.ucar.edu:388 20130818191231.025 TS_ENDT
> {{FSL2|UNIDATA, ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26021] NOTE: Starting Up(6.8.1):
> idd.cise-nsf.gov:388 20130818191231.025 TS_ENDT {{NEXRAD3, ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26019] NOTE: LDM-6 desired
> product-class: 20130818191231.025 TS_ENDT {{FSL2|UNIDATA, ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26021] NOTE: Previous
> product-information file ".6def16e8ae4b45a01e5341c188d8251f.info" doesn't
> exist
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26022] NOTE: Starting
> Up(6.8.1): idd.unidata.ucar.edu:388 20130818191231.025 TS_ENDT {{NEXRAD3,
> ".*"}}
> Aug 18 16:12:31 blizzard idd.cise-nsf.gov[26021] NOTE: LDM-6 desired
> product-class: 20130818191231.025 TS_ENDT {{NEXRAD3, ".*"}}
> Aug 18 16:12:31 blizzard pqact[26015] NOTE: Starting Up
> Aug 18 16:12:31 blizzard rtstats[26016] NOTE: Starting Up (26013)
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26020] NOTE: LDM-6 desired
> product-class: 20130818191231.025 TS_ENDT {{FSL2|UNIDATA, ".*"},{NONE,
> "SIG=7c9ad53720b698efcd32744f7be84efc"}}
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26022] NOTE: LDM-6 desired
> product-class: 20130818191231.025 TS_ENDT {{NEXRAD3, ".*"},{NONE,
> "SIG=86587a1240f88c361f6bea89f0951e5e"}}
> Aug 18 16:12:31 blizzard pqact[26015] NOTE: Starting from insertion-time
> 2010-10-26 18:32:36.767870 UTC
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26017] NOTE: Upstream LDM-6 on
> idd.unidata.ucar.edu is willing to be a primary feeder
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26022] NOTE: Upstream LDM-6 on
> idd.unidata.ucar.edu is willing to be an alternate feeder
> Aug 18 16:12:31 blizzard idd.unidata.ucar.edu[26020] NOTE: Upstream LDM-6 on
> idd.unidata.ucar.edu is willing to be an alternate feeder
> Aug 18 16:12:34 blizzard pqact[26015] WARN: Processed oldest product in
> queue: 0.000279 s
> Aug 18 16:13:13 blizzard idd.cise-nsf.gov[26018] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:13:13 blizzard idd.cise-nsf.gov[26021] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
> Aug 18 16:13:13 blizzard idd.cise-nsf.gov[26018] NOTE: LDM-6 desired
> product-class: 20130818191313.025 TS_ENDT {{CONDUIT, ".*"}}
> Aug 18 16:13:13 blizzard idd.cise-nsf.gov[26021] NOTE: LDM-6 desired
> product-class: 20130818191313.025 TS_ENDT {{NEXRAD3, ".*"}}
> Aug 18 16:13:13 blizzard idd.cise-nsf.gov[26019] ERROR: Disconnecting due to
> LDM failure; Couldn't connect to LDM on idd.cise-nsf.gov using either port
> 388 or portmapper; : RPC: Remote system error - Connection timed out
>
>
> #ps –ef
> ldm 26013 1 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26015 26013 0 16:12 ? 00:00:01 pqact
> ldm 26016 26013 0 16:12 ? 00:00:00 rtstats -h
> rtstats.unidata.ucar.edu
> ldm 26017 26013 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26018 26013 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26019 26013 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26020 26013 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26021 26013 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26022 26013 0 16:12 ? 00:00:00 rpc.ldmd -I 0.0.0.0 -P 388 -M
> 256 -m 3600 -o 3600 -q /usr/local/ldm/data/ldm.pq /usr
> ldm 26030 26015 0 16:12 ? 00:00:00 decoders/dcmetr -v 2 -a 500
> -m 72 -s sfmetar_sa.tbl -d data/gempak/logs/dcmetr.log -
> ldm 26031 26015 0 16:12 ? 00:00:02 decoders/dcmsfc -b 9 -a 10000
> -d data/gempak/logs/dcmsfc.log -e GEMTBL=/home/gempak/
> ldm 26032 26015 0 16:12 ? 00:00:00 decoders/dcmsfc -d
> data/gempak/logs/dcmsfc_6hr.log -a 6 -d data/gempak/logs/dcmsfc_6
> ldm 26034 26015 0 16:14 ? 00:00:00 decoders/dcuair -b 24 -m 16
> -s snstns.tbl -d data/gempak/logs/dcuair.log -e GEMTBL=/
>
>
> *Minutes later the data ingestion (decoders active) will cease (decoders
> inactive).*
>
>
>
> *We would greatly appreciate any help with this matter.*
>
>
> Sincerely,
> Steven Skubis
>
>
> -----------------------------------------------------------------------
> Dr. Steven Skubis
> Department of Earth Sciences
> Oswego State University of New York Phone: 315.312.2799
> Shineman Center Fax: 315.312.3059
> Oswego, NY 13126-3599 Email: address@hidden
> -----------------------------------------------------------------------
Regards,
Steve Emmerson
Ticket Details
===================
Ticket ID: XYT-473297
Department: Support LDM
Priority: Normal
Status: Closed
