[ldm-users] Restoring Service after Firewall failure

• To: ldm-users@xxxxxxxxxxxxxxxx
• Subject: [ldm-users] Restoring Service after Firewall failure
• From: Phil Birnie <phil@xxxxxxxxxxx>
• Date: Mon, 21 May 2012 22:37:09 -0400

Hey Everyone,

Looking for a bit of help for our network administrators here.  Our firewall 
went down last Friday and our network admins are in the process of restoring 
it.  Nothing has been changed on our boxes (aside from a reboot once everything 
came back online), but we are still having issues getting data.   Before this 
outage, everything was working fine.   We're now getting "Access denied 
messages" from our upstream hosts... (details below)

Any help is very much appreciated. 

- Phil 
Dept. of Geography
Ohio State University 


Our error logs are producing lots of the following: 

May 21 21:24:34 twister2 idd.unidata.ucar.edu[7214] ERROR: Disconnecting due to 
LDM failure; nullproc_6 failure to idd.unidata.ucar.edu; RPC: Unable to 
receive; errno = Connection reset by peer
May 21 21:24:34 twister2 idd.unidata.ucar.edu[7210] ERROR: Disconnecting due to 
LDM failure; nullproc_6 failure to idd.unidata.ucar.edu; RPC: Unable to 
receive; errno = Connection reset by peer
May 21 21:24:34 twister2 idd.unidata.ucar.edu[7216] ERROR: Disconnecting due to 
LDM failure; nullproc_6 failure to idd.unidata.ucar.edu; RPC: Unable to 
receive; errno = Connection reset by peer
May 21 21:24:35 twister2 stokes.metr.ou.edu[7211] NOTE: LDM-6 desired 
product-class: 20120522012435.723 TS_ENDT {{FNEXRAD|FSL2|UNIDATA,  ".*"},{NONE, 
 "SIG=3aae7c983d94935a8aa158fa4f2de51e"}}
May 21 21:24:35 twister2 stokes.metr.ou.edu[7211] ERROR: Disconnecting due to 
LDM failure; nullproc_6 failure to stokes.metr.ou.edu; RPC: Unable to receive; 
errno = Connection reset by peer

Running notifyme to all of our upstream feeds: 

[ldm@twister2 etc]$ notifyme -vl- -f ANY -h flood.atmos.uiuc.edu
May 22 02:20:50 notifyme[424] NOTE: Starting Up: flood.atmos.uiuc.edu: 
20120522022050.861 TS_ENDT {{ANY,  ".*"}}
May 22 02:20:50 notifyme[424] NOTE: LDM-5 desired product-class: 
20120522022050.861 TS_ENDT {{ANY,  ".*"}}
May 22 02:20:50 notifyme[424] INFO: Resolving flood.atmos.uiuc.edu to 
128.174.80.16 took 0.001245 seconds
May 22 02:20:55 notifyme[424] ERROR: NOTIFYME(flood.atmos.uiuc.edu): 7: Access 
denied by remote server

My first guess was that it was a nameserver issue as outlined in the following 
thread or that port 388 was not properly opened. 

https://www.unidata.ucar.edu/support/help/MailArchives/idd/msg05869.html

However, both forward and backward DNS queries look okay and port 388 appears 
to be open. 

Philips-MacBook-Pro:~ philbirnie$ nslookup twister2.sbs.ohio-state.edu
Server:         192.168.1.1
Address:        192.168.1.1#53

Non-authoritative answer:
Name:   twister2.sbs.ohio-state.edu
Address: 128.146.194.17

Philips-MacBook-Pro:~ philbirnie$ nslookup 128.146.194.17
Server:         192.168.1.1
Address:        192.168.1.1#53

Non-authoritative answer:
17.194.146.128.in-addr.arpa     name = twister2.sbs.ohio-state.edu.

Authoritative answers can be found from:
146.128.in-addr.arpa    nameserver = ns2.net.ohio-state.edu.
146.128.in-addr.arpa    nameserver = ns1.net.ohio-state.edu.
ns1.net.ohio-state.edu  internet address = 128.146.1.7
ns2.net.ohio-state.edu  internet address = 128.146.48.7

Philips-MacBook-Pro:~ philbirnie$ 

• Follow-Ups:
  • Re: [ldm-users] Restoring Service after Firewall failure
    • From: Gilbert Sebenste