Jeff:
You may recall our correspondence a couple of months ago about establishing
our LDM. We were able to get it going with the help of a colleague, Hans
Schwaiger, who's smarter than I.
One thing I was wondering: would it be possible to allow us to download
data two TWO USGS computers from idd.unidata.ucar.edu?
Currently, you're allowing downloads to drum.wr.usgs.gov
I'd like to add phobos.wr.usgs.gov
One of these computers (drum) and its user (Hans) will be moving to the USGS
office in Anchorage in July. We'd like to run a volcanic ash transport
model in both Anchorage (on drum) and Vancouver, WA (on phobos). The amount
of data we download is not very large, several tens of megabytes per day I
think per computer.
Thanks,
Larry Mastin
David A. Johnston Cascades Volcano Observatory
1300 SE Cardinal Court, Bldg. 10, Suite 100
Vancouver, WA 98683 USA
tel. 360-993-8925 fax 360-993-8980
http://vulcan.wr.usgs.gov/Projects/Mastin
-----Original Message-----
From: Jeff Weber [mailto:address@hidden]
Sent: Wednesday, April 07, 2010 2:05 PM
To: Larry Mastin
Subject: RE: writing files in the product queue
Hi Larry,
Did we have success?
Curious,
Jeff
---
On Tue, 6 Apr 2010, Larry Mastin wrote:
Thanks, Jeff. I made the change. Hopefully it will work.
By the way, is it possible to test pqact by running it directly, rather
than
waiting for the cron job to do it? I tried, for example, typing the
following at the command line while in ~/data:
pqact -v -q ldm.pq
After hitting the "return" key I got no response. I waited ten minutes
and
still nothing.
Thanks,
Larry
David A. Johnston Cascades Volcano Observatory
1300 SE Cardinal Court, Bldg. 10, Suite 100
Vancouver, WA 98683 USA
tel. 360-993-8925 fax 360-993-8980
http://vulcan.wr.usgs.gov/Projects/Mastin
-----Original Message-----
From: Jeff Weber [mailto:address@hidden]
Sent: Tuesday, April 06, 2010 8:33 AM
To: Larry Mastin
Subject: Re: writing files in the product queue
Hi Larry,
Apologies for the delay, got hectic here yesterday aft :)
I would add a -close arg after the FILE command.
i.e.
CONDUIT ^data/nccf/com/nam.*awipak*
FILE -close ~/data/nam/Alaska_45km/\1
Please keep in mind that there should only be tabs in pqact.conf and no
"blank space" to separate the entry.
Please let me know if this does not solve the problem, I think it should
:)
Cheers,
Jeff
---------------------------------------------------------------------
Jeff Weber address@hidden :
Unidata Program Center PH:303-497-8676 :
University Corp for Atmospheric Research 3300 Mitchell Ln :
http://www.unidata.ucar.edu/staff/jweber Boulder,Co 80307-3000 :
---------------------------------------------------------------------
On Mon, 5 Apr 2010, Larry Mastin wrote:
Dear Jeff:
You may recall our recent correspondence (below). I was trying to set
up the ldm to ingest NAM files from AWIPS grids 212, 216, and 104. I
think the files are being successfully loaded into the product queue, as
indicated by the following information.
ldm@phobos:~> pqmon
Apr 05 19:34:22 pqmon NOTE: Starting Up (8530)
Apr 05 19:34:22 pqmon NOTE: nprods nfree nempty nbytes maxprods
maxfree minempty maxext age
Apr 05 19:34:22 pqmon NOTE: 54672 1 67397 499993096 55609
2 66459 9720 39018
Apr 05 19:34:22 pqmon NOTE: Exiting
Also, with the command plotMetrics I can see that data are being
downloaded every six hours or so.
In the pqact.conf file I have commands to write these data out to
separate files, but for some reason they're not being written.
Here's the line in the pqact.conf file that's supposed to write out the
NAM AWIPS grid 216 file:
CONDUIT ^data/nccf/com/nam.*awipak* FILE ~/data/nam/Alaska_45km/\1
I've tried a few different ERE syntaxes, all of which have been tested
using the regex command and found to match NCEP file names of the format
given at:
http://www.unidata.ucar.edu/data/conduit/ldm_idd/nam_files.html
Unfortunately, none of the expressions I've tried have worked in getting
the files written from the product queue.
So, I wonder if I'm perhaps missing something else.
The ldmd.conf and pqact.conf files I'm using are attached, as well as a
shell script I've been using to test the regular expressions.
If you have any suggestions on how to get this to work, I'd appreciate
it.
Thanks,
Larry Mastin
On Thu, 2010-03-11 at 14:57 -0700, Jeff Weber wrote:
Hi Larry,
For sanity, please try:
notifyme -vl - -h idd.unidata.ucar.edu -f ANY
Thanks,
Jeff
---
On Thu, 11 Mar 2010, Larry Mastin wrote:
Jeff:
Here's what I get:
ldm@phobos:~> notifyme -vl - -h freshair.atmos.washington.edu -f ANY
Mar 11 21:47:35 notifyme[5173] NOTE: Starting Up:
freshair.atmos.washington.edu: 20100311214735.906 TS_ENDT {{ANY,
".*"}}
Mar 11 21:47:35 notifyme[5173] NOTE: LDM-5 desired product-class:
20100311214735.906 TS_ENDT {{ANY, ".*"}}
Mar 11 21:47:35 notifyme[5173] INFO: Resolving
freshair.atmos.washington.edu to 128.95.89.10 took 0.00027 seconds
Mar 11 21:47:36 notifyme[5173] ERROR:
NOTIFYME(freshair.atmos.washington.edu): 7: Access denied by remote
server
Mar 11 21:48:01 notifyme[5173] NOTE: LDM-5 desired product-class:
20100311214735.906 TS_ENDT {{ANY, ".*"}}
Mar 11 21:48:01 notifyme[5173] INFO: Resolving
freshair.atmos.washington.edu to 128.95.89.10 took 1e-05 seconds
Mar 11 21:48:01 notifyme[5173] ERROR:
NOTIFYME(freshair.atmos.washington.edu): 7: Access denied by remote
server
Mar 11 21:48:26 notifyme[5173] NOTE: LDM-5 desired product-class:
20100311214735.906 TS_ENDT {{ANY, ".*"}}
Mar 11 21:48:26 notifyme[5173] INFO: Resolving
freshair.atmos.washington.edu to 128.95.89.10 took 7e-06 seconds
Mar 11 21:48:26 notifyme[5173] ERROR:
NOTIFYME(freshair.atmos.washington.edu): 7: Access denied by remote
server
Mar 11 21:48:51 notifyme[5173] NOTE: LDM-5 desired product-class:
20100311214735.906 TS_ENDT {{ANY, ".*"}}
Mar 11 21:48:51 notifyme[5173] INFO: Resolving
freshair.atmos.washington.edu to 128.95.89.10 took 9e-06 seconds
Mar 11 21:48:52 notifyme[5173] ERROR:
NOTIFYME(freshair.atmos.washington.edu): 7: Access denied by remote
server
Mar 11 21:49:17 notifyme[5173] NOTE: LDM-5 desired product-class:
20100311214735.906 TS_ENDT {{ANY, ".*"}}
Mar 11 21:49:17 notifyme[5173] INFO: Resolving
freshair.atmos.washington.edu to 128.95.89.10 took 9e-06 seconds
Mar 11 21:49:17 notifyme[5173] ERROR:
NOTIFYME(freshair.atmos.washington.edu): 7: Access denied by remote
server
Mar 11 21:49:42 notifyme[5173] NOTE: LDM-5 desired product-class:
20100311214735.906 TS_ENDT {{ANY, ".*"}}
Mar 11 21:49:42 notifyme[5173] INFO: Resolving
freshair.atmos.washington.edu to 128.95.89.10 took 7e-06 seconds
Mar 11 21:49:42 notifyme[5173] ERROR:
NOTIFYME(freshair.atmos.washington.edu): 7: Access denied by remote
server
It looks like access is being denied.
Larry
On Thu, 2010-03-11 at 14:25 -0700, Jeff Weber wrote:
Hi Larry,
We have our User's Committee meeting today, apologies for the late
response:
As ldm run:
notifyme -vl - -h freshair.atmos.washington.edu -f ANY
..this will determine if freshair is willing to feed you, if so,
products
should scroll by.
I see your e-mail in support now as well, we'll figure this one
out
:)
Cheers,
Jeff
On Thu, 11 Mar 2010, Larry Mastin wrote:
Dear Unidata (Jeff?):
Six months ago you assisted me in setting up an ldm server at
the
USGS
(the message below may jog your memory). It turns out I didn't
finish
the job, but am now working on it again. I tried activating the
ldm
server yesterday (using "ldmadmin start") but thus far haven't
received
any data and have seen no messages in the log file
~/logs/ldmd.log. I'm
trying to troubleshoot my installation, some of which was done
last
September.
I found one issue that I could use advice one. I'm looking at
the
ldm
pre-installation instructions:
http://www.unidata.ucar.edu/software/ldm/ldm-6.8.1/basics/preinstallation.ht
ml
under "Edit file /etc/syslog.conf". It says to "Add the
following
entry
to file /etc/syslog.conf (or /etc/rsyslog.conf, depending on
your
operating system):
--localn.debug /usr/local/ldm/logs/ldmd.log
On my Linux Suse 11.0 system, my /etc folder contains neither a
syslog.conf nor an rsyslog.conf file. There is however a
/etc/syslog-ng
folder which contains a syslog-ng.conf file. Is that where this
line is
supposed to go?
Thanks for your help.
Larry Mastin
On Tue, 2009-09-29 at 11:27 -0600, Jeff Weber wrote:
Hi Harry,
Great, thanks for the quick response :)
Larry, please let Unidata know if we can help configure
anything
for you.
One caveat, CONDUIT is voluminous, so it is better to throttle
back the
request to only what you desire (we can help with that).
Cheers,
Jeff
---------------------------------------------------------------------
Jeff Weber address@hidden
:
Unidata Program Center PH:303-497-8676
:
University Corp for Atmospheric Research 3300 Mitchell Ln
:
http://www.unidata.ucar.edu/staff/jweber Boulder,Co
80307-3000 :
---------------------------------------------------------------------
On Tue, 29 Sep 2009, Harry Edmon wrote:
freshair.atmos.washington.edu is the alias to our current
operational
system and should be the address they should use. It is
all
set up.
Jeff Weber wrote:
Hi Harry,
The good people at the USGS David A. Johnston Cascades
Volcano
Observatory, would like a feed:
They are requesting:
HDS
CONDUIT
CMC (GEM)
FNMOC
They also want NGRID, but I see you do not have that feed,
so Larry,
please point your NGRID request at us:
idd.unidata.ucar.edu
Larry,
I know I suggested getting the CMC (GEM) data and FNMOC
from
us, but I see
U of W has those, so let's not ship it over to Washington
twice, if Harry
will place the allow for you let's feed those 4 feeds from
U
of W and we
will feed you the NGRID feed...sound good?
Harry, which machine would you like Larry to feed from if
you choose to
grant access to the feed?
freshair2.atmos.washington.edu or
freshair1.atmos.washington.edu?
Thanks for considering this request, I have cc'd Larry
Mastin, contact at
USGS on this e-mail. The machine name to "allow" is:
phobos.wr.usgs.gov
Cheers,
Jeff
---------------------------------------------------------------------
Jeff Weber
address@hidden :
Unidata Program Center
PH:303-497-8676 :
University Corp for Atmospheric Research 3300
Mitchell
Ln :
http://www.unidata.ucar.edu/staff/jweber Boulder,Co
80307-3000 :
---------------------------------------------------------------------
--
Dr. Harry Edmon E-MAIL:
address@hidden
206-543-0547 FAX: 206-543-0308
address@hidden, address@hidden
Director of IT, College of the Environment and
Director of Computing, Dept of Atmospheric Sciences
University of Washington, Box 351640, Seattle, WA
98195-1640
