[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #DPA-233086]: Automatic insertion of products into queue


> One of the faculty members here generates daily WRF model output on a
> research cluster.  Currently the data is primarily distributed as images on
> a web server, but we'd like to send it out via our LDM server in order to
> provide easier access.  The first step is to figure out the best way to
> automate the process.  From what I've been able to gather from the LDM
> manpages, it seems pqinsert is the appropriate command.  As a test, I was
> able to send data using the following command
> pqinsert -v -l - -f EXP /project/wxp/a/wxp/www/html/wrfdata/dailyrun/*
> While that successfully got data from the server to a test recipient I set
> up, it's not clear to me how it should be automated.  The same command run
> every few minutes or so from cron would have to re-process every file and
> error on the ones that are already in the queue (and potentially re-send
> files that have expired out of the queue).

We have several sites that use pqinsert(1) to distribute data.  The trick
is to run pqinsert(1) from a script that removes the file if pqinsert(1)
terminates successfully.  That way you don't have to worry about duplicating

> Another option would be to give pqinsert as an -exec argument to find, but
> that seems...like something I would do. :-)
> It could be that I've missed something, but it seems like most of the
> documentation is (understandably) written toward receiving data, not
> originating.  Any advice you could provide would be greatly appreciated.
> Thanks,
> BC
> --
> Ben Cotton
> Unix Systems Administrator
> Department of Earth and
> Atmospheric Sciences
> Purdue University
> http://www.purdue.edu/eas/resources/it/
> CIVL 4212
> O: 765-49-61546
> C: 765-404-7986

Steve Emmerson

Ticket Details
Ticket ID: DPA-233086
Department: Support LDM
Priority: Normal
Status: Closed

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.