Re: question about IDV: running from automated scripts, triggered by events?

Bill-

Bill Hibbard wrote:
Don or other Unidata folks,

I am really impressed by the IDV, and have a question.

Thanks.  Couldn't have done it with out VisAD and
your support.

One common way that people use Vis5D is via automated
scripts triggered by events. The event can be either a
timer event from CRON, or an event generated by a model
run finishing. This triggers Vis5D to start, running the
commands in a Vis5D TCL script that read data files,
generate graphics, and save them to GIF images. Then a
shell script may combine GIF images into MPEG animations
and/or link them into web pages.

My questions is whether the IDV can be used in this way
or if there are plans for this? I note that the IDV can
be scripted somewhat from HTML pages, and Tom tells me
that he has scripted it from Python.

Scripting of the IDV has been a requirement a requirement
from the beginning, but one that hasn't
been completely implemented yet.  You can run the IDV
with a -nogui option with the intent that it would be
used for this.  We also use it for testing during our
nightly builds.

There are two problems we've encountered:

1) making sure the display is completed before making
the JPEG.  Using the sync for getImage helps with this,
but it not fail-safe.

And the biggest obstacle:
2) The excessive time it takes to invoke the JVM, just
to create a display.  If you look at the way that most
Unidata sites use GEMPAK and McIDAS to do something
similar to what you say Vis5D users do, then the number
of JVMs running at any time on a system would be a
great load.  Consider generating a GIF for every NEXRAD
Level III base reflectivity image that comes in (~20/minute)
on a data feed and it takes 5 seconds just to start the JVM.
You would rapidly get behind. You might be able to get away
with having an IDV running in the background, and then using
that to generate products in, but then you'd have to worry about
threading issues (e.g., concurrent actions to the IDV).

The event triggering probably only requires a way to
start the IDV with a script file specified on the command
line.

You can start the IDV with a "bundle" which can be used
to load in various datasources/displaycontrols.  You can
actually have a default bundle that would set up the
map projection, zoom factor, display size, map backgrounds
and then load in another bundle which would specify the
datasources and the displays. The sticky thing we haven't
addressed there is the notion of a realtime dataset. Currently,
the dataset used is tied to the actual file that was used to
display the bundle.  We are working on this issue as well.

We've tried to design the IDV to be operated programatically
with the idea of using this to auto generate products (and
automate testing).  We will be looking at using the offscreen
display's in VisAD and other ways to do the type of scripting.
We use Jython extensively in the IDV, so using that for scripting
has always been an option.  It's just so many things to do, so
little time/resources.  We're always looking for collaborators to
work on enhancing the IDV, so anyone who wants to tackle this
before we get to it is welcome to take a shot. ;-)  Even if you
only have suggestions on how to accomplish this, let
us know.

Don
*************************************************************
Don Murray                               UCAR Unidata Program
dmurray@xxxxxxxxxxxxxxxx                        P.O. Box 3000
(303) 497-8628                              Boulder, CO 80307
http://www.unidata.ucar.edu/staff/donm
*************************************************************