[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20000208: More on ADDE from MSFC and imagery in the UW stream



>From: Gilbert Sebenste <address@hidden>
>Organization: NIU
>Keywords: 200001311730.KAA02738 McIDAS ADDE accounting

Gilbert,

I am responding to your last question about loading loops of images.

>Thanks again for letting me have access to the images! They are WAY cool!
>
>I was wondering if somehow, they can be shared with the UNIDATA community.
>Now that the data is "compressed" on the fly, how much of a bandwidth
>issue is it? I would love to have other Universities share in what you've
>given me...to give students an opportunity to do some mesometeorology
>using satellite data. On the regular UNIDATA datastream, you can't do it
>well, since the images only come across once an hour, and the resolution
>is 4 KM for visible images.
>
>Of course, I don't want to annhilate your server or network either...but I
>would assume that they wouldn't be used continuously. And even if they
>are, how much of a bandwidth issue is it, either way?
>
>Hopefully the compression will help on your end with your users, in
>bringing up the data faster. It definitely helps here...when I turn
>compression off, there is a HUGE difference in download times for each
>image.
>
>Second question...is there a way to bring up, say, the last 6 images for a
>loop using IMGDISP? Is there a command like, say, "IMGLOOP" to do this?

Yes, IMGDISP has the ALL= keyword that allows you to load several images.
As an example:

IMGDISP RTIMAGES/GE-IR STA=KMIA MAG=2 EU=IMAGE SF=YES REFRESH='EG (GRA);MAP H 
GRA=(GRA)' ALL=1 10

This command will load 10 images from the RTIMAGES/GE-IR dataset starting
in frame 1 and continuing through frame 10.  The load will be centered
on Miami, FL and blown up by a factor of two.  The IMAGE enhancement will
be used for each frame and a high resolution map will be drawn on top of
the image in each frame.

Some important items in the above command line:

o ALL=n m  - check out the online McIDAS help for IMGDISP or review the
  manual entry for it in the online McIDAS-X Users Guide

o SF=YES   - force flipping to the frame after the image is displayed;
  if you didn't do this; and if you forgot to tell the EG and MAP commands
  where to do their thing, then the erasure and map drawing would be done
  on the frame that you are currently looking at AND that frame would
  not change

o EG (GRA) - says to erase the current graphic frame; this changes as
  each image gets loaded by virtue of the SF=YES keyword

o MAP H GRA=(GRA) - tell MAP explicitly which frame to draw the map on

The shortcomings of IMGDISP for loading loops are:

o the loop bounds do not get set like they did in ALOOP

o since the loop bounds do not get set, the dwell rates also do not
  get set

o if you have fewer than 10 images, then only the first 'n' (where 'n'
  is the number you actually have) will get loaded; this makes it difficult
  to make a generic construct like 'ALL=1 10;LB=1 10;DR 9*3 10' since
  you don't know apriori that all of frames 1-10 should be included in a loop

We have at various times considered adding logic to IMGDISP to be aware
of how many frames were loaded and to set loop bounds and dwell rates.
The reason that we have not is that we are trying to decrease the number
of changes we have to make to the SSEC distribution each time we get one.
Given that there is now only me doing McIDAS support at Unidata, I have
to be a lot more careful on how much I hack into the SSEC code since each
new hack hardly ever goes away (read this to mean that SSEC hardly ever
picks up the mods that we make to McIDAS (a bit of an overstatement, but
reasonably true)).
        
>Muchas gracias and keep up the great work! I greatly appreciate it!!!

So, the big carrot for you in the MSFC imagery is the temporal
resolution for all imagery and higher spatial resolution for visible
imagery?  It must be since the IR and WV imagery that we send the
broadcast are at full resolution (modulo the removal of the 1.7 times
oversampling in the element dimension).

How many sites, with their current internet capabilities, do you think
could ingest twice the number of image products than they are currently
getting?  Also, how many sites do you think could ingest visible images
that are 4 (2 km res.) or 16 (1 km res.) times the size of the current
images?  I do not ask this to be argumentative, but, rather, to get one
user's perspective how much data can effectively be handled in the
current IDD.

Our feeling has been that the great majority of sites are having
problems getting the images in its current size.  To put some numbers
on these products, a current VIS GOES East of West image in the
broadcast is anywhere from 1.2 to 1.8 MB (assuming that it is not dark;
those images compress really well).  Is uncompressed size is on the
order of 2.3 MB.  From these numbers, you can see that we are only
getting a compression ratio of 75%.  Some experimentation at the UPC by
Steve Chiswell has shown that use of PNG compression can give us much
better compression numbers; broadcast products on the order of 35-40%
of their original size.

Even with these savings, this would translate into VIS products of the
following size:

resolution   product size   Comments
------------+--------------+---------
4 km         840 KB         current resolution
2 km         3.3 MB         double LINe and ELEment resolution
1 km        13.3 MB         quadruple LINe and ELEment resolution

Of course, this is the worst case scenario.  IR imagery is already being
sent out at its maximum resolution, and it compresses better than VIS
imagery.

So, what to do?

The first steps that we will be taking are:

o PNG compress new imagery as it is added to the Unidata-Wisconsin
  datastream

o after sites get PNG uncompression routines in place (e.g, a new ldm-mcidas
  decoder), switch the old images from delta encoding (the old standard
  McIDAS comprssion) to PNG compression

o figure out which is more important to the typical user:

  o the same images at twice the temporal resolution
  o bigger images at the same temporal resolution
  o bigger images at twice the temporal resolution
  o something different (ADDE access to high resolution imagery
    at top tier IDD sites?)

The ADDE option is attractive from the standpoint of allowing users to
go get as much imagery as they can handle, but has the disadvantage of
forcing sites not using McIDAS to use it to get desired imagery.  This
may be a bitter pill for some sites to swallow, but I don't know for
sure.

OK, enough for now.  Please let me know what you think about the above
issues.  The will be useful for the next User's Committee Meeting.

Tom