>From: "Barbara Hickman" <address@hidden> >Organization: NOAA >Keywords: 200210281539.g9SFd5q15745 Hi Barb, re: PNG compression techniques used at the UPC >Do you have any similar information on the timing delays per product >introduced with PNG? No, but my feeling is that they can't be very long. The system we use for NOAAPORT ingest (one we built ourselves) has very little buffering (4 MB), and that buffering is used by all 4 channels. This means that -- at a maximum --, 3.5 T1s of data is pouring into that 4 MB buffer, and no products are being lost during ingest. To me, this says that that the on-the-fly PNG compression that we are running (we are reading image products out of the ingest card buffer and compressing the image bytes as they become available) does not add any significant time delay to the injection of any NOAAPORT product into our IDD.\ I don't think that you can use our time delay experience to judge what the impact would be on your end, however. I can put together some numbers of how long it takes to PNG compress full NOAAPORT images stored in files if you like, but this will take a little time to pull together. I can more quickly put together some numbers on the time it takes to PNG compress images stored in McIDAS AREA files. My opinion is that our PNG compression will be significantly faster than wavelet compression techniques since it is a lot less CPU intensive. >I'm also curious as to where/how you obtained the >compression utility package (i.e. cost & source) The PNG stuff is like Zlib (and uses Zlib): it is available free-of-charge. The PNG Home Page (which is linked off of the Zlib Home Page) is: http://www.libpng.org/pub/png/ The following entry in the PNG FAQ page sheds light on PNGs availability: Q: I thought you said PNG was patent-free--what about Stac, PKWARE, Apple, etc.? A: The PNG image format was designed in 1995 specifically in response to the patent problems with the LZW algorithm used in GIF. To the best of the PNG group's knowledge, PNG was then--and still is--completely patent-free. However, the fact that is possible to implement PNG without infringing any known patents certainly does not imply that it is impossible to implement it without infringing any patents. Patents are a minefield, and several are known to be closely related to PNG technologies: o Stac, LZ77 sorted hash, deflate algorithm o PKWARE, whatever, deflate algorithm o Apple, alpha mask images >and how it is tuned/configured for your usage, We use a standard build of PNG in our compression routines. We have not done any additional tuning (yet) even though this would likely improve the compression ratios that we can get for the NOAAPORT products. Customization is an important point. The PNG Home Page gives several examples of how people have created custom filters that greatly improve compression of specific types of data. Some of the results listed are spectacular. Our compression application simply reads the input, writes out the unaltered header and then compresses the image data and appends it to the output. I include a schematic of the procedure at the end of this note. >type of platform on which it is run, Our PNG compression/uncompression routines run on all of the OSes we support for our applications: Compaq Tru64 5.1 FreeBSD 4.[57] HP-UX B 11.00 IBM AIX 4.3, 5.1 RedHat Linux 6.x, 7.x (we will look at 8.x soon) SGI IRIX, IRIX64 6.5.x Sun Solaris 5.[6789] (SPARC and x86) I have not tried this on Windows, but I can't say for sure that it would work there, but given that PNG compressed images are supported in Microsoft IE, I have to imagine that it would. >how the platform is equipped, etc. The code runs on the wide variety of platforms in use at our sites. This ranges from 200 Mhz Linux PCs with 64 MB of RAM to Sun Enterprise E450s and Sun Blade 1000, etc. >Your overall numbers look >consistent with our generalized product findings (Vis - least, WV - >most, product subsat, resolution, and coverages by channel also play >into the rates achieved, etc.) although the mean ratios do look >significantly better. I am glad that our results are consistent. >Right now, though, NWS is specifically working with zlib since they >believe that it is more readily avaialable and will have less impact to >both government AWIPS users and commercial NOAAport users. PNG uses Zlib compression techniques. Both packages are freely available and run on a very wide range of OSes (lots more than we support our applications on). Before dismissing PNG in favor of a Zlib-only approach, I would hope that you read through the PNG Home Page information and consider that display of PNG compressed imagery is supported in virtually all current web browsers. As far as the last comment goes, our sites can strip off the uncompressed header of the PNG compressed files we send in our IDD and look at the images in their web broswer. This makes it extremely easy for them to take quick looks at imagery without even uncompressing it. Our PNG compression applications effectively do the following: NOAAPORT GINI: +-----------------------+ +-------------------------+ | Header | | Header | +-----------------------+ +-------------------------+ | | | | | | PNG | | | Uncompressed | =====> | Compressed | | Data | | Data | | | | | | | | | | | +-------------------------+ | | | | | | | | | | +-----------------------+ McIDAS AREA: +-----------------------+ +-------------------------+ | Header | | Header | +-----------------------+ +-------------------------+ | Nav | | Nav | +-----------------------+ PNG +-------------------------+ | Cal | =====> | Cal | +-----------------------+ +-------------------------+ | AUX | | AUX | +-----------------------+ +-------------------------+ | | | Comment Cards | | | +-------------------------+ | | | | | | | | | Uncompressed | | Compressed | | Data | | Data | | | | | | | | | | | +-------------------------+ | | | | | | | | | | +-----------------------+ | Comment Cards | +-----------------------+ We keep both the GINI and AREA header information uncompressed so that the header information can be used without uncompressing it first. For instance, a McIDAS ADDE server for either only needs the header information to do things like IMGLISTs. The data only has to be uncompressed when it is needed by things like IMGDISP, IMGCOPY, IMGREMAP, etc. Tom > -- +-----------------------------------------------------------------------------+ * Tom Yoksas UCAR Unidata Program * * (303) 497-8642 (last resort) P.O. Box 3000 * * address@hidden Boulder, CO 80307 * * Unidata WWW Service http://www.unidata.ucar.edu/* +-----------------------------------------------------------------------------+
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.