Re: [thredds] WMS servers side image cache

Hi Jay, Guan, Ethan, all,

Just to pick up on this old thread, sorry for my late arrival!  As Guan says, 
ncWMS does cache extracted data arrays: it doesn't cache images, but the arrays 
of extracted data that are used to generate images. This allows fast altering 
of image styling parameters like the colour map: the data don't have to be 
re-extracted to change the palette.

However, this isn't in THREDDS yet.  I think a big part of the problem is 
ensuring cache consistency - it's hard to know when an underlying dataset might 
have changed.  A partial solution could be found by setting a very short expiry 
time on items in the cache (say 1 minute), which might give some benefit for 
visual data browsing in Godiva2, while ensuring that the data were unlikely to 
be out of date.

With a bit more head-scratching this could probably be done - any more votes 
from the user community...?

Cheers, Jon


Message: 2
Date: Thu, 12 May 2011 13:20:54 -0600
From: Ethan Davis <edavis@xxxxxxxxxxxxxxxx>
To: thredds@xxxxxxxxxxxxxxxx
Subject: Re: [thredds] WMS servers side image  cache
Message-ID: <4DCC3316.100@xxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset=ISO-8859-1

Hi Jay,

The TDS does use ehcache in a number of ways. I believe mainly for tracking 
collections of real-time updated files on disk. John can go into the details if 
you're interested.

The TDS does not use the ncWMS data array caching code that Guan mentions 
below. Most of the current TDS caching is at the dataset level rather than at 
the level of the actual data.


On 5/12/2011 12:41 PM, Jay Alder wrote:
> There is a ehcache.xml spring config file in WEB-INF which looks 
> promising but I don't see anything about WMS in the cache documentation.
> figXMLFile.html#Cache_Locations
> My cache directories are essentially empty. Can someone confirm or 
> deny thredds WMS is using or could use ehcache? It would be great if I 
> could use ehcache to cache the data for my tiles (and have them 
> persist) so I can change the image styling on the fly (best of both worlds).
> On 05/12/2011 10:36 AM, Guan Wang wrote:
>> In the original ncWMS package, caching has been considered through 
>> Ehcache.
>> /resc/n
>> cwms/cache/
>> Not sure if this is still kept after the integration with THREDDS.
>> Guan
>> -----Original Message-----
>> From: thredds-bounces@xxxxxxxxxxxxxxxx 
>> [mailto:thredds-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jay Alder
>> Sent: Thursday, May 12, 2011 1:09 PM
>> To: THREDDS Users
>> Subject: [thredds] WMS servers side image cache
>> Hi, I'm playing around with using Thredds WMS using a tiling scheme 
>> similar to google maps and I was wondering if there was a way (or 
>> something that could be added in the future) to cache WMS maps from 
>> the thredds servlet? Since I'm using tiles, the requests are 
>> identical. I was thinking thredds would be able to hash (say md5) the 
>> request url and write the image to temporary folder. On the second 
>> request thredds would look in the cache based on the hash, and return 
>> the image if its there otherwise generate a new image.  Thredds would 
>> likely need to wipe the cache on a restart in case the data changed.
>> Using IDL I wrote some code that downloaded all the tiles at all 
>> levels for a particular dataset, but it took about 12 hours to run 
>> and generated 400,000+ tiles for only one variable. If thredds cached 
>> images it would be a more "on demand" approach rather than trying to 
>> pre generate millions of tiles. In the tiled world, caching would say 
>> a lot of server side processing (especially for high resolution 
>> datasets) and speed up the user experience.
>> -Jay Alder
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: