Hi Jon,
Thank you for the reply.
As for the performance issue on ncWMS, I did experience slow rendition
through WMS. By talking to some developers who faced similar issues, I can
tell it appears to only happen on un-regular grids. My understanding is when
rending an un-regular grid to an image, re-sampling data to regular grid
takes a lot of CPU usage.
I had the test on one of our production rack server: a WMS request from a
single client in the multiple tile mode could easily beat the CPU to the
peak. It would perform better with Single tile mode WMS. But still can't
satisfy the need in a web environment (no benchmark though).
Let me know if you need my details.
Thanks,
Guan
-----Original Message-----
From: Jon Blower [mailto:j.d.blower@xxxxxxxxxxxxx]
Sent: Sunday, September 11, 2011 3:08 PM
To: gwang@xxxxxxx; thredds@xxxxxxxxxxxxxxxx
Subject: RE: [thredds] WMS servers side image cache
Hi Guan,
Sorry for the slow reply. In your point 3, do you mean that ncWMS's
algorithm for resampling needs to be faster? For which datasets do you find
it slow?
I agree that a user-configured cache would be useful, at the expense of some
complexity. Unidata guys - what do you think about this?
Cheers, Jon
-----Original Message-----
From: Guan Wang [mailto:gwang@xxxxxxx]
Sent: 19 August 2011 17:20
To: 'Jon Blower'; thredds@xxxxxxxxxxxxxxxx
Subject: RE: [thredds] WMS servers side image cache
Hi Jon,
Thank you for the follow up on this one!
Here is my wish-list on caching:
1. Cache for historical datasets is a good idea as long as the costs are
acceptable;
2. Cache for current datasets that subject to change could be offered in a
way that user can set the timer on the expiry time;
3. Cache image is not necessary. But the improvement on algorithm in ncWMS
for data re-sampling/interpolation (alternative could be through cache??) is
needed (correct me if this has been done in the new release against
irregular grids);
4. In order to improve the hit rate on cache and limit the consumption on
the resources, "user controlled cache" may be necessary. Finer configuration
levels could be offered, say only caching the dataset for a particular
period, in a specific area or certain parameters;
Thanks,
Guan
-----Original Message-----
From: thredds-bounces@xxxxxxxxxxxxxxxx
[mailto:thredds-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jon Blower
Sent: Friday, August 19, 2011 11:50 AM
To: thredds@xxxxxxxxxxxxxxxx
Subject: Re: [thredds] WMS servers side image cache
Hi Jay, Guan, Ethan, all,
Just to pick up on this old thread, sorry for my late arrival! As Guan
says, ncWMS does cache extracted data arrays: it doesn't cache images, but
the arrays of extracted data that are used to generate images. This allows
fast altering of image styling parameters like the colour map: the data
don't have to be re-extracted to change the palette.
However, this isn't in THREDDS yet. I think a big part of the problem is
ensuring cache consistency - it's hard to know when an underlying dataset
might have changed. A partial solution could be found by setting a very
short expiry time on items in the cache (say 1 minute), which might give
some benefit for visual data browsing in Godiva2, while ensuring that the
data were unlikely to be out of date.
With a bit more head-scratching this could probably be done - any more votes
from the user community...?
Cheers, Jon
------------------------------
Message: 2
Date: Thu, 12 May 2011 13:20:54 -0600
From: Ethan Davis <edavis@xxxxxxxxxxxxxxxx>
To: thredds@xxxxxxxxxxxxxxxx
Subject: Re: [thredds] WMS servers side image cache
Message-ID: <4DCC3316.100@xxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset=ISO-8859-1
Hi Jay,
The TDS does use ehcache in a number of ways. I believe mainly for tracking
collections of real-time updated files on disk. John can go into the details
if you're interested.
The TDS does not use the ncWMS data array caching code that Guan mentions
below. Most of the current TDS caching is at the dataset level rather than
at the level of the actual data.
Ethan
On 5/12/2011 12:41 PM, Jay Alder wrote:
> There is a ehcache.xml spring config file in WEB-INF which looks
> promising but I don't see anything about WMS in the cache documentation.
>
> http://www.unidata.ucar.edu/projects/THREDDS/tech/reference/ThreddsCon
> figXMLFile.html#Cache_Locations
>
>
> My cache directories are essentially empty. Can someone confirm or
> deny thredds WMS is using or could use ehcache? It would be great if I
> could use ehcache to cache the data for my tiles (and have them
> persist) so I can change the image styling on the fly (best of both
worlds).
>
>
>
> On 05/12/2011 10:36 AM, Guan Wang wrote:
>> In the original ncWMS package, caching has been considered through
>> Ehcache.
>>
>> http://www.resc.rdg.ac.uk/trac/ncWMS/browser/trunk/src/java/uk/ac/rdg
>> /resc/n
>>
>> cwms/cache/TileCache.java
>>
>> Not sure if this is still kept after the integration with THREDDS.
>>
>> Guan
>> -----Original Message-----
>> From: thredds-bounces@xxxxxxxxxxxxxxxx
>> [mailto:thredds-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jay Alder
>> Sent: Thursday, May 12, 2011 1:09 PM
>> To: THREDDS Users
>> Subject: [thredds] WMS servers side image cache
>>
>> Hi, I'm playing around with using Thredds WMS using a tiling scheme
>> similar to google maps and I was wondering if there was a way (or
>> something that could be added in the future) to cache WMS maps from
>> the thredds servlet? Since I'm using tiles, the requests are
>> identical. I was thinking thredds would be able to hash (say md5) the
>> request url and write the image to temporary folder. On the second
>> request thredds would look in the cache based on the hash, and return
>> the image if its there otherwise generate a new image. Thredds would
>> likely need to wipe the cache on a restart in case the data changed.
>>
>> Using IDL I wrote some code that downloaded all the tiles at all
>> levels for a particular dataset, but it took about 12 hours to run
>> and generated 400,000+ tiles for only one variable. If thredds cached
>> images it would be a more "on demand" approach rather than trying to
>> pre generate millions of tiles. In the tiled world, caching would say
>> a lot of server side processing (especially for high resolution
>> datasets) and speed up the user experience.
>>
>> -Jay Alder
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
