Re: [thredds] aggregation / arithmetic between variables

Hi Roland:

Just curious, and this may be way off the target.  when you create the virtual 
variable, are you defining WMS as a service that is available to it in however 
you create the virtual catalog entry.  All the  other examples use OpeNDAP.  
Has anyone tested WCS or nay other service to se if that works?  That should 
tell a lot.

-Roy



On Apr 15, 2011, at 8:30 AM, Roland Schweitzer wrote:

> Hi,
> 
> I'd like to take up this conversation again and really this is a question for 
> Jon Blower and the ncWMS folks.
> 
> Benno reported privately that the previous example had a few problems with 
> just the raw data, so I've put up what I hope is a very simple example we can 
> use to work through the issues.
> 
> The data URL is: 
> http://cirrus.handwx.com:8080/thredds/dodsC/examples/average.jnl
> which consists of a monthly climatology and one transformed variable 
> (sst_average) a dynamically generated average of all 12 months of the sst 
> variable.
> 
> We can plot the sst_average with our standard tools (Ferret, toolsUI, etc), 
> but the WMS request for GetCapabilities fails (just following the link in the 
> browser) with the following error.
> 
> Jon, any suggestions for how to track down this error?
> 
> Roland
> 
> 2011-04-15T15:15:20.028 +0000 [     14083][      19] INFO  - 
> thredds.server.wms.ThreddsWmsController - Remote host: 74.192.0.105 - 
> Request: "GET /thredds/wms
> /examples/average.jnl?service=WMS&version=1.3.0&request=GetCapabilities 
> HTTP/1.1"
> 2011-04-15T15:15:20.939 +0000 [     14994][      19] ERROR - 
> thredds.server.wms.ThreddsWmsController - dispatchWmsRequest(): Exception:
> java.lang.NullPointerException
>    at 
> uk.ac.rdg.resc.ncwms.cdm.CdmUtils.getOptimumDataReadingStrategy(CdmUtils.java:221)
>    at thredds.server.wms.ThreddsDataset.<init>(ThreddsDataset.java:102)
>    at 
> thredds.server.wms.ThreddsWmsController.dispatchWmsRequest(ThreddsWmsController.java:164)
>    at 
> uk.ac.rdg.resc.ncwms.controller.AbstractWmsController.handleRequestInternal(AbstractWmsController.java:195)
>    at 
> org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
>    at 
> org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
>    at 
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
>    at 
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
>    at 
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
>    at 
> org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
>    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>    at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>    at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>    at 
> thredds.servlet.filter.RequestQueryFilter.doFilter(RequestQueryFilter.java:121)
>    at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>    at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>    at 
> thredds.servlet.filter.RequestPathFilter.doFilter(RequestPathFilter.java:105)
>    at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>    at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>    at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>    at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>    at 
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
>    at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>    at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>    at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>    at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>    at 
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
>    at 
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>    at java.lang.Thread.run(Thread.java:662)
> 2011-04-15T15:15:20.940 +0000 [     14995][      19] INFO  - 
> thredds.server.wms.ThreddsWmsController - Request Completed - 500 - -1 - 912
> 
> 
> 
> 
> On 04/12/2011 09:38 PM, Jay Alder wrote:
>> Have there been any updates on this? It would be ideal if I could use F-TDS 
>> to create anomalies between two datasets and plot them via WMS. It is 
>> possible to do this offline and add the product NetCDF files to the thredds 
>> catalog, but there are so many permutations its would make my thredds 
>> catalog very verbose/cluttered.
>> 
>> On Apr 1, 2011, at 4:11 PM, Roland Schweitzer wrote:
>> 
>>> On 04/01/2011 11:12 AM, Jay Alder wrote:
>>>> While we're on the topic, can F-TDS serve WMS or just OpENDAP? Essentially 
>>>> what I'd like to do is get a WMS map for dataset A - dataset B.
>>> A virtual variable in F-TDS behaves like any other netCDF variable being 
>>> served by TDS, so if it has the characteristics of a variable for which WMS 
>>> works it will also be available via WMS.  At least in theory.
>>> 
>>> However, when we tried it ourselves we get this error from the WMS software:
>>> 
>>> 2011-04-01T15:50:05.338 -0700 [    326689][      26] ERROR - 
>>> thredds.server.wms.ThreddsWmsController - dispatchWmsRequest(): Exception:
>>> java.lang.NullPointerException
>>>         at 
>>> uk.ac.rdg.resc.ncwms.cdm.CdmUtils.getOptimumDataReadingStrategy(CdmUtils.java:221)
>>>         at thredds.server.wms.ThreddsDataset.<init>(ThreddsDataset.java:102)
>>>         at 
>>> thredds.server.wms.ThreddsWmsController.dispatchWmsRequest(ThreddsWmsController.java:164)
>>>         at 
>>> uk.ac.rdg.resc.ncwms.controller.AbstractWmsController.handleRequestInternal(AbstractWmsController.java:195)
>>>         at 
>>> org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
>>>         at 
>>> org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
>>>         at 
>>> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
>>>         at 
>>> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
>>>         at 
>>> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
>>>         at 
>>> org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>         at 
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>>>         at 
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>>>         at 
>>> thredds.servlet.filter.RequestQueryFilter.doFilter(RequestQueryFilter.java:121)
>>>         at 
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>>>         at 
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>>>         at 
>>> thredds.servlet.filter.RequestPathFilter.doFilter(RequestPathFilter.java:105)
>>>         at 
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>>>         at 
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>>>         at 
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>>>         at 
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>>>         at 
>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
>>>         at 
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>>         at 
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>>         at 
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>>         at 
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>>>         at 
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
>>>         at 
>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>>>         at 
>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>>>         at java.lang.Thread.run(Thread.java:662)
>>> 
>>> The two netCDF URL's are:
>>> 
>>> Original data set:
>>> http://ferret.pmel.noaa.gov/geoide/carbontracker.html?dataset=ct_flux
>>> 
>>> Data set with a new variable (ft1000):
>>> http://ferret.pmel.noaa.gov/geoide/catalog/las/catalog.html?dataset=las/test_ftds_wms.jnl
>>> 
>>> Roland
>>> 
>>>> Thanks
>>>> 
>>>> On Apr 1, 2011, at 8:40 AM, Roy Mendelssohn wrote:
>>>> 
>>>>> yes it can.  Instructions are at that link.  I've done it several times 
>>>>> and can help people walk through it once they get Ferret installed.
>>>>> 
>>>>> -Roy
>>>>> 
>>>>> On Apr 1, 2011, at 8:37 AM, Kevin Manross wrote:
>>>>> 
>>>>>> Kyle,
>>>>>> 
>>>>>> I'm interested in the same thing.  Can F-TDS be run without the entire 
>>>>>> LAS system?
>>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>> -kevin.
>>>>>> 
>>>>>> 
>>>>>> On Fri, Apr 1, 2011 at 10:23 AM, Kyle Wilcox<KWilcox@xxxxxxxxxxxxxx>  
>>>>>> wrote:
>>>>>> That functionality doesn't exists in TDS.  You will need to postprocess 
>>>>>> the files, or use F-TDS (Ferret TDS) to accomplish:
>>>>>> 
>>>>>> http://ferret.pmel.noaa.gov/LAS/documentation/the-ferret-thredds-data-server-f-tds/
>>>>>> 
>>>>>> ---------
>>>>>> Kyle Wilcox, Engineer
>>>>>> Applied Science Associates
>>>>>> 55 Village Square Drive
>>>>>> South Kingstown, RI 02879
>>>>>> p: (401) 789-6224
>>>>>> e: kwilcox@xxxxxxxxxxxxxx
>>>>>> 
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: thredds-bounces@xxxxxxxxxxxxxxxx [mailto:thredds-
>>>>>>> bounces@xxxxxxxxxxxxxxxx] On Behalf Of Ivan PRICE
>>>>>>> Sent: Friday, April 01, 2011 11:14 AM
>>>>>>> To: thredds@xxxxxxxxxxxxxxxx
>>>>>>> Subject: [thredds] aggregation / arithmetic between variables
>>>>>>> 
>>>>>>> 
>>>>>>> Hi again,
>>>>>>> 
>>>>>>> I'm sorry to ask so many questions, i am looking around before asking
>>>>>>> these..
>>>>>>> 
>>>>>>> My next problem is that I'd like to define a 'virtual variable' that is
>>>>>>> the result of simple operations on existing variables, for example a
>>>>>>> new
>>>>>>> variable that is the sum of 2 existing ones.
>>>>>>> 
>>>>>>> I have read all about aggregating between datasets and over dimensions
>>>>>>> such as time, but i think this is different. My goal is to have a
>>>>>>> virtual WMS layer that is the result of the relationship between
>>>>>>> existing variables.
>>>>>>> 
>>>>>>> Does this sort of functionality exist at this stage, or should i be
>>>>>>> pre-processing the dataset before the thredds stage ?
>>>>>>> 
>>>>>>> cheers and have great weekends
>>>>>>> 
>>>>>>> -i
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> thredds mailing list
>>>>>>> thredds@xxxxxxxxxxxxxxxx
>>>>>>> For list information or to unsubscribe,  visit:
>>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>> _______________________________________________
>>>>>> thredds mailing list
>>>>>> thredds@xxxxxxxxxxxxxxxx
>>>>>> For list information or to unsubscribe,  visit: 
>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -- 
>>>>>> +-----------------------------------------------------+
>>>>>> Kevin L. Manross           |  ** New Address **
>>>>>> CIMMS Research Associate   |     120 David L. Boren Bvd
>>>>>> NSSL : WRDD : SWAT         |     Rm 3923
>>>>>> 
>>>>>> <kevin.manross@xxxxxxxx>
>>>>>>   |     405.325.6385
>>>>>> 
>>>>>> www.cimms.ou.edu/~kmanross
>>>>>> |
>>>>>> "My opinions are my own and not representative of
>>>>>> CIMMS, NSSL, NOAA or any affiliates"
>>>>>> +-----------------------------------------------------+
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> thredds mailing list
>>>>>> thredds@xxxxxxxxxxxxxxxx
>>>>>> For list information or to unsubscribe,  visit: 
>>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>>> **********************
>>>>> "The contents of this message do not reflect any position of the U.S. 
>>>>> Government or NOAA."
>>>>> **********************
>>>>> Roy Mendelssohn
>>>>> Supervisory Operations Research Analyst
>>>>> NOAA/NMFS
>>>>> Environmental Research Division
>>>>> Southwest Fisheries Science Center
>>>>> 1352 Lighthouse Avenue
>>>>> Pacific Grove, CA 93950-2097
>>>>> 
>>>>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>>>>> voice: (831)-648-9029
>>>>> fax: (831)-648-8440
>>>>> www: http://www.pfeg.noaa.gov/
>>>>> 
>>>>> "Old age and treachery will overcome youth and skill."
>>>>> "From those who have been given much, much will be expected"
>>>>> 
>>>>> _______________________________________________
>>>>> thredds mailing list
>>>>> thredds@xxxxxxxxxxxxxxxx
>>>>> For list information or to unsubscribe,  visit: 
>>>>> http://www.unidata.ucar.edu/mailing_lists/
>>>> _______________________________________________
>>>> thredds mailing list
>>>> thredds@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe,  visit: 
>>>> http://www.unidata.ucar.edu/mailing_lists/
>>> _______________________________________________
>>> thredds mailing list
>>> thredds@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe,  visit: 
>>> http://www.unidata.ucar.edu/mailing_lists/
> 
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/ 

**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"