Re: [thredds] nco as a web service

On Jun 18, 2012, at 4:19 PM, Doug Lindholm wrote:

> Hi,
> I've been implementing "filters" for our LaTiS server 
> ( using the OPeNDAP (DAP2) URL 
> "function" syntax. For example,
> It's not ideal, but it's a start. DAP2 says very little about functions. 
> Maybe DAP4 could say more.
> I'd also strongly suggest that people think about Functional Programming 
> paradigms. I suspect that such formalisms will lend themselves better than an 
> OO biased approach.

I've been thinking about just this topic for some time. I am pretty certain 
that a functional language is the way to go. While the DAP2 specification say 
little about server functions, Hyrax does come with a few and the parser & 
evaluator does support composing functions. In addition there are two groups 
now working on server functions that perform some significant operations: 
unstructured grid subsetting and satellite data reprojection (satellite corrds 
to mercator, ...).

I'd certainly be interested in talking about this stuff more.

> Doug
> On 6/18/12 3:56 PM, Roy Mendelssohn wrote:
>> But for a useful service, the form and syntax of the URL should be
>> independent of the mechanism that does the server side calculations
>> (which rules out SWAMP).  So for example, both Grads an F-TDS use the
>> same format in the URL to say that "this is an expression", but the
>> expressions themselves are platform specific.  That is not the way to
>> get overarching services.
>> Instead, we need agreement on how in the URL request we signal a
>> server-side function, the syntax of that function (independent of the
>> engine underneath) and a few simple functions to start  (say simple
>> data transformations, differencing and averaging on a dimension(s)).
>> Then the server back-end can parse the request and use Ferret or
>> Grads or NCO or Python or whatever is desired, and like with any good
>> service, the back end could change  without having any affect on the
>> URL or the user.
>> I know Matthew Arrott at least used to like the approach in Chapter
>> 12 of "Python Scripting for Computation Science".  But a lot of that
>> is the engine underneath.  I am more interested in the form of the
>> URL.  Get some agreement on that, and some real implementation could
>> proceed.
>> -Roy
>> On Jun 18, 2012, at 2:37 PM, Russ Rew wrote:
>>> Jeff,
>>>> However, we have to keep in mind performance ramifications. It
>>>> still takes a long time to move gigabytes of data across a
>>>> network. This brings up the importance of moving the computation
>>>> to the data, instead of moving the data to the computation. For
>>>> some data sets and many use cases remote access to data works
>>>> very well so things like brokering are tractable. However, for
>>>> *big* data sets (e.g., climate model output) we need to come up
>>>> with richer mechanisms (like the NCO on local data) to bring
>>>> computation to the data.
>>> See Daniel Wang's SWAMP (the Script Workflow Analysis for
>>> MultiProcessing), built on top of NCO:
>>> --Russ
>>> _______________________________________________ thredds mailing
>>> list thredds@xxxxxxxxxxxxxxxx For list information or to
>>> unsubscribe,  visit:
>> ********************** "The contents of this message do not reflect
>> any position of the U.S. Government or NOAA." **********************
>> Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS
>> Environmental Research Division Southwest Fisheries Science Center
>> 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097
>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address) voice:
>> (831)-648-9029 fax: (831)-648-8440 www:
>> "Old age and treachery will overcome youth and skill." "From those
>> who have been given much, much will be expected" "the arc of the
>> moral universe is long, but it bends toward justice" -MLK Jr.
>> _______________________________________________ thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx For list information or to unsubscribe,
>> visit:
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 

James Gallagher
jgallagher at

  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: