Hi all,
Just a quick note to emphasize a "use case" that I am especially interested
in. That is the case where an end user wants to invoke a server side
process from within an html document. Being able to specify the process
in a URL makes this possible.
On the other hand, having the user construct the URL by hand is not
practical. Roy's approach allows the user to set up the process using a
browser-based client to set up the process interactively and them offers
the resulting URL for the user to embed in a document. From the user
viewpoint, this combination is very powerful, but I'm not sure how much it
limits the complexity of the process that can be specified.
-- Ben
On Sun, Jul 1, 2012 at 11:55 PM, Tom Kunicki <tkunicki@xxxxxxxx> wrote:
>
> On (C) I definitely concur. I am not against simplicity and HTTP GET
> requests. I just want to make sure that the approach is discussed and that
> one doesn't fall into the trap of believing HTTP GET is a panacea of
> simplicity. These URLs that have been posted are pretty complex and aren't
> the kinds of things that anyone but expert users will be crafting by hand.
> There will be a client implementation in front of them and they will need
> to be updated if the server processing API behind them changes. In this
> case, the client implementation will have to change in tandem with the
> server side processing API. This will be true regardless of whether the
> request is GET, POST, PUT, etc. One benefit of GET is an embeddable link,
> to my knowledge this isn't easily done with POST or PUT.
>
> Our group uses WPS. We had issues with some holes with some
> implementation and the specification so we made a choice to join on to the
> WPS 2.0 SWG.
>
> There are advantages to the WPS specification. Implementations can list a
> set of supported operations and processes using the GetCapabilities request
> (a GET or POST, we use GET). Each process can be queried for it's API
> including supported inputs and outputs (name, mime-type and schema if xml)
> using a DescribeProcess request (GET or POST, we use GET). If you know the
> arguments and types you can parse the DescribeProcess response and
> automatically generate a UI. We have implemented this in JavaScript for
> our Web-based brokering services. There are python clients as well as an
> Arc plugin in-progress (completed?) by ERSI and 52n, also a qGIS plugin
> among others. Processes can be executed with an Execute request (a GET or
> POST request, we use POST). POST for us because we deal with some pretty
> complex inputs (WFS calls with server side geometry filtering by reference
> to a GET or POST request; or Base64 encoded shapefiles sent in-line).
> These would bump us into some URL len
> gth restrictions we have dealt with in the past. We don't have to use
> these complex inputs but since WPS offers this flexibility we are happy to
> leverage it. When we execute processes we have the options to execute them
> synchronously or asynchronously (and an implementation can control these
> options by advertising them per process.) We can query the executing
> process for it's completion state (POST, don't know if GET is possible as I
> haven't looked into it). We can request executions results in-line with
> the response or by reference. We provide inputs to WPS calls as the
> results of other WPS calls. WPS processing implementations can be complex
> or simple. Given our use cases we made an architectural decision to
> leverage some of the more advanced components of the specification. We've
> developed some complex processing that does some cool and useful things
> that we are able to leverage in other projects and share with other groups.
> With our processing endpoints we can a
> dd a process and have it automatically be displayed in our UIs. One of
> the benefits of WPS was processing end-points became self-documenting.
>
> Now, the WPS execute by GET is pretty tricky as it requires so double URL
> encoding. We are happy using POST and didn't delve too much into GET. If
> there was a need and someone wanted to look at this with me (ahem, Roy?) I
> would be more that happy to submit some change requests to simplify the
> specification for some use cases. In my experience with the OGC standards
> almost everything can be done with GET, it's when you get into the outlying
> use cases you have to represent your requests with POST.
>
> WPS is an OGC specification. I think the last 2 words of the previous
> sentence instantly turn people off. But there's some real value to the
> work that's been done. We've used it as a thin wrapper on process
> execution. Our initial cut at processing involved using simple GET-based
> services. We found we had to generate a whole suite of utility/supporting
> GET-based services relying on clients to perform operations with correct
> ordering. The architecture was becoming difficult to maintain and
> document. A large number of tasks have now been implemented with the OGC
> standards suite and available standards implementations. This has saved
> our group a lot of development time and in turn taxpayer dollars.
>
> Tom Kunicki
> Center for Integrated Data Analytics
> U.S. Geological Survey
> 8505 Research Way
> Middleton, WI 53562
>
>
>
> On Jul 1, 2012, at 11:34 PM, Gerry Creager wrote:
>
> > Roy,
> >
> > That's a good explanation, and one I can live with. However, I also
> agree with Jeff's later comments, that A) in general, the same interpreter
> can handle GET and POST, and B) file uploads can't happen with a GET.
> >
> > And, most important: C) KISS is a good mantra.
> >
> > I'll sit back and listen to the debate again.
> >
> > gerry
> >
> > On Sun, Jul 1, 2012 at 3:13 PM, Roy Mendelssohn <
> roy.mendelssohn@xxxxxxxx> wrote:
> > BTW - a discussion we have been having around these parts is can you do
> enough in the way of server-side functions without a POST (ie the URL
> defines the function). That is why I would like to hear more from people
> who are running F-TDS and GDS - how many requests do they get for server
> side functions, but is the usual response time and download for these
> request, how large are the usual expressions? And then contrast it with a
> WPS or WCPS approach. I clearly believe in one approach, but I would
> welcome people who are using some of these other approaches to describe
> what they have done, the benefits of doing things that way, and what it
> means for a client.
> >
> > Thanks,
> >
> > -Roy
> >
> > On Jul 1, 2012, at 11:25 AM, Dennis Heimbigner wrote:
> >
> > > Roy-
> > >
> > > > ... One comment. I think you misunderstood my point about
> > > > Matlab and R. I am not interested in Matlab specific
> > > > implementations. The point was because the URL completely
> > > > defines the request, I can implement scripts in any application
> > > > that can send an URL and receive a file in terms of functions
> > > > built-in to that application - that is my clients do not break as
> > > > the application or operating system change.
> > >
> > > Not quite sure I understand. This phrase "...receive a file in
> > > terms of functions built-in to that application" sounds
> > > like you are creating an association between functions defined
> > > on the client side and functions defined on the server side.
> > > Can you elaborate?
> > >
> > > > Why I strongly prefer, if it is at all reasonable, services that
> > > > only use GET, not POST.
> > >
> > > Again, that is only possible if you keep your requests
> > > short enough to not violate the URL length restrictions.
> > >
> > > =Dennis Heimbigner
> > > Unidata
> > >
> > >
> > >
> > > Roy Mendelssohn wrote:
> > >> Hi Dennis:
> > >> Thanks. One comment. I think you misunderstood my point about
> Matlab and R. I am not interested in Matlab specific implementations. The
> point was because the URL completely defines the request, I can implement
> scripts in any application that can send an URL and receive a file in terms
> of functions built-in to that application - that is my clients do not
> break as the application or operating system change.
> > >> While I understand why this occurred, a few years ago we had straight
> OPeNDAP implementations. We had a lot of users using scripts we developed
> for Matlab, running under Windows. Due to updates in both Windows and
> Matlab, the OPeNDAP files for Windows stopped working (at least for
> Matlab). We had a lot of users that were left stranded and stranded for
> quite a long time. Developing and maintaining clients, particularly
> clients that are working within an application for which you have to write
> code, very quickly becomes a non-trivial exercise.
> > >> Since we switched to a service where the URL completely defines the
> request, our Matlab and R scripts have survived quite nicely any number of
> updates both to the applications themselves and to the operating systems.
> That is because the clients now only use functions built into the
> applications.
> > >> Why I strongly prefer, if it is at all reasonable, services that only
> use GET, not POST.
> > >> -Roy
> > >> On Jun 28, 2012, at 1:03 PM, Dennis Heimbigner wrote:
> > >>>> I am old and slow, but suppose I am in OpeNDAP, are you proposing
> > >>>> to separate say constraint expressions and server-side function
> > >>>> requests basically the same (ie I just scan what is after each
> > >>>> comma) or do you propose some method that signifies in the URL
> > >>>> that what follows is an expression? In F-TDS and GDS the form of
> > >>>> the URL is:
> > >>> First, I am proposing to subsume DAP constraints.
> > >>> Second, I am proposing, like DAP, to put the expressions
> > >>> in the query part of the URL (i.e. after the '?').
> > >>>
> > >>>> http://machine:port
> /thredds/dodsC/dataset_expr_{dataset2,dataset3,...}{expression1;expression2;...}.URLsuffix?constraint
> > >>> So, I would rewrite this as something more-or-less like this:
> > >>> http://machine.../dataset?expression1,expression2,...
> > >>> Where the expressions would include the references to dataset2,
> dataset3,
> > >>> and the constraint.
> > >>>
> > >>>> BTW, the reason I have asked about the experience of people who
> > >>>> are using F-TDS and GDS on whether synchronous requests can cover
> > >>>> the large majority of cases, is because I am very partial to
> > >>>> systems where the URL completely defines the request, and hence
> > >>>> essentially use GET as the verb.
> > >>> The synchronous/asynchronous issue is, for me, a separable issue.
> > >>> I should note that GET has a limit on the size of URLS, so
> > >>> there needs to be ways to deal with that. Two possibilities are
> > >>> 1) use POST or PUT, or 2) provide a way to upload a long expression
> > >>> in parts USING multiple GETs.
> > >>>
> > >>>> The reason for this is long
> > >>>> experience. where client code has broken with changes in
> > >>>> operating system and/or application, fixes were slow in coming,
> > >>>> so many users were left with nothing working. In a system where
> > >>>> the URL completely defined the request, say ERDDAP, in Matlab:
> > >>>>
> > >>>>>> link='
> http://coastwatch.pfeg.noaa.gov/erddap/griddap/erdBAsstamday.mat?sst[(2010-01-16T12:00:00Z):1:(2010-01-16T12:00:00Z)][(0.0):1:(0.0)][(30):1:(50.0)][(220):1:(240.0)]
> ';
> > >>>>>> F=urlwrite(link,'cwatch.mat');
> > >>>>>>
> > >>>> Will get the related file, and the entire command is in Matlab,
> > >>>> no extra code required. The same in R is:
> > >>>>
> > >>>>>> download.file(url="
> http://coastwatch.pfeg.noaa.gov/erddap/griddap/erdBAsstamday.nc?sst[(2010-01-16T12:00:00Z):1:(2010-01-16T12:00:00Z)][(0.0):1:(0.0)][(30):1:(50.0)][(220):1:(240.0)]",
> destfile="AGssta.nc",mode='wb')
> > >>>>>>
> > >>>> again, "download.file" is an R command.
> > >>> I think that we do not want to be R/MATLAB specific
> > >>> in a proposal to put stuff in URLs. I would rather
> > >>> propose to allow uploading of R/MATLAB scripts to serve
> > >>> as additional, user-defined functions.
> > >>>
> > >>> I would prefer to
> > >>>> maintain this simplicity and cover 80% of the cases if possible,
> > >>>> than cover the rest but where more complex, application specific
> > >>>> code would have to be developed and maintained.
> > >>> Agreed. However my assumption is the the output of any function that
> > >>> is not assigned to a single-assignment variable will be returned as
> part
> > >>> of the response; but other ways of specifying this are possible
> within
> > >>> the functional framework I am proposing.
> > >>>
> > >>> =Dennis Heimbigner
> > >>> Unidata
> > >> **********************
> > >> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> > >> **********************
> > >> Roy Mendelssohn
> > >> Supervisory Operations Research Analyst
> > >> NOAA/NMFS
> > >> Environmental Research Division
> > >> Southwest Fisheries Science Center
> > >> 1352 Lighthouse Avenue
> > >> Pacific Grove, CA 93950-2097
> > >> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> > >> voice: (831)-648-9029
> > >> fax: (831)-648-8440
> > >> www: http://www.pfeg.noaa.gov/
> > >> "Old age and treachery will overcome youth and skill."
> > >> "From those who have been given much, much will be expected" "the arc
> of the moral universe is long, but it bends toward justice" -MLK Jr.
> >
> > **********************
> > "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> > **********************
> > Roy Mendelssohn
> > Supervisory Operations Research Analyst
> > NOAA/NMFS
> > Environmental Research Division
> > Southwest Fisheries Science Center
> > 1352 Lighthouse Avenue
> > Pacific Grove, CA 93950-2097
> >
> > e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> > voice: (831)-648-9029
> > fax: (831)-648-8440
> > www: http://www.pfeg.noaa.gov/
> >
> > "Old age and treachery will overcome youth and skill."
> > "From those who have been given much, much will be expected"
> > "the arc of the moral universe is long, but it bends toward justice"
> -MLK Jr.
> >
> > _______________________________________________
> > thredds mailing list
> > thredds@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
> >
> > _______________________________________________
> > thredds mailing list
> > thredds@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
