Re: [wcsplus] asynchronous response [was: Re: WCS 1.0+]

Hi Stefano, Paolo, all,

I agree "Asynchronous Access" is not very clear. "Store" is much clearer. Does "Persistent Storage" add some specific meaning for you to "storage"?

I agree that "Push" capabilities might be handy. However, I think it is loaded with difficulties. The main issue is that there will be many firewall issues to get around on the client side. Most firewalls are setup to allow outgoing HTTP requests and incoming responses but not incoming requests. The other issue is that it means a client needs to also be an HTTP server so it can accept HTTP POST requests. If there are a lot of clients trying to receive PUSH responses, we are back to the firewall which may only have one or two machines each with one or two ports open for HTTP. There are ways to deal with all these issues (e.g., you mention the possibility of an upload server (proxy?) that then deals with the client) but all the PUSH issue adds a lot of complexity to the already quite complex asynchronous issue. Because of this, I think we (WCS 1.0+) should skip the PUSH issue for now.

One comment on the response codes, I think we should use the 201 (Created) HTTP response code rather than 302 (Found) for the immediate/store case. I think the meaning of the 201 code (the request has caused a new resource to be created and it can be found at the given URI) more closely matches this case than the meaning of the 302 code (the resource is not at the request URI but can be found at the given redirect URI). A subtle difference perhaps but I think it is important to be careful that our mapping matches the standard meaning of the HTTP response codes.

I don't understand your delayed/non-stored/pull case. Isn't it implicitly the same as the delay/store/pull? The server starts processing on the first request, ignores any requests till it is done, and once finished stores the data till it is requested again. Why not use the 202 response to send information about when to check again and all the other stuff recommended as content in the 202 response body.

So, I would suggest we drop the PUSH option and drop the delay/no-store/pull case which drops us to three cases. I have a few more thoughts on these cases and another one we haven't mentioned yet. But I will save that for another email (later today, I hope).

Ethan

Stefano Nativi wrote:
Hi Ethan and all,

With Paolo we tried to expand this theme in order to come up with wider model and understand possible use cases. Starting from it, we tried to produce simple requirements to support most common use cases with WCS.

Hope could be useful.


--Stefano

P.S. I attached a Word document, let me know if you prefer a PDF version




Hi Stefano, all,

Yes, asynchronous response is part of WCS 1.0+ so I'm including your CR doc in this email.


John and I talked about asynchronous responses a bit the other day. Below are a few thoughts from our discussion (all of which are based in the REST/ROA realm rather than the SOAP/WS-* realm). I was going to suggest we continue this conversation but postpone implementation in WCS 1.0+ until later in the process. But with this CR being submitted, maybe it is worth moving forward sooner rather than later. Thoughts?

Use cases:
1) "Asynchronous Access" - The client wants the data stored for access in future step of chaining or some such. 2) "Asynchronous Response" - The server determines that it will take time to process and respond so wants to let client know to come back and get the data later.

Does anyone have more concrete use cases of the "Asynchronous Response" type? Jeremy, Bruce, this was on your list. What is(are) your use case(s)?

Negotiation:
1) For the "Asynchronous Access" use case, I think the current way a WCS server lets the client know that it can "store" the data is fine (i.e., AllowedValues: "True", "False").

2) For the "Asynchronous Response" use case, the client needs 1) a way to know if the server might make an asynchronous response and 2) a way to indicate to the server that it wants a request fulfilled even if it is as an asynchronous response. Perhaps "AllowAsync". So the server indicates that it might do an asynchronous response by specifying the "AllowAsync" parameter with AllowedValues of "True" and "False", similar to the "store" parameter.

I'm guessing the WCS.RWG won't like the extra parameter as they seem to lean towards simplicity by reduction of parameters. But I don't see a good way to handle both the use cases above without a parameter for each of the above use cases (e.g., "store" and "AllowAsync").

Response:

For the "Asynchronous Access" use case, I think what the WCS has now seems fine except I would like to specify that the HTTP response code will be a 201 (Created). I assume the body is already defined as a XML coverage/manifest containing URLs to the data.

For the "Asynchronous Response" use case: If the client leaves out "AllowAsync" from their request, the server must either respond synchronously or with an exception that points to missing parameter "AllowAsync". If the client specifies "allowAsync=False", the server must either respond synchronously or with an exception pointing to a bad parameter value in "AllowAsync". If the client specifies "allowAsync=True", the server may respond synchronously or asynchronously. Where the asynchronous response would be an HTTP response code of 202 (Accepted). As the CR discusses, the body of the response should contain some indication of current status, some way to monitor the status, and an estimate of when it will be done. Should the CR include some suggestions for XML encoding of this information? I'm sure there are examples of this kind of thing in the SOAP/WS-* realm but I'm not familiar with that.

Ethan

--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO 80307-3000 http://www.unidata.ucar.edu/ ---------------------------------------------------------------------------



Return-Path: <nativi@xxxxxxxxxxx>
X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on
        uni8.unidata.ucar.edu
X-Spam-Level:
X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00,
MIME_QP_LONG_LINE,RDNS_NONE,SUBJ_ALL_CAPS autolearn=no version=3.2.3
X-Original-To: edavis@xxxxxxxxxxxxxxxx
Delivered-To: edavis@xxxxxxxxxxxxxxxx
Received: from mail.prato.unifi.it (unknown [150.217.13.19])
        by laraine.unidata.ucar.edu (Postfix) with ESMTP id 3E76ECB18E;
        Wed,  3 Oct 2007 02:48:59 -0600 (MDT)
Received: from localhost (localhost [127.0.0.1])
        by mail.prato.unifi.it (Postfix) with ESMTP id 82764214001;
        Wed,  3 Oct 2007 05:32:58 +0200 (CEST)
Received: from mail.prato.unifi.it ([127.0.0.1])
by localhost (mail.prato.unifi.it [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id 27495-02; Wed,  3 Oct 2007 05:32:58 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
        by mail.prato.unifi.it (Postfix) with ESMTP id 0241A214002;
        Wed,  3 Oct 2007 05:32:58 +0200 (CEST)
Received: from telquad.imaa.cnr.it (eos.pin.unifi.it [150.217.13.78])
        by mail.prato.unifi.it (Postfix) with ESMTP id A2467214001;
        Wed,  3 Oct 2007 05:32:57 +0200 (CEST)
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Wed, 03 Oct 2007 10:48:28 +0200
To: "Tandy, Jeremy" <jeremy.tandy@xxxxxxxxxxxxxxxx>,
"Dominic Lowe" <d.lowe@xxxxxxxx>, "Ethan Davis" <edavis@xxxxxxxxxxxxxxxx>
From: Stefano Nativi <nativi@xxxxxxxxxxx>
Subject: RE: WCS 1.0+
Cc: "Woolf, A (Andrew)" <A.Woolf@xxxxxxxx>,
        "Ben Domenico" <Ben@xxxxxxxxxxxxxxxx>,
        "John Caron" <caron@xxxxxxxxxxxxxxxx>,
        "Lawrence, BN (Bryan)" <B.N.Lawrence@xxxxxxxx>,
        "Wright, Bruce" <bruce.wright@xxxxxxxxxxxxxxxx>
In-Reply-To: <3B4DB2B04B0DCA4484D767A775D89E6A81F3F3@xxxxxxxxxxxxxxxxxxx
        rd.metoffice.com>
References: <3B4DB2B04B0DCA4484D767A775D89E6A81F3F3@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Mime-Version: 1.0
Message-Id: <20071003033257.A2467214001@xxxxxxxxxxxxxxxxxxx>
X-Virus-Scanned: by amavisd-new at prato.unifi.it
Content-Type: multipart/mixed; boundary="=====================_7846813==_"; x-avg-checked=avg-ok-6DF89EF

All,

As anticipated, I attached the draft of the RFC we'd like to send to WCS-RWG for the asynchronous interaction.

Any comment, correction, contribution or support is very welcome.

I hope this could be useful for WCS 1.0+, as well. Anyway, I didn't send this to the mailing list because I'm not sure whether this topic was included in the WCS 1.0+


--Stefano

_______________________________________________
wcsplus mailing list
wcsplus@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/




--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
---------------------------------------------------------------------------