Re: [thredds] [opendap-tech] A request for server developers

On Thu, Apr 25, 2013 at 7:46 AM, Benno Blumenthal <benno@xxxxxxxxxxxxxxxx>wrote:

> So if we can persuade you of the value of Cross-Origin access, and me of
> the value of the discussion despite detours, we will be all set.
>

Until something else appears, CORS will be required if we want the data to
be fully accessible. i agree that making it on for all hosts by default is
maybe not the best solution, but it definitely should be supported one way
of another.


> Seems like we need a security person to formulate an accurate statement of
> risk to counterbalance what one might find on the web...
>

I found some more information here: http://enable-cors.org/. The site
includes a list of APIs that supports CORS, including Amazon S3, Google,
Github, Dropbox and Facebook: http://enable-cors.org/resources.html. And
their message is "If you serve public content, please consider using CORS
to open it up for universal JavaScript/browser access."


> Also, I have not been able to get cross-origin access to work in IE8, and
> I am not sure about more recent versions of IE.  So we need Microsoft to
> get its act together enough to upgrade users out of IE8, or fail utterly.
>

IE 10 has native support, but in IE 8 & 9 you need to use XDomainRequest
instead; here's how I do it:


http://code.dealmeida.net/opendap-streaming/src/9a589c780e1dcdd14a9d8468ace27ca0a4f5027d/static/js/dap.js?at=default#cl-72

Although I haven't tested the code on IE yet.

--Rob



> On Thu, Apr 25, 2013 at 10:24 AM, Gerry Creager - NOAA Affiliate <
> gerry.creager@xxxxxxxx> wrote:
>
>> Benno,
>>
>> You make a couple of good points about our missions of data sharing and
>> interoperability, and the fact that we shouldn't fear a security audit.
>> However, some of us DO live in an environment where this discussion has to
>> occur, and I disagree that the conversation's been hijacked.
>>
>> I don't see the ORIGIN "security" example as particularly pertinent: As
>> you point out, it's not really security, it's noise. However, that noise is
>> the first place my IT Security Officer is likely to find when I ask about
>> something like this... or the second if this conversation is already
>> indexed on Google.
>>
>> I don't live in fear of the security audits, in fact, I welcome them, as
>> an academic exercise in beating the system where rote methods are the
>> anticipated outcome. But in able to beat the audit, we, especially in
>> Federal service (or, as in my case, supporting it) have to completely
>> understand the implications of a change like this. And, my first sense,
>> upon reading Rob's request was, "No way: It opens too much." I may change
>> my mind upon further reflection and study, but not yet
>>
>> gerry
>>
>>
>> On Thu, Apr 25, 2013 at 9:08 AM, Benno Blumenthal <benno@xxxxxxxxxxxxxxxx
>> > wrote:
>>
>>> Sorry for the following bluntness, but this conversation has been
>>> hijacked.
>>>
>>> As a community, we are in the business of cross-platform, cross-site
>>> sharing of data.  As Robert points out, the cross-origin blocking in
>>> javascript prevents that.  Note that cross-origin blocking is a
>>> half-hearted attempt to lock a site's data to a site's apps, not a
>>> fundemental security flaw.  In fact, javascript allows cross-site code
>>> imports, while blocking cross-site data and cross-site formatting (css) --
>>> not a coherent security policy.
>>>
>>> There are security implications in everything we do, there are people
>>> who figure these things out, really that is what a security audit is for.
>>>  Ignorant fear of an audit is not a legitimate argument, one can manage a
>>> site that way, but please don't impose it on the rest of us -- I already
>>> have a finance dept that behaves that way.
>>>
>>> Bottom line is we need to make sure our data security is consistent with
>>> cross-origin data access -- which it probably is already.
>>>
>>>
>>> On Thu, Apr 25, 2013 at 7:54 AM, Lynnes, Christopher S. (GSFC-6102) <
>>> christopher.s.lynnes@xxxxxxxx> wrote:
>>>
>>>>  On Apr 24, 2013, at 8:56 PM, Roy Mendelssohn - NOAA Federal <
>>>> roy.mendelssohn@xxxxxxxx> wrote:
>>>>
>>>> > I don't know enough to add anything technical to the discussion, but
>>>> I know most of us these days operate in an environment of increased
>>>> security, and anything that generally opens up holes is frowned upon  ( we
>>>> have started restricting our http ,ethos to GET, POST and HEAD for 
>>>> example).
>>>> >
>>>> > I agree that more discussion of the security implications is needed,
>>>>   and the fact that someone has implemented it and hasn't seen anything
>>>> nefarious won't past muster on security scans.
>>>>
>>>> To reinforce Roy's comments, from the perspective of a U.S. Govt. data
>>>> provider, if the security implications are not crystal-clear, we have to
>>>> treat it as a potential increase in risk.  We already have an uphill battle
>>>> in getting any OPeNDAP server approved at all, security-wise.  As a result,
>>>> for our purposes, we're not comfortable with server distributions that
>>>> default to completely open CORS.
>>>>
>>>> I'm generally sympathetic to the desire to enable this kind of
>>>> functionality, but the history of computing is rife with cases where really
>>>> cool functionality clashed with the potential security implications.  I
>>>> also recognize that not all data providers operate within our security
>>>> constraints, so we will be interested in watching this unfold, with an
>>>> emphasis on watching.
>>>>
>>>> BUt note that absence of evidence doesn't mean it's safe, as Roy points
>>>> out.  The burden of proof for us is on proving from first principles that
>>>> there is no increase in vulnerability due to open CORS. That is, given a
>>>> malicious (inadvertent or otherwise) site, and malicious JS code using it
>>>> for a cross-site scripting, will it be crystal clear that our CORS policy
>>>> cannot be remotely associated with such an exploit?  In the govt., any kind
>>>> of guilt by association is almost as bad as actual guilt.
>>>>
>>>>
>>>> >
>>>> > -roy
>>>> >
>>>> > On Apr 24, 2013, at 5:40 PM, Dennis Heimbigner <dmh@xxxxxxxxxxxxxxxx>
>>>> wrote:
>>>> >
>>>> >> Turning this on by default is not a good idea.
>>>> >> As I indicated elsewhere, if data leaks, then unidata/opendap/...
>>>> >> are responsible and that makes us look really bad.
>>>> >>
>>>> >> =Dennis Heimbigner
>>>> >> Unidata
>>>> >>
>>>> >> Roberto De Almeida wrote:
>>>> >>> That was fast, Tom! How does this work, does it add an option for
>>>> turning
>>>> >>> on CORS on THREDDS?
>>>> >>> As I was saying to Dennis, what I'm suggesting here is: Javascript
>>>> is
>>>> >>> becoming a major player as a programming language, and if we want
>>>> our data
>>>> >>> to be widely accessed and consumed by the new generation of
>>>> applications in
>>>> >>> ways that we cannot anticipate, we should add these headers. If
>>>> security is
>>>> >>> important we can explain to users how to configure authentication,
>>>> how to
>>>> >>> setup SSL certificates, and how to turn off CORS (or restrict the
>>>> hosts
>>>> >>> instead of using a wildcard).
>>>> >>> I think this should be on by default (like I did in Pydap) because
>>>> >>> otherwise no one will turn it on, and all the data currently
>>>> available via
>>>> >>> OPeNDAP will be inaccessible to Javascript, because this is not a
>>>> policy
>>>> >>> that can be changed on the browser. But if we start doing it now,
>>>> providers
>>>> >>> will eventually upgrade their servers, and maybe next year we'll
>>>> have WebGL
>>>> >>> applications that can display data from any DAP server on the
>>>> browser in
>>>> >>> 3D, and analyse it using
>>>> >>> asm.js<http://ejohn.org/blog/asmjs-javascript-compile-target/>
>>>> >>> .
>>>> >>> --Rob
>>>> >>> On Wed, Apr 24, 2013 at 12:52 PM, Tom Kunicki <tkunicki@xxxxxxxx>
>>>> wrote:
>>>> >>>> I've created a maven project to create a CORS enable thredds war:
>>>> >>>>
>>>> >>>> https://github.com/tkunicki-usgs/thredds-cors
>>>> >>>>
>>>> >>>> CORS is useful for working around single origin issues in browser
>>>> apps.  A
>>>> >>>> CORS enabled server essentially tells the browser that it's ok to
>>>> let code
>>>> >>>> loaded from a different server utilize resources from the CORS
>>>> enabled
>>>> >>>> server.
>>>> >>>>
>>>> >>>> Tom Kunicki
>>>> >>>> Center for Integrated Data Analytics
>>>> >>>> U.S. Geological Survey
>>>> >>>> 8505 Research Way
>>>> >>>> Middleton, WI  53562
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> On Apr 24, 2013, at 2:11 PM, Dennis Heimbigner <
>>>> dmh@xxxxxxxxxxxxxxxx>
>>>> >>>> wrote:
>>>> >>>>
>>>> >>>> Perhaps more concretely, the thredds server
>>>> >>>> currently supports access controls such as
>>>> >>>> passwords and client-side keys. How would
>>>> >>>> CORS affect those?
>>>> >>>>
>>>> >>>> =Dennis Heimbigner
>>>> >>>> Unidata
>>>> >>>>
>>>> >>>> Roberto De Almeida wrote:
>>>> >>>>
>>>> >>>> Hi, guys!
>>>> >>>> In 2006 I wrote an implementation of an OPeNDAP client in
>>>> Javascript called
>>>> >>>> jsdap (https://code.google.com/p/jsdap/). At the time Javascript
>>>> was still
>>>> >>>> a toy language and the XML HTTP Request (XHR) was unable of
>>>> handling binary
>>>> >>>> data, but I managed to hack a full client that worked in all major
>>>> browsers
>>>> >>>> (including IE by injecting vbscript!). And while it was written
>>>> more as a
>>>> >>>> proof-of-concept the client is actually used in some data portals
>>>> like
>>>> >>>> http://www.ifremer.fr/oceanotronPortal/. (A Node.js OPeNDAP
>>>> server was
>>>> >>>> also
>>>> >>>> added 3 years ago.)
>>>> >>>> Fast forward 7 years and we now have a lot of new technologies on
>>>> the
>>>> >>>> table: a new XHR object with support for binary transfers, typed
>>>> arrays and
>>>> >>>> WebGL. I've been playing again with using Javascript as an OPeNDAP
>>>> client,
>>>> >>>> in particular to display real time information from OPeNDAP
>>>> servers. I have
>>>> >>>> set up a small OPeNDAP server on one of my VPS streaming the
>>>> system load
>>>> >>>> information:
>>>> >>>> http://vps.dealmeida.net:5000/.dds
>>>> >>>> http://vps.dealmeida.net:5000/.das
>>>> >>>> This is an infinite dataset (try "curl
>>>> http://vps.dealmeida.net:5000/.asc
>>>> >>>> "),
>>>> >>>> and it will keep streaming the data at one record per second until
>>>> the
>>>> >>>> connection is broken. Keep in mind that this is a regular OPeNDAP
>>>> Sequence,
>>>> >>>> and nothing was changed in the specification to make this work.
>>>> >>>> Nevertheless, I'm not aware of OPeNDAP clients that can access the
>>>> stream
>>>> >>>> other than the development version of Pydap.
>>>> >>>> On another machine I have a widget displaying the information on a
>>>> real
>>>> >>>> time graph: http://dealmeida.net/opendap-streaming/
>>>> >>>> You can see how everything was implemented on this Mercurial
>>>> >>>> repository<
>>>> >>>>
>>>> http://code.dealmeida.net/opendap-streaming/src/356dde80f6e55603c2ab7e581244015663504fda?at=demo
>>>> >>>>> .
>>>> >>>> The
>>>> >>>> data is displayed by fetching the .dods response and parsing it.
>>>> We still
>>>> >>>> need a few hacks to do this, but only because the data is being
>>>> streamed
>>>> >>>> (Mozilla handles it nice; Chrome cannot stream binary data, so it
>>>> still
>>>> >>>> fetches it as string). Handling regular OPeNDAP datasets should be
>>>> pretty
>>>> >>>> straightforward with the new XHR, and I plan to rewrite jsdap as
>>>> soon as I
>>>> >>>> have some free time.
>>>> >>>> *Now to my request:* the only reason that the demo works -- having
>>>> a page
>>>> >>>> in one host displaying data from an OPeNDAP server on another --
>>>> is because
>>>> >>>> I enabled<
>>>> >>>>
>>>> http://code.dealmeida.net/pydap/commits/4c2d38b5822ba8f5f61e83bcb23230a2ca7e5da1
>>>> >>>> CORS <http://en.wikipedia.org/wiki/Cross-origin_resource_sharing>
>>>> on
>>>> >>>> Pydap.
>>>> >>>> By default, now all DODS, DAS and DDS responses from Pydap have the
>>>> >>>> following additional headers:
>>>> >>>> Access-Control-Allow-Origin: *
>>>> >>>> Access-Control-Allow-Headers: Origin, X-Requested-With,
>>>> Content-Type
>>>> >>>> These headers (the first one, actually) allow the responses to be
>>>> accessed
>>>> >>>> through XHR from any host. As far as I know there is no downside
>>>> in doing
>>>> >>>> this. Even if your server use cookies for authenticating access to
>>>> certain
>>>> >>>> datasets the cookies *will not* be sent unless
>>>> >>>> the Access-Control-Allow-Credentials header is set (and set to
>>>> true), which
>>>> >>>> would allow other sites to "steal" your data and download it by
>>>> >>>> impersonating a logged user.
>>>> >>>> My request is that all OPeNDAP servers enable CORS from any host
>>>> by default
>>>> >>>> today, at least in the DODS, DAS and DDS responses; and if not by
>>>> default,
>>>> >>>> at least as an option. This way, by the time Javascript matures
>>>> enough so
>>>> >>>> that its performance on the browser becomes comparable to desktop
>>>> >>>> applications we can start building rich web applications that use
>>>> all the
>>>> >>>> data available through OPeNDAP.
>>>> >>>> Some resources
>>>> >>>> About CORS:
>>>> http://en.wikipedia.org/wiki/Cross-origin_resource_sharing /
>>>> >>>> https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS
>>>> >>>> Security concerns:
>>>> >>>>
>>>> https://code.google.com/p/html5security/wiki/CrossOriginRequestSecurity
>>>> >>>> Thank you,
>>>> >>>> Rob
>>>> >>>>
>>>> >>>>
>>>> >>>> _______________________________________________
>>>> >>>> thredds mailing list
>>>> >>>> thredds@xxxxxxxxxxxxxxxx
>>>> >>>> For list information or to unsubscribe,  visit:
>>>> >>>> http://www.unidata.ucar.edu/mailing_lists/
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> You received this message because you are subscribed to the Google
>>>> Groups
>>>> >>>> "pydap" group.
>>>> >>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an
>>>> >>>> email to pydap+unsubscribe@xxxxxxxxxxxxxxxx.
>>>> >>>> To post to this group, send email to pydap@xxxxxxxxxxxxxxxx.
>>>> >>>> Visit this group at http://groups.google.com/group/pydap?hl=en.
>>>> >>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>
>>>> >> --
>>>> >> You received this message because you are subscribed to the Google
>>>> Groups "OPeNDAP Tech" group.
>>>> >> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to opendap-tech+unsubscribe@xxxxxxxxxxx.
>>>> >> To post to this group, send email to opendap-tech@xxxxxxxxxxx.
>>>> >> Visit this group at
>>>> http://groups.google.com/a/opendap.org/group/opendap-tech/?hl=en.
>>>> >> For more options, visit
>>>> https://groups.google.com/a/opendap.org/groups/opt_out.
>>>> >>
>>>> >>
>>>> >
>>>> > **********************
>>>> > "The contents of this message do not reflect any position of the U.S .
>>>> Government or NOAA."
>>>> > **********************
>>>> > Roy Mendelssohn
>>>> > Supervisory Operations Research Analyst
>>>> > NOAA/NMFS
>>>> > Environmental Research Division
>>>> > Southwest Fisheries Science Center
>>>> > 1352 Lighthouse Avenue
>>>> > Pacific Grove, CA 93950-2097
>>>> >
>>>> > e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
>>>> > voice: (831)-648-9029
>>>> > fax: (831)-648-8440
>>>> > www: http://www.pfeg.noaa.gov/
>>>> >
>>>> > "Old age and treachery will overcome youth and skill."
>>>> > "From those who have been given much, much will be expected"
>>>> > "the arc of the moral universe is long, but it bends toward justice"
>>>> -MLK Jr.
>>>> >
>>>> > --
>>>> > You received this message because you are subscribed to the Google
>>>> Groups "OPeNDAP Tech" group.
>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to opendap-tech+unsubscribe@xxxxxxxxxxx.
>>>> > To post to this group, send email to opendap-tech@xxxxxxxxxxx.
>>>> > Visit this group at
>>>> http://groups.google.com/a/opendap.org/group/opendap-tech/?hl=en.
>>>> > For more options, visit
>>>> https://groups.google.com/a/opendap.org/groups/opt_out.
>>>> >
>>>> >
>>>>
>>>> --
>>>> Dr. Christopher Lynnes, NASA/GSFC, ph: 301-614-5185
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "OPeNDAP Tech" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to opendap-tech+unsubscribe@xxxxxxxxxxx.
>>>> To post to this group, send email to opendap-tech@xxxxxxxxxxx.
>>>> Visit this group at
>>>> http://groups.google.com/a/opendap.org/group/opendap-tech/?hl=en.
>>>> For more options, visit
>>>> https://groups.google.com/a/opendap.org/groups/opt_out.
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Dr. M. Benno Blumenthal          benno@xxxxxxxxxxxxxxxx
>>> International Research Institute for climate and society
>>> The Earth Institute at Columbia University
>>> Lamont Campus, Palisades NY 10964-8000   (845) 680-4450
>>>
>>> _______________________________________________
>>> thredds mailing list
>>> thredds@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe,  visit:
>>> http://www.unidata.ucar.edu/mailing_lists/
>>>
>>
>>
>>
>> --
>> Gerry Creager
>> NSSL/CIMMS
>> 405.325.6371
>> ++++++++++++++++++++++
>> “Big whorls have little whorls,
>> That feed on their velocity;
>> And little whorls have lesser whorls,
>> And so on to viscosity.”
>> Lewis Fry Richardson (1881-1953)
>>
>
>
>
> --
> Dr. M. Benno Blumenthal          benno@xxxxxxxxxxxxxxxx
> International Research Institute for climate and society
> The Earth Institute at Columbia University
> Lamont Campus, Palisades NY 10964-8000   (845) 680-4450
>
> --
> You received this message because you are subscribed to the Google Groups
> "OPeNDAP Tech" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to opendap-tech+unsubscribe@xxxxxxxxxxx.
> To post to this group, send email to opendap-tech@xxxxxxxxxxx.
> Visit this group at
> http://groups.google.com/a/opendap.org/group/opendap-tech/?hl=en.
> For more options, visit
> https://groups.google.com/a/opendap.org/groups/opt_out.
>
>
>



-- 
Roberto De Almeida, PhD
http://dealmeida.net/
:wq
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: