Re: [bww-users] Community-managed cloud storage services

NOTE: The bww-users mailing list is no longer active. The list archives are made available for historical reasons.

  • To: Mohan Ramamurthy <mohan@xxxxxxxx>
  • Subject: Re: [bww-users] Community-managed cloud storage services
  • From: Carlos Maltzahn <carlosm@xxxxxxxxxxxx>
  • Date: Sat, 13 Feb 2016 20:09:31 -0800
Reading the About page <http://occ-data.org/about/>:

> To better understand our role, it is helpful to divide projects in these 
> areas into three groups:
> Individual researchers and small projects typically do not need much 
> computing infrastructure and can either operate their own or use a public 
> cloud service provider such as Amazon.
> The OCC is designed to serve medium to large size research projects by 
> managing and operating a cloud computing infrastructure that can be shared 
> across these projects.
> Very large research projects, such as the LHC, the LSST, and the OOI, 
> typically develop their own dedicated computing infrastructure.
It sounds like OCC is building their own cloud infrastructure instead of 
leveraging commercial cloud providers. Also, the membership benefits 
<http://occ-data.org/images/occ-fees-2016.pdf> do not include actual usage of 
the infrastructure and is more about participating in a standardization effort.

I’m interested in models of "community-managed” cloud storage services where 
the management involves cost and usage (as opposed to operating the hardware 
infrastructure). But I couldn’t find anything on the OCC web site that 
addresses that. 

Or do I miss something? 

Carlos

> On Feb 11, 2016, at 2:49 PM, Mohan Ramamurthy <mohan@xxxxxxxx> wrote:
> 
> On 2/11/16 3:44 PM, Scott Collis wrote:
>> So is this along the same lines as AWS S3?
>> 
> Yes.
>> Does it still rely on a download and compute framework?
>> 
>> 
> At the moment, this is true but we are working to working to develop 
> data-proximate, server-side processing/analysis capabilities by moving our 
> wares (and client tools) to the cloud, and through the development and 
> implementation of DAP4 protocol that supports asynchronous computing 
> capabilities.
> 
> Mohan
>>> Mohan Ramamurthy <mailto:mohan@xxxxxxxx> February 11, 2016 at 4:42 PM
>>> Carlos,
>>> 
>>> Unidata is working with Open Commons Consortium ( 
>>> <http://occ-data.org/>http://occ-data.org/ <http://occ-data.org/>), which 
>>> provides "community-managed" cloud storage and computing services. At the 
>>> moment, Unidata's collaboration with OCC is focused on the NOAA Big Data 
>>> project, but we expect that to grow beyond the scope of that project.
>>> 
>>> Mohan
>>> 
>>> On 2/11/16 2:34 PM, Carlos Maltzahn wrote:
>>> 
>>> _______________________________________________
>>> bww-users mailing list
>>> bww-users@xxxxxxxxxxxxxxxx <mailto:bww-users@xxxxxxxxxxxxxxxx>
>>> For list information, to unsubscribe, or change your membership options, 
>>> visit: http://www.unidata.ucar.edu/mailing_lists/ 
>>> <http://www.unidata.ucar.edu/mailing_lists/>
>>> Carlos Maltzahn <mailto:carlosm@xxxxxxxx> February 11, 2016 at 4:34 PM
>>> All,
>>> 
>>> This is a request for examples of community-managed cloud storage services 
>>> where
>>> 
>>> “community-managed” means that the cost of the cloud storage service as 
>>> well as its usage is managed by an institution serving a (scientific 
>>> community), including very large communities such as earth sciences or 
>>> smaller ones such as numerical weather prediction, and
>>> “cloud storage services” are commercial, highly available “pay-as-you-go” 
>>> services that provide safe and economic storage of large amounts of data 
>>> and allow global sharing of that data controlled by the party who pays, but 
>>> disappear as soon as payment for these services stop.
>>> 
>>> Today commercial cloud storage services are readily available and 
>>> successfully hide the many technical challenges of highly available 
>>> long-term storage at very attractive cost. Cloud storage also provides an 
>>> excellent platform for naming and sharing large (and small) datasets which 
>>> is essential for collaboration and reproducibility in data-intensive 
>>> scientific disciplines. Yet science communities are slow to adopt cloud 
>>> storage. There are probably many reasons for that but one that I repeatedly 
>>> came across: the data stored in cloud storage disappears when funding for 
>>> the service runs out. 
>>> 
>>> If the availability of a particular data set depends on a single community 
>>> member's availability of funding, the likelihood of loosing data can be 
>>> quite high and makes cloud storage too brittle for a reliable medium for 
>>> scientific data. A better approach might be to make the availability of all 
>>> data sets depend on the availability of funding within an entire community. 
>>> Such an arrangement would benefit that community by facilitating data 
>>> sharing, collaboration, and maintaining greater reproducibility of 
>>> scientific results. 
>>> 
>>> But community-funded cloud storage has all the management challenges of a 
>>> commons. For example, how should the storage space be governed? How much 
>>> money should the community spend on cloud storage? How is the money raised 
>>> among the members of the community? How do communities prevent The Tragedy 
>>> of the Commons <https://en.wikipedia.org/wiki/Tragedy_of_the_commons>?
>>> 
>>> Please let me know of any examples you are aware of. Who is working on 
>>> this? Do examples exist with somewhat different definitions of 
>>> "community-managed" or "cloud storage services”?
>>> 
>>> Thanks,
>>> Carlos
>>> 
>>> 
>>> -- 
>>> Carlos Maltzahn
>>> Adjunct Professor   
>>> Computer Science Department
>>> University of California, Santa Cruz        
>>>  <http://users.soe.ucsc.edu/%7Ecarlosm/>http://users.soe.ucsc.edu/~carlosm/ 
>>> <http://users.soe.ucsc.edu/~carlosm/>
>>> _______________________________________________
>>> bww-users mailing list
>>> bww-users@xxxxxxxxxxxxxxxx <mailto:bww-users@xxxxxxxxxxxxxxxx>
>>> For list information, to unsubscribe, or change your membership options, 
>>> visit: http://www.unidata.ucar.edu/mailing_lists/ 
>>> <http://www.unidata.ucar.edu/mailing_lists/>
>> 
> 
> _______________________________________________
> bww-users mailing list
> bww-users@xxxxxxxxxxxxxxxx <mailto:bww-users@xxxxxxxxxxxxxxxx>
> For list information, to unsubscribe, or change your membership options, 
> visit: http://www.unidata.ucar.edu/mailing_lists/ 
> <http://www.unidata.ucar.edu/mailing_lists/>

  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the bww-users archives: