Re: [netcdfgroup] NetCDF on Databricks

• To: Shannon Sheppard <dbashep@xxxxxxxxx>
• Subject: Re: [netcdfgroup] NetCDF on Databricks
• From: "H. Joe Lee" <hyoklee@xxxxxxxxxxxx>
• Date: Fri, 23 Sep 2022 10:07:11 -0500
• Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=hdfgroup.org; dmarc=pass action=none header.from=hdfgroup.org; dkim=pass header.d=hdfgroup.org; arc=none
• Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6fRWGyXcmcvuPxXyf8q1/i6wDN0wVpzLfEZ10QjCLTA=; b=COoPqD7bRZiXUb1RYg5DXPBbfoOPxrhf8Fbp8OBkZmhWiwHdo+GeZDk00+QTWX6i2ZbWyHt6C930qz3P8V3q1i0t4tzb4ZEwslIrk0bocXmOhR+aP12f0sqqM1sTA31AZbwhQ80JBQCEe0weENtc9E3SNKqOagHIipX93lC2DpI8ikR9s4AalgJBni7BDZh0/Dvr5+MBH//k7hL5z9Ki9ahoybD/R6Qu1x350H6pjpOprqKh+nREiJxKmw0J6dhDcpqQPLkQDRAgfDm6m6BseTEhmxx1IlPra1EcE3VI59TEtxJQIO3Oxt0hVn9sm/oYcY4txr0YS8g7XvKqh3jwNg==
• Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Lhb3Is9YNCYJFyy1+I7yHNVLSDX1JtZwhtVP9O1IGwh7BjkVR4N1XKgvssRTw/9PlYsIcC/WHYdjLF+NawIKeX0hgJxV/VhJZh0ySZoVmQZyaKew4efPXilW+uZvJEpIvyHx8Eb8eiphlZBgSRnsfTbhz9Ehb5k3lwU24fAi+SyIMUR2qtxj06xjW1F1OfKxQKdYSQIn3V+ZwVWaADnJM7476+1zp81g12MdpmsIBwx8IeuElMGrVrGn7QEpX8Ai/ldd84hwQRE06FtxyGmNUOt1igoo9qjJEg+N7dgQw1yR1un6h+jAyVwePte9krpoWtOgJYiC4M7DhLEwz8FyLQ==
• Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=hdfgroup.org;

Hi, Shannon!

I hope the following example can help you if you haven't seen it already:

https://hdfeos.org/examples/spark.php

Also, please check the following URL if you deal with netCDF-4/HDF5:

https://github.com/HDFGroup/hdf5-spark-connector

- Joe
---
Build the Metaverse w/ HDF.



On Thu, Sep 22, 2022 at 12:54 PM Shannon Sheppard via netcdfgroup <
netcdfgroup@xxxxxxxxxxxxxxxx> wrote:

> Hello Gang,
>
> Is it possible to run NetCDF on Databricks or Spark?  If there is it does
> not appear to be obvious.  There does not appear to be a package for
> processing these fils on Spark, at least not Spark 3.0+
>
> Shep
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> https://www.unidata.ucar.edu/mailing_lists/
>

• References:
  • [netcdfgroup] NetCDF on Databricks
    • From: Shannon Sheppard