Re: [netcdfgroup] storing sparse matrices data in NetCDF

  • To: Ken Mankoff <mankoff@xxxxxxxxx>
  • Subject: Re: [netcdfgroup] storing sparse matrices data in NetCDF
  • From: Sourish Basu <Sourish.Basu@xxxxxxxxxxxx>
  • Date: Mon, 18 Mar 2019 14:12:58 -0600
  • Autocrypt: addr=Sourish.Basu@xxxxxxxxxxxx; prefer-encrypt=mutual; keydata= mQINBFZ40gQBEADvGCqRa5XGgaROg6TYUieMAh5GDTy6lclqxdKqu4oYSROUFYkEuT4tOHpV 4k6Ruhg1EYXMl0siTQ4VsTcvaFBR3RLKiOdRxsh4jPVrZI1TWJPJRWlvNg5iPXczTkH3diyA 2Pp4CBNfpw/M5uUHMgfL/A+1AAT0ciMUq9eR5U8bcjvfemg6Js/+tsNOwyHGlZEEXOrod/eb NrqOB1FA0WFHvEkFgJ1Ed/g3ulu0ylel0HoC/rCv0pU/PX+wNqucQbk00xjOw8ts02keX2z+ LQumHPWfSNrQpPh5u4L6XLAcc0RjEig6WkwHJQtjdEIoI+TNXtrDdQ09yrOg1dgQwz/kgnp4 oLBLjs2K3XMymsSorPCcInAoG3kchRadsmv69WX+YxIWPJaKOVrNs5K7Jf0nX9cyGe5Q1XuW 8sbk7IGKux9sy0S9HaYqU/w6HUhGl85522ogCeyZ7xnZoKuPthHGOBULsS8YD7BqYrxgNwnA JsDgefkSqLUiKixie/Tb8V/dSDpFfXEm6/ixNPm6iXZAza8ZVreQqQ3gs44VIHZi1c0qHtC0 kP2dx3IujfkdeUL3g2m84GMHmeG5Q207P50rqWk0kPzHRu5xMDQYICQ4wJwaEo69oM776sQ7 YxRXmFqzo2UX45rTVoq5xaOS0NRFteN4lJDB35911h5El46l+QARAQABtChTb3VyaXNoIEJh c3UgPFNvdXJpc2guQmFzdUBjb2xvcmFkby5lZHU+iQI4BBMBAgAiBQJWeNrBAhsjBgsJCAcD AgYVCAIJCgsEFgIDAQIeAQIXgAAKCRDdna2p+Lv9IC5NEADjZ9A1SZWzlN/trUcRIL9Vt2xZ oesJDGbv24gXUTbe7O0aSB6EfQCCBS2wRCjtvHGBaTEMbL0oGYTIBS2VZg/xL4LFXtKqwkWe 27Z+6erRGpIVCvO2gj5uVLBvi6MGrxc+TNfKSsH+6sxnL0lHZe2H9ptpn4+RXlSchQyH9x9D qf0o9C3iUxVthdwzfS6lpJsXnTM7DfLZz/2vr7eSfTYh106fQU++WUE4KcWcH/p/DG9R0hRg e2WJQ5oVuFS7tPKJuRyEBfhDuk92HAviLg/FgisfTNNRsrVaQJfBI3sDweTV/ueP7D9TqByF l+6Xl1h3gflMhTX6llQmTHXYtU30fjk8V8yjEr90mfpBbdnWbqbI0kqCqa4f+X1L0F20vSrf slhq0JWsZR96yljonXorW93aYu/4LCvO5AGtSx4LUTX7/jVM2DWQfa59/Ioqygz0V2EYDzQF poST5TznXPlsz+0kIUzUoLjv9+ES93idZt8rRNmklHdOyA9eEAIqv/b2mXSpYMjk7HMbxPgR h+jX7WiyfbFi12z4ApQR8MbHe5iZwTsrMhwlzoJEDp4tL3NcWf5nQoRnjZ+6i0zNzMy4T1Jp LIcK82J/Vam93eLXm4E8wJbz70VEuSGO8Ei1nevG4BwSNw9vbjo0t5GvCLt93RKBXsNaJ5y4 C9W/ewSg+rkCDQRWeNIEARAAypsoemKwvg4pZv54DXN/bkmWTgSiYHWQqrUhMyP5UTi3hWw6 yuXtcDJ8QlHE9TBzO+JIKmf5q8ueANV7Rj1XXk2HiqDLggHgFy7lT/Vjv/cxp7l36kSn9iFM Y/pkg5C297g/dOmuxP/igInh5tpkIHU9qbbAGjLmplR95MEowivJKPbgs6QIFGcfuHCxNz+w 9vgqG+oZmtG9yE34/vS651v/9qJc4WW2t/oywUCm5ti/FwLV0MJ7hXmK48DpTzAVo5bAwkWB ALFvIbgGShncg5Ubn2xxe2dkgUAdxhX6bWPA3P7mC+3xrHtV0uRCBbYDCDH8LOGPWKK0poRn iUcWlKY6PAGSiAXzBmgex3lv/EymYUHH4D1QJTxaoLpO/8O17AharvkuAD2Wi5s3j/9PiE5d ilxww5Df+43memityqJzFoFLgvlftXYsnQ5dsJGXOVhnf5IE+xzWnP/W5qTuDswlV8ZJ4OjT +KiZkePhirXiKLObcwpODZ97VCE0O1DHOWNfuvg6aQd97FHo51wRs2CI5SBa2xbpEhbwKu3p Py11Lkn0NQ/3qPrnKOs4bxb5nn+mUGkLMeQLantWmnWF8r7WxELkf+06jYliG6LTdCmVld6r PqW8E6/KQaZJXjRcbJ01b80IilyFCE9l2uA+ZgVieCHWHFuQ28+yNscvJVEAEQEAAYkCHwQY AQIACQUCVnjSBAIbDAAKCRDdna2p+Lv9IPQUEACPYzMYudTbWC9w615+fpW6kZdWXRByGCqJ G8fM2zkADi521ZH3nzWzdOjAxXZ94ujEUuNMeEBDlk4lmmb1i4jstyRWf5FJBqbGM52PiPMn 5mcI8GzIayvYMugDCoyMH1WGEI3lmQbIAr7kkyjLDbhTa74YmMvzgtmHRgDSHqHqAKCrBKde HqkvxEFu8clL50KRsUm45RU4BOupbzHnw2zxzhEmK1PJaJ5WqCMSX4icftGlkWNEJq3KmmSf JPmIO48ACXneslTzF3hRTslEreAHYvQJprYZDj3Cr1ttftCcrhs7L4Fz64BhYBoye+j78z4A 8EFbaWzGuZt1SRBvd8a2qiq9kj8g2FzzxqXsCluILwN9GNPZbYY2aXPemZcbbUYtqEA4yTRs vVvd68NXxIINTfXDlAYHr1DcCHvwqr+oZuG+J70zl1vVxjCl6BvdsG6VdMM0ag6RGhuRms81 EQj1oyFg3EBFJoJ/6vXV0rTTs+Yw6DsaFlNrM/xnGUf1hD5uh7utRyYfGpLssdJQzn/dVsRU iUPL5w35WZ/za2VJIs7Mv6f9DmxaRd6FrtCmc1GoXYspcj95ytrcFHKi+MviOUkhuEnOz/tB odfNf7h8Mkb5mVHONfWrFdIyF2ZqngD+Lx2YgITXdakyBq9WOFwGoyHblqQbO4PKxD38b7av /A==
Hi Ken,


> Thank you for your offer to help. Attached is a small sample file: 10 lon,lat 
> locations by 5 times.

In your example dataset, there are five values for the time coordinate.
However, the values of x, y, lat, lon, and elev do not seem to depend on
the values of time. Is this true in general for your data? If that's
true (while still allowing x, y etc. to vary from year to year, or file
to file), that makes packaging even simpler.

-Sourish


>
> I've been trying to do this in Python with Pandas and xarray (eventually 
> incorporating Dask once I've figured this out) but I can work in almost any 
> language, if you're willing to share code.
>
> For interest - I've mapped all the hydrologic outlets at the coast of 
> Greenland at very high resolution. These are the 20,000 (lon,lat) locations, 
> with some metadata. Then I've partitioned regional climate model (RCM) 
> outputs based on the basin feeding each outlet. I have daily values of rain 
> and melted ice, from 2 RCMs. So the attached file could be considered 1 
> variable, and I have six per outlet. But I'd be OK with six separate NetCDF 
> files. Vars are ice_runoff_RCM1, ice_runoff_RCM2, and 4 more: 
> rain_over_{ice,land}_RCM{1,2}.
>
>   -k.
>
>

Attachment: signature.asc
Description: OpenPGP digital signature

  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: