All:
The quantize feature was added to help large data producers (like NOAA,
NASA, and ESA) better handle ever-increasing Earth and space science data
volumes. When used with zlib (or zstandard), quantize will significantly
increase the compression ratio achieved. To use quantization, set the
number of significant digits to preserve for a variable. Bits not needed
for the specified precision are compressed away.
Here's a graph from the EGU presentation that Charlie Zender and I gave in
May of this year. It shows the compression ratio achieved with zlib alone
(a.k.a. deflate), and then the compression ratios achieved when adding
quantization at various settings for the number of significant digits.
[image: image.png]
Here's our EGU presentation, "Adding Quantization to the NetCDF C and
Fortran Libraries to Enable Lossy Compression":
* presentation:
https://www.researchgate.net/publication/360815823_EGU22-13259_Adding_Quantization_to_the_NetCDF_C_and_Fortran_Libraries_to_Enable_Lossy_Compression_1
* extended abstract:
https://www.researchgate.net/publication/360814804_Adding_Quantization_to_the_NetCDF_C_and_Fortran_Libraries_to_Enable_Lossy_Compression
Special thanks to Ward and Dennis at Unidata for their help and support in
getting this important feature into the netCDF C and Fortran libraries! ;-)
Keep on NetCDFing!
Ed Hartnett
On Fri, Jul 29, 2022 at 4:21 PM Ward Fisher <wfisher@xxxxxxxx> wrote:
> Hello all,
>
> The Unidata netCDF team is happy to announce the release of netCDF-Fortran
> 4.6.0. This release has a number of bug fixes, but also provides two new
> features we are excited about. They provide hooks into the `Quantization`
> and `ZStandard Compression` functionality introduced with netCDF-C v4.9.0.
> In addition to these new user-facing features, a lot of under-the-hood
> refactoring went on to help us maintain the viability and health of the
> Fortran code base. Thank you very much to all who contributed to make this
> happen, particularly Ed Hartnett, Charlie Zender, Dennis Heimbigner, and
> others!
>
> Full release notes can be found at the Github link below.
>
> Resources:
>
> * Download source:
> * Unidata: https://downloads.unidata.ucar.edu/netcdf/
> * GitHub:
> https://github.com/Unidata/netcdf-fortran/releases/tag/v4.6.0
> * Documentation: https://docs.unidata.ucar.edu/netcdf-fortran/current/
>
> Our next netCDF priority will be bringing this functionality to the C++
> interface, and getting the 4.9.1 release of the C library bundled up.
>
> Please feel free to contact us with any bug reports or questions, thank
> you all very much have a great weekend!
>
> -Ward
>
> Ward Fisher
> wfisher@xxxxxxxx
> Unidata/UCP
> NetCDF Technical Team Lead
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
