[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #SNE-995575]: NetCDF Compression



You should see attributes for each variable saying
    _DeflateLevel = 6
Since I do not see those, it looks like, for some reason russ.nc
is not compressed. See if the input file has _DeflateLevel attributes.


> Both the input and output file are 37574768 bytes after the nccopy
> operation
> 
> /usr/local/netcdf-4.1.3/bin/ncdump -hs russ.nc
> netcdf russ {
> dimensions:
> ÂÂÂÂÂÂÂ lon = 2501 ;
> ÂÂÂÂÂÂÂ lat = 1251 ;
> variables:
> ÂÂÂÂÂÂÂ float lat(lat) ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lat:standard_name = "lat" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lat:long_name = "latitude" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lat:units = "degrees_north" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lat:_FillValue = -999.f ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lat:_Storage = "contiguous" ;
> ÂÂÂÂÂÂÂ float lon(lon) ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lon:standard_name = "lon" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lon:long_name = "longitude" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lon:units = "degrees_east" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lon:_FillValue = -999.f ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ lon:_Storage = "contiguous" ;
> ÂÂÂÂÂÂÂ float TPW(lat, lon) ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ TPW:name = "TPW" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ TPW:long_name = "Blended Total Precipitable 
> Water" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ TPW:units = "mm" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ TPW:coordinates = "Latitude Longitude" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ TPW:_FillValue = -999.f ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ TPW:_Storage = "contiguous" ;
> ÂÂÂÂÂÂÂ float observation_age(lat, lon) ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ observation_age:name = "observation_age" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ observation_age:long_name = "age of
> observation(analysis_time - observation_time)" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ observation_age:units = "hours" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ observation_age:coordinates = "Latitude 
> Longitude" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ observation_age:_FillValue = -999.f ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ observation_age:_Storage = "contiguous" ;
> ÂÂÂÂÂÂÂ float Satellite_Number(lat, lon) ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Satellite_Number:name = "satellite_number" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Satellite_Number:long_name = "the ID number of 
> the
> satellite which made the observation" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Satellite_Number:units = "none" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Satellite_Number:coordinates = "Latitude 
> Longitude" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Satellite_Number:_FillValue = -999.f ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Satellite_Number:_Storage = "contiguous" ;
> 
> // global attributes:
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :Metadata_Link =
> "BHP-TPW_v01r0_blend_s201906110131507_e201906111041388_c201906111148307.nc"
> ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :date_created = "2019-06-11-T11:48:30Z" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :time_coverage_start = "2019-06-11-T01:31:50Z" 
> ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :time_coverage_end = "2019-06-11-T10:41:38Z" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :project = "Blended-HydroMet Products" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :naming_authority = "NDE" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :production_site = 
> "DOC/NOAA/NESDIS/OSPO/ESPC/NDE,
> S-NPP Data Exploitation" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :production_environment = "NDE Data Handling 
> System" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :platform = "Linux" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :algorithm_version = "v3.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :DAP_version = "v01r0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :Convention = "CF-1.6" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :Metadata_Conventions = "CF-1.6, Unidata 
> Dataset
> Discovery v1.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :standard_name_vocabulary = "CF Standard Name 
> Table
> (version 17, 24 March 2011)" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :satellite_name =
> "NOAA-19,NOAA-20,METOP-A,METOPB,Suomi-NPP,GCOM-W1,GPM,Met-GPS,GOES-15" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :instrument_name = "MiRS,ATMS,AMSR2,GPS,ABI 
> GOES Imager" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :title = "Blended Hydrometeorological Products 
> (BHP)" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :summary = "Unified TPW global map merges the 
> TPW
> retrievals from various satellites/algorithms" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :references =
> "https://www.ospo.noaa.gov/Products/bTPW/index.html"; ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :keywords = "Multi-Satellites merged product, 
> NDE
> Operation, OSPO" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :institution = "DOC/NOAA/NESDIS/OSPO/NDE, 
> S-NPP Data
> Exploitation, OSPO, NESDIS, NOAA, U.S. Department of Commerce" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :number_of_granules = "inapplicable" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :geospatial_lat_min = "-90.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :geospatial_lat_max = "90.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :geospatial_lon_min = "-180.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :geospatial_lon_max = "180.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :geospatial_lat_units = "degrees_north" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :geospatial_lon_units = "degrees_east" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :creator_name = "DOC/NOAA/NESDIS/OSPO, Blended 
> Hydro
> Team, Office of Satellite and Product Operations, OSPO, NESDIS, NOAA,
> U.S. Department of Commerce" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :creator_email = "address@hidden" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :creator_url =
> "https://www.ospo.noaa.gov/Products/bTPW/index.html"; ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :publisher_name = "DOC/NOAA/NESDIS/OSPO/NDE 
> S-NPP Data
> Exploitation, NESDIS, NOAA, U.S. Department of Commerce" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :publisher_email = "address@hidden" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :publisher_url =
> "https://www.ospo.noaa.gov/Products/suites.html"; ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :history = "Created by BHP Version 3.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :processing_level = "NOAA Level 4 data" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :cdm_data_type = "GRID" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :source = "MIRS_ATMS_IMG_33min; AMSR2_OCEAN " ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :PDA_short_name = "BHP-TPW" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :missing_value = "-999.0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :compression_level = "0" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :nc4_compression_flag = "on" ;
> ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ :_Format = "netCDF-4 classic model" ;
> }
> 
> On 6/11/2019 3:12 PM, Unidata netCDF Support wrote:
> > This will be difficult for us to debug since we do not have
> > access to an AIX system. Also, netcdf 4.1.3 is pretty old;
> > any chance you can upgrade to the latest version?
> > In any case, try this command and send us the output
> > (assuming its not proprietary):
> >> ncdump -hs <name of compressed file>
> >
> >> Description of problem: My programmer issues the following command using 
> >> NetCDF 4.1.3 to compress a NetCDF file on AIX and it does not compress at 
> >> all.
> >>
> >> On Linux using NetCDF V4.6.1 it does compress successfully.
> >>
> >> /usr/local/netcdf-4.1.3/bin/nccopy -d 6
> >> BHP-TPW_v01r0_blend.nc russ.nc
> >>
> >> -rw-r--r--    1 Pdpeas   dpeas      37574958 Jun 11 18:06 
> >> BHP-TPW_v01r0_blend.nc
> >>
> >> -rw-r--r--    1 root     system     37574768 Jun 11 18:55 russ.nc
> >>
> >>
> >>
> > =Dennis Heimbigner
> >    Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: SNE-995575
> > Department: Support netCDF
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > Unidata inquiry tracking system and then made publicly available through 
> > the web.  If you do not want to have your interactions made available in 
> > this way, you must let us know in eac
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: SNE-995575
Department: Support netCDF
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.



NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.