Sebastien,
Another approach you could take to compress your sparse radar data
would be to convert your existing files to NetCDF4 files with
deflation. Rutgers took this approach for a large collection of
remote sensing data and it worked great, giving good performance and
decreasing the size of the collection by a factor of 30. And the
THREDDS data server can handle NetCDF4 files just fine.
To convert the NetCDF3 files to deflated NetCDF4, they used the latest
version of NCO tools, which allows you to convert to NetCDF4 with
deflation with this simple command line:
ncks -4 -L 1 netcdf_v3.nc netcdf_v4_deflated.nc
where the "-4" means NetCDF 4 and "-L 1" means level 1 deflation.
They then used the THREDDS Data Server to aggregate these NetCDF4
files along the time dimension and served the collection up as a
single dataset.
Good luck,
-Rich
On Wed, Mar 31, 2010 at 5:15 AM, Jonathan Blower
<j.d.blower@xxxxxxxxxxxxx> wrote:
> Hi Sebastien,
>
>
>
> Good to hear from you, thanks for getting in touch. I think you are the
> first person to ask me this question! ncWMS does not make any special
> provision for reduced horizontal grids or compression by gathering.
> However, I don’t know if there’s any support for this in the NetCDF-Java
> libraries, so I’m copying this message to their list.
>
>
>
> If NetCDF-Java doesn’t support this kind of grid it may be possible to
> implement it in ncWMS. However, I’m travelling at the moment so I can’t
> give this much attention now I’m afraid. I’ll wait to hear from Unidata,
> then I’ll have a think about how to handle this.
>
>
>
> Best wishes,
>
> Jon
>
>
>
> From: Mancini Sebastien [mailto:Sebastien.Mancini@xxxxxxxxxxx]
> Sent: 31 March 2010 06:41
> To: jdb@xxxxxxxxxxxxxxxxxxxx
> Subject: ncWMS question
>
>
>
> Hi Jon,
>
>
>
> I do not know if you remember me. My name is Sebastien. I worked at eMII
> (IMOS) in Hobart.
>
> We met last year at the Ocean Obs conference in Venice.
>
>
>
> We are using at the moment ncWMS to visualise some of IMOS satellite data
> (gridded netcdf file).
>
>
>
> I am doing some tests on displaying some radar data in ncWMS.
>
> The radar data is not created on a filled grid but on a sparse grid in order
> to save space in the NetCDF file.
>
>
>
> I am trying to modify those NetCDF files and make them compliant with the CF
> convention using paragraph 5.3 on the « Reduced Horizontal Grid » and
> paragraph 8.2 on the « compression by gathering ».
>
>
>
> I created a NetCDF file example.
>
> I used ncdump to extract the following lines.
>
>
>
> dimensions:
>
> TIME = 1;
>
> POSITION = 1650;
>
> latdim = 80;
>
> londim = 80;
>
>
>
> variables:
>
> double TIME(TIME) ;
>
> TIME:standard_name = "time" ;
>
> TIME:long_name = "analysis_time" ;
>
> TIME:units = "days since 1950-01-01 00:00:00" ;
>
> TIME:axis = "T" ;
>
> TIME:_FillValue = -9999. ;
>
> double LATITUDE(POSITION) ;
>
> LATITUDE:standard_name = "latitude" ;
>
> LATITUDE:long_name = "latitude" ;
>
> LATITUDE:units = "degrees_north" ;
>
> LATITUDE:axis = "Y" ;
>
> LATITUDE:_FillValue = 9999. ;
>
> double LONGITUDE(POSITION) ;
>
> LONGITUDE:standard_name = "longitude" ;
>
> LONGITUDE:long_name = "longitude" ;
>
> LONGITUDE:units = "degrees_east" ;
>
> LONGITUDE:axis = "X" ;
>
> LONGITUDE:_FillValue = 9999. ;
>
> double POSITION(POSITION) ;
>
> POSITION:long_name = "grid position index" ;
>
> POSITION:compress = "latdim londim" ;
>
> double SPEED(TIME, POSITION) ;
>
> SPEED:standard_name = "sea_water_speed" ;
>
> SPEED:long_name = "sea water speed" ;
>
> SPEED:units = "m s-1" ;
>
> SPEED:_FillValue = 9999. ;
>
> SPEED:coordinates = "LONGITUDE LATITUDE" ;
>
>
>
> I tried to visualise this file with ncWMS but it did not work.
>
>
>
> Is it possible to visualise this kind of NetCDF file using the actual
> version of ncWMS?
>
>
>
> If yes, I certainly made a mistake in the NetCDF file in the declaration of
> the different variables. Will you be able to send me an example of this kind
> of NetCDF file?
>
>
>
>
>
> Thanks for your help.
>
>
>
> Regards
>
>
>
> Sebastien
>
>
>
>
>
>
>
>
>
> Sebastien Mancini
>
> Project Officer
>
>
>
> eMII Project
>
> University of Tasmania
>
> Private Bag 21
>
> Hobart TAS 7001
>
>
>
> Ph: (03) 6226 8571
>
> Fax: (03) 6226 2997
>
> sebastien.mancini@xxxxxxxxxxx
>
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Ncwms-users mailing list
> Ncwms-users@xxxxxxxxxxxxxxxxxxxxx
> https://lists.sourceforge.net/lists/listinfo/ncwms-users
>
>
--
Dr. Richard P. Signell (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598
