Re: [netcdfgroup] ESA Copernicus program and netCDF-4

Ed,

Thanks for sending this out. As it happens, I have been experimenting with the 
readability of some Sentinel Level 2 netCDF data in my go-to netCDF tool, 
Panoply.

In general, it's pretty readable by Panoply, EXCEPT for the handling of 
auxiliary coordinates for geolocation.

For instance, Sentinel 1A OCN IW (Ocean Winds) contains auxiliary coordinates 
owiLat and owiLon (yay!), but does not include the :coordinates attribute.  
I've seen this shortcoming enough that it's the first thing I check, but the 
average science user may be clueless in this area.  Also, it takes me only 30 
seconds to fix it with ncatted (thanks, NCO!), which again, the average user 
might not have in their toolbox.

The second case was Sentinel 3A Level 2 OLCI (Veg Index, etc.). Here, the 
auxiliary coordinates are actually in a separate file.  The Sentinel Toolbox 
knows to look in that file, but other off-the-shelf netCDF software may not.  
Again, NCO to the rescue to add the coordinates into the data file, plus add 
the :coordinates attribute.

I'm actually going to bring this up nex tweek at an upcoming meeting of the 
international WGISS group, but as a community, we should think about how to 
handle these cases where very minor omissions or deviations make the data 
appear unreadable (at least in geographic context).  (I have also seen cases in 
NASA where including commas in the :coordinates attribute ("lat, lon") rendered 
them undiscoverable by the software.)

On Oct 17, 2018, at 07:22, Ed Hartnett 
<edwardjameshartnett@xxxxxxxxx<mailto:edwardjameshartnett@xxxxxxxxx>> wrote:

Fellow NetCDFers,

While on leave, I recently had a chance to stop by two science events from the 
European Space Agency Copernicus program.

I learned a lot, and thought I would share it. I very much welcome correction 
or elaboration from anyone involved with the Copernicus program.

Starting with the Sentinel-3 spacecraft, all Copernicus ground data, from level 
1 up, are in NetCDF-4.

I was not previously aware of the scope and ambition of the Copernicus system. 
There a snappy high-level description here: 
https://www.youtube.com/watch?v=xcflQZJ5n88.

Here are some notes:

  *   Sentinel-1 and sentinel-2 missions are successful and on-going. But who 
cares, since they don't use netCDF!
  *   NetCDF-4 was selected for all data starting with the Sentinel-3 mission, 
because of the ubiquity of netCDF combined with built-in data compression. 
(Everyone loves that built-in data compression!)
  *   There are currently two Sentinel-3 spacecraft in polar orbit, 2 more 
being built on the ground as replacements, so that there will always be 2 
operational, for 15-30 years. (And this will be the case for all Sentinel 
missions.)
  *   The sentinel-4 mission involves two instruments on the EUMESATs 3rd 
generation satellites. There are two spacecraft in geocentric orbit and more 
replacements standing by. (These are the EUMETSAT equivalent to NOAA's GOES 
satellites.) These include a *sweet* hyper-spectral spectrometer.
  *   Sentinel-5P (for precursor) is a bonus spacecraft to fill a potential 
instrument gap between existing instruments and the sentinel-5 satellites, 
which are being built now for 2021 launch.
  *   Sentinel 6-10 missions are also planned, as is a gap-filling Sentinel-6P 
mission.
  *   All Copernicus data are freely available in near real-time.
  *   As is generally the case, each generation of instrument sends lots more 
data than the previous generation. So most(?) of their data holdings are 
currently in netcdf-4.
  *   The Copernicus data stream is fast enough, and of high-enough resolution, 
that it is being used in natural disaster situations (for example to show which 
specific neighborhoods and houses are flooded).
  *   The data are also used in law enforcement, for environmental pollution 
detection.

Copernicus is a very impressive program for Earth observation. The quality and 
capability of the instruments, combined with the long-term support for the 
program, will guarantee that Copernicus will become the backbone of a lot of 
science.

I would very much welcome any comments from any Copernicus data users or 
programmers about how netCDF-4 is working for you, and what are the pain-points 
if any.

I would also be happy to share information on high-performance computing with 
netCDF data, which can involved other packages like pnetcdf and PIO, to give 
good parallel I/O performance.

Please contact me if you are using Copernicus data, and tell me about what you 
are doing with it.

Thanks,
Ed Hartnett






_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

—
Christopher Lynnes   NASA/GSFC    301-614-5185
"Perfection is achieved, not when there is nothing more to add, but when there 
is nothing left to take away.”  A. de St. Exupery




  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: