Re: [thredds] Tile aggregation with Lon / Lat dimensions

Thanks a lot Antonio !

It gives me a good example of using "section" and "logicalsection" attributes.

Regards

Vincent


Le 21/12/2016 à 19:49, Antonio S. Cofiño a écrit :

Vincent,

I'm attaching a tile aggregation where the lon coordinate, and the respective data array, are changed from 0 360 range to -180 180.

Best regards

Antonio


--
Antonio S. Cofiño
Associate Professor and Researcher
Grupo de Meteorología de Santander
Dep. of Applied Mathematics and Computer Sciences
Universidad de Cantabria (Spain)

Academic Visitor
National Centre for Atmospheric Science
Department of Meteorology
School of Mathematical, Physical and Computational Sciences
University of Reading (UK)

http://antonio.cofino.es
On 21/12/16 14:56, Vincent THIERION wrote:

I well undertand the benefit of virtual datasets.

Even I chose a physical aggregation, do you think, it's possible to send / show me a NCML example of tiled aggregation you made ?

Regards

Vincent


Le 21/12/2016 à 15:40, Antonio S. Cofiño a écrit :

Yes, if you have already a clear data structure it's better to persist the view as an physical case.

In my case I have some dataset defined in longitude from -180 to 180, but some users want it from 0 to 360 ... then I create a virtual dataset, with 2 logical views from the data and one tiled aggregation of them. This provided to me 2 views served from TDS but from the same physical dataset without replication.

Regards

Antonio



On 21/12/16 14:25, Vincent THIERION wrote:

Antonio,

Thank you for your answer.

I think my best option is to stich / spatially aggregate individual files (not virtually) and serve the aggregated image with TDS.

Vincent


Le 21/12/2016 à 11:22, Antonio S. Cofiño a écrit :

Vicent,



On 21/12/16 08:58, Vincent THIERION wrote:
Hello,

By reading different posts on tiled aggregation (i.e. aggregation along longitude and latitude dimensions), such as :

  * 
http://www.unidata.ucar.edu/mailing_lists/archives/thredds/2011/msg00270.html
  * 
http://www.unidata.ucar.edu/mailing_lists/archives/thredds/2013/msg00500.html

better than "labor-intensive" NCML implementation it looks like this capability is handled by GridDataset and the Netcdf SubsetService object : http://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2011/msg00133.html, only accessible with java library while I work with the python library.

All these posts are quite old and I ask me what the better way is now ? Maybe stich all the netcdf files in one NetCDF before serve on TDS ?


The tiled aggregation it's one problem and subsetting is different one. To aggregate on non-outer dimensions the only existing in netcdf-java it's tiled one. Never released with full support but it's there and working. The most common cases are aggregation on outer-dimensions or non-existing ones.

This aggregation could be made using netcdf-java ncml and serving, using TDS, as opendap/netcdfsubsetting endpoints.

To subsetting dataset (aggregated on non-aggregated) relies on tool/client side. Python pyDap or Netcd4 packages are able to access opendap services to subsetting purposes. Please note that the Netcdf4 package needs to access to netcdf library compiled with opendap reading capabilities.

Only specific implementation like x-array python packages has some multifile aggregation
http://xarray.pydata.org/en/stable/generated/xarray.open_mfdataset.html#xarray.open_mfdataset

NcML aggregation is general specification but only netcdf-java has full support of it. Some other implemetations exist:
https://github.com/axiom-data-science/pyncml

or the BES implemetation:
http://docs.opendap.org/index.php/BES_-_Modules_-_NcML_Module

But neither of them supports Tiled aggregation.

Regards

Antonio S. Cofiño


Thank you for your useful feedbacks
--
-------------------------------
Vincent THIERION
Ingénieur de recherche - THEIA - CES OSO
vincent.thierion [at] cesbio.cnes.fr
vincent.thierion [at] inra.fr

CESBIO (Bureau 123)
(UMR 5126 CNRS/CNES/UPS/IRD)
Bpi 2801
18 Av. Edouard BELIN
31401 TOULOUSE CEDEX 9

INRA - Toulouse - Dynafor
(UMR 1201 INRA/INP-ENSAT/INP-EI Purpan)
24 chemin de Borde-Rouge
Auzeville BP 52627
31326 CASTANET-TOLOSAN CEDEX
-------------------------------


_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:http://www.unidata.ucar.edu/mailing_lists/



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:http://www.unidata.ucar.edu/mailing_lists/

--
-------------------------------
Vincent THIERION
Ingénieur de recherche - THEIA - CES OSO
vincent.thierion [at] cesbio.cnes.fr
vincent.thierion [at] inra.fr
T. +33.(0)5.61.55.85.34 
-https://dynafor.toulouse.inra.fr/dynafornet/index.php/fre/Collaborateurs/Permanents/Thierion

CESBIO (Bureau 123)
(UMR 5126 CNRS/CNES/UPS/IRD)
Bpi 2801
18 Av. Edouard BELIN
31401 TOULOUSE CEDEX 9

INRA - Toulouse - Dynafor
(UMR 1201 INRA/INP-ENSAT/INP-EI Purpan)
24 chemin de Borde-Rouge
Auzeville BP 52627
31326 CASTANET-TOLOSAN CEDEX
-------------------------------


_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:http://www.unidata.ucar.edu/mailing_lists/



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:http://www.unidata.ucar.edu/mailing_lists/

--
-------------------------------
Vincent THIERION
Ingénieur de recherche - THEIA - CES OSO
vincent.thierion [at] cesbio.cnes.fr
vincent.thierion [at] inra.fr
T. +33.(0)5.61.55.85.34 
-https://dynafor.toulouse.inra.fr/dynafornet/index.php/fre/Collaborateurs/Permanents/Thierion

CESBIO (Bureau 123)
(UMR 5126 CNRS/CNES/UPS/IRD)
Bpi 2801
18 Av. Edouard BELIN
31401 TOULOUSE CEDEX 9

INRA - Toulouse - Dynafor
(UMR 1201 INRA/INP-ENSAT/INP-EI Purpan)
24 chemin de Borde-Rouge
Auzeville BP 52627
31326 CASTANET-TOLOSAN CEDEX
-------------------------------


_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:http://www.unidata.ucar.edu/mailing_lists/


--
-------------------------------
Vincent THIERION
Ingénieur de recherche - THEIA - CES OSO
vincent.thierion [at] cesbio.cnes.fr
vincent.thierion [at] inra.fr
T. +33.(0)5.61.55.85.34 - 
https://dynafor.toulouse.inra.fr/dynafornet/index.php/fre/Collaborateurs/Permanents/Thierion

CESBIO (Bureau 123)
(UMR 5126 CNRS/CNES/UPS/IRD)
Bpi 2801
18 Av. Edouard BELIN
31401 TOULOUSE CEDEX 9

INRA - Toulouse - Dynafor
(UMR 1201 INRA/INP-ENSAT/INP-EI Purpan)
24 chemin de Borde-Rouge
Auzeville BP 52627
31326 CASTANET-TOLOSAN CEDEX
-------------------------------

  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: