[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: CDM class structure/code



Dear Sean, dear Ryan,
 
     thanks for your email. It took us a while to respond, because we were using the last couple of days to dig into the GitHub code repository trying to understand how things are organized and done. This is really a quite complex library with a lot of good thinking behind it! As we are currently working under a rather tight schedule (we need to have a demonstration of JOIN running at the GEIA meeting in Boulder in June, and I will not be able to work for about one month between now and then), we don’t see that we can put in a lot of effort to help “translating” CDM to python right now. Nevertheless, we are interested to learn more about this and would like to discuss some details with you when we will be in Boulder. A couple of questions on your strategy for doing this follow below.
 
   We are now considering two options for the JOIN demonstration – perhaps you can provide us with some guidance on the first one? (1) use some tool like java2python, jpype, etc. to actually use the Java code in our python framework. That may not be ideal performance-wise, but at least it would give us the full CDM functionality without much coding (so we hope). (2) If (1) fails (i.e. we don’t get it to work within a week), we would write some very simple code just to load some files for the demonstration and then come back to python CDM after June.
 
    There is one specific question on the class structure of the Java implementation:
  1. when reading the documentation we got the impression that the CDM itself is supposed to be abstract, i.e. independent of the file type. Yet, the variable class definition imports netcdfFile and uses this in the read() [reallyread()] method, so variable is not independent of the actual data access. Since we also access data from databases in JOIN, we are wondering whether it wouldn’t be better to have the link to netcdfFile only in the variableDS class?? Perhaps we misunderstood something here?
 
    And here a few questions on your ideas about the python version:
  1. First of all: how far have you gotten with this? Did you already convert a big chunk of code or are you still at the beginning?
  2. Are you attempting a 1:1 conversion of the Java library, including type-safe values etc.? Or are you liberating yourself from Java and try to do things the python way?
As an example: Martin looked into a translation of AxisType.java – a seemingly simple task. Of course it is rather straightforward to build something that allows enumeration and access as in AxisType.GeoX (basically “class Enum(tuple): __getattr__ =       tuple.index” does the trick), but then one cannot access the AxisType methods from this construct which is used in the CoordSysBuilder as in AxisType.GeoX.getCFAxisName. An easy workaround would be to define cfAxisName as list in the AxisType module and access this as cfAxisName[AxisType.GeoX], but of course this looks somewhat different from the original code. There may be places where such “concept translations” actually make it hard to provide the same functionality or interfaces, and sometimes it may indeed be more efficient to do certain things differently in python – but on the other hand, this makes it of course much harder to maintain consistent code versions.
  1. Are you trying to build the whole thing at once or are you starting from a subset and try to make this work properly before adding more functionality (like the interface with THREDDDS, etc.)?
 
 
We are looking forward to hear from you again and will be glad to contribute if we can be of help – but, as said above, we won’t be able to do much before July.
 
Best regards,
 
Snehal and Martin
 
 
>-----Original Message-----
>From: Sean Arms [mailto:address@hidden]
>Sent: Wednesday, February 19, 2014 9:21 PM
>To: Waychal, Snehal; 'address@hidden'
>Cc: Ryan May
>Subject: Re: CDM class structure/code
>
>Greetings Snehal!
>
>We are also starting to work on a Python implementation of the CDM. I've
>included Ryan May, our newest Software Engineer at Unidata, who is also
>heavily involved in this effort. Perhaps we could combine some efforts on this
>front?
>
>There are three levels to the CDM - the data access layer, the coordinate
>system layer, and the scientific feature type layer. Please see this page for a
>bit more detail:
>
>http://www.unidata.ucar.edu/software/thredds/current/netcdf-
>java/CDM/index.html
>
>Cheers!
>
>Sean
>
>On 2/19/14, 3:58 AM, Waychal, Snehal wrote:
>> Dear Sir/Madam,
>>
>> In our JOIN development, we are currently working on the data model and
>decided to adopt the CDM or a subset of it for this purpose.
>> Since JOIN is written in Python it would help to either obtain some Python
>code or at least the class structure of the Java implementation (used in
>Thredds data server).
>>
>> Could you please provide us this information?
>>
>> Thanks & Regards,
>> Snehal Waychal & Martin Schultz
>>
>>
>>> -----Original Message-----
>>> From: John Caron [mailto:address@hidden]
>>> Sent: Thursday, October 17, 2013 11:05 PM
>>> To: Waychal, Snehal
>>> Subject: thredds info
>>>
>>> http://thredds.ucar.edu/thredds/catalog.html
>>>
>>>
>>> docs:
>>>
>>> http://www.unidata.ucar.edu/software/thredds/current/tds/TDS.html
>>>
>>>
>http://www.unidata.ucar.edu/software/thredds/current/tds/interfaceSpe
>>> c/
>>> NetcdfSubsetService_4_3.html
>>>
>>> http://www.unidata.ucar.edu/software/thredds/current/tds/catalog/inde
>>> x.h
>>> tml
>>>
>>> http://www.unidata.ucar.edu/software/thredds/current/tds/tds4.3/tutor
>>> ial/i
>>> ndex.html
>>>
>>> python:
>>>
>>> address@hidden
>>>
>>> questions:
>>>
>>> address@hidden
>>
>>
>> ----------------------------------------------------------------------
>> --------------------------
>> ----------------------------------------------------------------------
>> --------------------------
>> Forschungszentrum Juelich GmbH
>> 52425 Juelich
>> Sitz der Gesellschaft: Juelich
>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
>> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
>> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten
>> Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr.
>> Sebastian M. Schmidt
>> ----------------------------------------------------------------------
>> --------------------------
>> ----------------------------------------------------------------------
>> --------------------------
>>