[netcdf-java] Date precision while aggregating data

[Roland Schweitzer]

Hi,

I am a big fan of Joda.  I've written two Chronologies 
(AllLeapChronology.java and NoLeapChronology.java) which we've found to 
be helpful when using Joda with model data.  These are distributed as 
part of our addXML package if anybody needs them: 
ftp://ftp.pmel.noaa.gov/ferret/pub/las/addXML/addXML-1.5.1.tar.gz.  
addXML won't do you much good unless you're using LAS, but you can steal 
the Chronologies out of the source tree.

I did offer them back to the Joda folks, but didn't stir up much interest.

Cheers,
Roland

John Caron wrote:
> I think you've all more or less pointed to the fact that storing millisecs in a long is a good solution (which both java.util.Date and the joda library also decided on).
>
> Rich you are seeing that CDM/NcML is just fine with longs, even though they cant be stored in netcdf-3 (they are ok in netcdf-4).
>
> Anyway, I think one improvement is to make sure behind the scenes im using millisecs in a long instead of seconds in doubles. Im pretty sure that will solve the roundoff problem. Im hoping it will be easy enough to put in both 2.2 and 4.0.
>
> I will probably also start to use joda, though that is still TBD, and wont affect the API for now. I will also support ISO dates in CDM/NcML, in addition to udunit dates. Any of that new developement will be in 4.0 only.
>
> thanks for everyone's thoughts about this....
>
> Rich Signell wrote:
>   
>> Bob,
>>
>> I like this solution.    While I would like to use units of days and
>> have rounding to milliseconds in NetCDF Java, that might not appeal to
>> someone who wanted even more accuracy.   And since both our existing
>> USGS database and Java have fundamental units of milliseconds, it
>> makes sense to use milliseconds for our units.
>>
>> So my original NcML looked like:
>>
>> <variable name="time" shape="time" type="double">
>>   <attribute name="units" value="days since 1858-11-17 00:00:00 UTC"/>
>>   <values start="47865.7916666665110000" increment="0.0416666666666667"/>
>> </variable>
>>
>> While the NEW one looks like:
>>
>> <variable name="time" shape="time" type="double">
>>   <attribute name="units" value="milliseconds since 1858-11-17 00:00:00 UTC"/>
>>   <values start="4154364000000" increment="3600000"/>
>> </variable>
>>
>> This NEW form for the NcML returns the desired "05-Dec-1989 19:00:00"
>> as the start time.
>>
>> And IMHO, it's even a bit "more readable"!    ;-)
>>
>> Thanks,
>> Rich
>>     
>
>
>
>   
>> On Fri, May 16, 2008 at 12:01 PM, Bob Simons <Bob.Simons at noaa.gov> wrote:
>>     
>>> While one can debate whether the netCDF libraries should round (not
>>> truncate) date/time values stored as doubles to the nearest millisecond
>>> (I think they should), or whether it's a bug in Java (I think not - Java
>>> does date/time calculations with longs), it doesn't solve the immediate
>>> problem of storing date/time values in netCDF files for use with the
>>> current netCDF libraries.
>>>
>>> I think there is a simple solution: store date/time values with units
>>> that allow all of the date/time values to be expressed as integer values
>>> (in the mathematical sense). In practice, the values often will need to
>>> be calculated as Java longs and stored as Java doubles in netCDF files
>>> to gain extra precision, since there is no support for longs in netCDF
>>> files.
>>>
>>> For example, use "seconds since 1970-01-01T00:00:00Z" if your dates/time
>>> data is measure to the nearest second.  Or, use milliseconds, minutes,
>>> hours, or days, ... as appropriate.  Even when millisecond values are
>>> stored as doubles and even if the "since" time is in the 1800's, there
>>> should be no round-off error and Java and netCDF-java should calculate
>>> the dates/times correctly.
>>>
>>>
>>> Sincerely,
>>>
>>> Bob Simons
>>> Satellite Data Product Manager
>>> Environmental Research Division
>>> NOAA Southwest Fisheries Science Center
>>> 1352 Lighthouse Ave
>>> Pacific Grove, CA 93950-2079
>>> (831)658-3205
>>> bob.simons at noaa.gov
>>>
>>> The contents of this message are mine personally and
>>> do not necessarily reflect any position of the
>>> Government or the National Oceanic and Atmospheric
>>> Administration.
>>> <>< <>< <>< <>< <>< <>< <>< <>< <><
>>> _______________________________________________
>>> netcdf-java mailing list
>>> netcdf-java at unidata.ucar.edu
>>> For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
>>>
>>>       
>>
>>     
> _______________________________________________
> netcdf-java mailing list
> netcdf-java at unidata.ucar.edu
> For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ 
>