[netcdf-java] Date precision while aggregating data

[John Caron]

I think you've all more or less pointed to the fact that storing millisecs in a long is a good solution (which both java.util.Date and the joda library also decided on).

Rich you are seeing that CDM/NcML is just fine with longs, even though they cant be stored in netcdf-3 (they are ok in netcdf-4).

Anyway, I think one improvement is to make sure behind the scenes im using millisecs in a long instead of seconds in doubles. Im pretty sure that will solve the roundoff problem. Im hoping it will be easy enough to put in both 2.2 and 4.0.

I will probably also start to use joda, though that is still TBD, and wont affect the API for now. I will also support ISO dates in CDM/NcML, in addition to udunit dates. Any of that new developement will be in 4.0 only.

thanks for everyone's thoughts about this....

Rich Signell wrote:
> Bob,
> 
> I like this solution.    While I would like to use units of days and
> have rounding to milliseconds in NetCDF Java, that might not appeal to
> someone who wanted even more accuracy.   And since both our existing
> USGS database and Java have fundamental units of milliseconds, it
> makes sense to use milliseconds for our units.
> 
> So my original NcML looked like:
> 
> <variable name="time" shape="time" type="double">
>   <attribute name="units" value="days since 1858-11-17 00:00:00 UTC"/>
>   <values start="47865.7916666665110000" increment="0.0416666666666667"/>
> </variable>
> 
> While the NEW one looks like:
> 
> <variable name="time" shape="time" type="double">
>   <attribute name="units" value="milliseconds since 1858-11-17 00:00:00 UTC"/>
>   <values start="4154364000000" increment="3600000"/>
> </variable>
> 
> This NEW form for the NcML returns the desired "05-Dec-1989 19:00:00"
> as the start time.
> 
> And IMHO, it's even a bit "more readable"!    ;-)
> 
> Thanks,
> Rich



> 
> On Fri, May 16, 2008 at 12:01 PM, Bob Simons <Bob.Simons at noaa.gov> wrote:
>> While one can debate whether the netCDF libraries should round (not
>> truncate) date/time values stored as doubles to the nearest millisecond
>> (I think they should), or whether it's a bug in Java (I think not - Java
>> does date/time calculations with longs), it doesn't solve the immediate
>> problem of storing date/time values in netCDF files for use with the
>> current netCDF libraries.
>>
>> I think there is a simple solution: store date/time values with units
>> that allow all of the date/time values to be expressed as integer values
>> (in the mathematical sense). In practice, the values often will need to
>> be calculated as Java longs and stored as Java doubles in netCDF files
>> to gain extra precision, since there is no support for longs in netCDF
>> files.
>>
>> For example, use "seconds since 1970-01-01T00:00:00Z" if your dates/time
>> data is measure to the nearest second.  Or, use milliseconds, minutes,
>> hours, or days, ... as appropriate.  Even when millisecond values are
>> stored as doubles and even if the "since" time is in the 1800's, there
>> should be no round-off error and Java and netCDF-java should calculate
>> the dates/times correctly.
>>
>>
>> Sincerely,
>>
>> Bob Simons
>> Satellite Data Product Manager
>> Environmental Research Division
>> NOAA Southwest Fisheries Science Center
>> 1352 Lighthouse Ave
>> Pacific Grove, CA 93950-2079
>> (831)658-3205
>> bob.simons at noaa.gov
>>
>> The contents of this message are mine personally and
>> do not necessarily reflect any position of the
>> Government or the National Oceanic and Atmospheric
>> Administration.
>> <>< <>< <>< <>< <>< <>< <>< <>< <><
>> _______________________________________________
>> netcdf-java mailing list
>> netcdf-java at unidata.ucar.edu
>> For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
>>
> 
> 
>