Re: [netcdf-java] Data precision while aggregating data

John,

> how did you calculate that 47865.791666666511 was the correct value for your 
> desired time?

Well of course it isn't *exactly* the correct value.   The start time
is *exactly*  1989-Dec- 05 19:00:00, but 19/24 is a repeating decimal
(0.66666666666...), so although 47865.791666666667 would be a slightly
more accurate value at the same level of precision, the difference is
only 0.013 milliseconds.

It turns out that 47865.791666666667 does produce the desired
05-Dec-1989 19:00:00 when run through the NetCDF Java routines.  So
why not just use that?  Because we can't depend on the math to be that
accurate to generate that value.      The 47865.791666666511 value
resulted from this operation in Matlab:

   sprintf('%.16f',2447866 + 68400000/3600/24/1000 - 1721059

where we were taking the Julian start date,  converting milliseconds
to fractional day, and then the time origin from Julian to MJD.

Unfortunately, this value of 47865.791666666511, as mentioned
originally, results in 05-Dec-1989 18:59:59,  despite being only 0.013
milliseconds less.

This seems consistent with Sachin's results, suggesting that our
problem is caused by NetCDF Java time routines truncating instead of
rounding to the nearest millisecond.

I could work around the problem by adding 0.5 milliseconds to my start
times, but it would be better to have NetCDF-Java round instead of
truncate, wouldn't it?

Thanks,
- Rich

-- 
Dr. Richard P. Signell (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598


  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: