Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Rich and all,Looks like the problem may not be in parsing as double and loosing that one decimal place. It may
be somewhere else in udunits.Just wrote a small test method in java and looks like I have to drop atleast 8 decimal places to make
a difference of one second.
>>>
import java.uti.*
public static String getPreciseDate(double startValue) {
final double daysToMS = 24*60*60*1000;
Calendar cal = Calendar.getInstance(new
SimpleTimeZone(0,"GMT"));
cal.set(1858, 10, 17,0, 0, 0); // 11/17/1858.
long calTimeMS = cal.getTimeInMillis();
calTimeMS = calTimeMS + (long) (startValue*daysToMS);
Date newDate = new Date(calTimeMS);
return(newDate.toGMTString());
}
>>>
test results:
val1: 47865.7916666665110000
Date: 5 Dec 1989 19:00:00 GMT
val2: 47865.79166666651
Date: 5 Dec 1989 19:00:00 GMT
val3: 47865.7916666665
Date: 5 Dec 1989 19:00:00 GMT
val4: 47865.791666666
Date: 5 Dec 1989 19:00:00 GMT
val5: 47865.79166666
Date: 5 Dec 1989 19:00:00 GMT
val6: 47865.7916666
Date: 5 Dec 1989 19:00:00 GMT
val7: 47865.791666
Date: 5 Dec 1989 19:00:00 GMT
val8: 47865.79166
Date: 5 Dec 1989 18:59:59 GMT
-Sachin
Rich Signell wrote:
John, Four replies to your four comments: ;-) On Wed, May 14, 2008 at 9:08 PM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:Im not quite sure where the inaccuracy comes in, likely converting between Date and udunits representation. Ill have to see what I can do. A few comments: 1) double has 53 bits of accuracy giving slightly under 16 decimal digits of accuracy. thus: public void testDoublePrecision() { double dval = 47865.7916666665110000; System.out.println(" dval= "+dval); } prints: dval= 47865.79166666651Okay, you lost the lowest bit, but you should still be fine. You still have 11 places after the decimal point. In Matlab, which uses double precision arithmetic, I don't get a problem converting to gregorian until we drop to 8 places after the decimal point: datestr(datenum([1858 11 17 0 0 0]) + 47865.791666666511) => 05-Dec-1989 19:00:00 datestr(datenum([1858 11 17 0 0 0]) + 47865.79166666651) => 05-Dec-1989 19:00:00 datestr(datenum([1858 11 17 0 0 0]) + 47865.7916666665) => 05-Dec-1989 19:00:00 datestr(datenum([1858 11 17 0 0 0]) + 47865.791666666) => 05-Dec-1989 19:00:00 datestr(datenum([1858 11 17 0 0 0]) + 47865.79166666) => 05-Dec-1989 18:59:592) preserving lowest bits of accuracy is tricky, and requires care, which i promise has not (yet) happened in the CDM units handling. in general, relying lowest bits being preserving is dicey.That's okay -- we don't need to preserve that lowest bit.3) what is the definition of a "day". how accurate do you need that? All I could find was this note in the units package: * Interval between 2 successive passages of sun through vernal equinox * (365.242198781 days -- see * http://www.ast.cam.ac.uk/pubinfo/leaflets/, * http://aa.usno.navy.mil/AA/ * and http://adswww.colorado.edu/adswww/astro_coord.html): you may agree, but what if someone uses a different meaning for "day" ??Take a look at udunits.dat: http://www.unidata.ucar.edu/software/udunits/udunits-1/udunits.txt A "day" is precisely defined as 86400 seconds. A "sidereal day" is a different unit.4) IMHO, using udunits for calender date is a mistake. its a units package, not a calender package.Maybe, but I think to solve the current problem, we could just find out where the computations are dropping the double precision.5) "47865.7916666665110000 days since 1858-11-17 00:00:00 UTC" is, um, unreadable to humans.What is not unreadable about that? Yes, it's a big number with a lot of precision, and a older date, but I think it's perfectly readable and unambigous. And as I mentioned, it's a an international recognized convention called "Modified Julian Date".6) I earlier proposed to CF that we allow ISO date strings, more readable, not ambiguous, and doesnt have a precision problem. Various CF authorities thought it wasnt needed because it was redundant with the udunits representation.I think allowing ISO date strings in CF would be a good idea, and I also think allowing a two integer representation in CF would be a good idea (we use Julian day, and milliseconds since midnight as our two integer vectors). But that idea was also not too popular. Several people thought it would be a good idea, including Balaji, but there was concern about to need to modify all existing CF applications to handle these new time conventions. But if this was just handled in UDUNITS, I don't think this would be much problem, as I would think that most CF-compliant apps have used the UDUNITS library to to their math. -RichRich Signell wrote:Jon, The precision of the time vector with "units since XXXX" must definitely be considered carefully, but we did think about this. We want to store all our oceanographic time series data with the same time convention to facilitate aggregation and minimize mods to existing software. Choosing time as double precision with units of "days since 1858-11-17 00:00" should give us a precision of: - Better than 3.0e-5 milliseconds until August 31, 2132 and - Better than 3.0e-4 milliseconds until October 12, 4596! (This is actually is the definition of "Modified Julian Day", which is one of the few internationally recognized time conventions that starts at midnight. See http://tycho.usno.navy.mil/mjd.html for more info. It also has the advantage of being a date by which nearly all the world had finally switched to a Gregorian calendar, and early enough so that most of the data we want to represent will have positive time values.) The bug Sachin reported is a big deal for us, since we want to use NcML and THREDDS as a way of serving our hundreds of oceanographic time series files as CF compliant using NcML with the THREDDS data server without changing any of the original files. The original files are NetCDF, but with a non-standard convention for time: an integer array with julian day, and a second integer array with milliseconds since midnight. This allows integer math with time to give results with no round off problems. We have a script in Matlab (that uses double precision math) to take our two integer format for time and create NcML for a CF-compliant time array using start and increment. That script produces NcML like this: <variable name="time" shape="time" type="double"> <attribute name="units" value="days since 1858-11-17 00:00:00 UTC"/> <attribute name="long_name" value="Modified Julian Day"/> <values start="47865.7916666665110000" increment="0.0416666666666667"/> </variable> As Sachin mentioned, the start time for this file is "05-Dec-1989 19:00:00", and as proof that we have sufficient precision, when we simply load the time vector in NetCDF-java and do the double precision math in Matlab, we get the right start time: datestr(datenum([1858 11 17 0 0 0]) + 47865.791666666511) ans = 05-Dec-1989 19:00:00 but when we use the NetCDF-Java time routines to convert to Gregorian, we get 05-Dec-1989 18:59:59 GMT Clearly our users will not accept this. I hope this can get resolved soon!!!! -Rich On Tue, May 13, 2008 at 2:52 AM, Jon Blower <jdb@xxxxxxxxxxxxxxxxxxxx> wrote:Hi, I have seen similar issues (time values being out by a second or two). I was wondering whether it's something to do with udunits and calculating dates on the basis of "units since XXXXXX". I seem to remember an earlier conversation on this list (or maybe on the CF list) concerning how udunits defines the length of certain time-spans (e.g. a month) and wondered whether this might be the issue? Jonathan Gregory recommended against using "months since" and "years since" and sticking to seconds or days to avoid ambiguities in the length of a month/year. But maybe this is a red herring. Whatever the issue is I'd be very keen to understand it as it's affecting me too! Cheers, Jon On Mon, May 12, 2008 at 9:31 PM, Sachin Kumar Bhate <skbhate@xxxxxxxxxxxxxxx> wrote:John,> > The NcML file shown below attempts to aggregate time series files, > overriding > the time values for each 'time' variable. > > The aggregation works great and I can access the time values as well, > but I see that there is loss of precision in the new time values, when I > access > values for a coordinate data variable. > > For example: > > <<<< > URI = > 'http://www.gri.msstate.edu/rsearch_data/nopp/test_agg_precision.ncml'; > String var="T_20"; > > GridDataset gid = GridDataset.open(URI); > GeoGrid Grid = gid.findGridByName(var); > GridCoordSys GridCoordS = (GridCoordSys) Grid.getCoordinateSystem(); > > java.util.Date d[] = GridCoordS.getTimeDates(); > > System.out.println("DateString: "+d[0].toGMTString()); > >>>>> > > The output from the above code for the 1st time value in the java Date > array. > > DateString: 5 Dec 1989 18:59:59 GMT > > But, the correct value should be > > DateString: 5 Dec 1989 19:00:00 GMT > > > Just out of curiosity I tried to print the 1st time value being read > from the NcML, > by 'ucar.nc2.ncml.NcmlReader.readValues()'. I get, > > Start = 47865.79166666651; (Parsed as double) > > but, the 1st start value specified in NcML is '47865.7916666665110000'. > > Don't care about the tailing '0s', but the digit '1' in the 12th decimal > place is being dropped and may be causing this > problem. > > Although, parsing it as a 'BigDecimal' does read in the correct value. > > Start-BigDecimal: 47865.7916666665110000 > > > I am just guessing here, I am not sure if this is what causing the > precision problem. > > Will appreciate your help. > > thanks.. > > Sachin > > -- > Sachin Kumar Bhate, Research Associate > MSU-High Performance Computing Collaboratory, NGI > John C. Stennis Space Center, MS 39529 > http://www.northerngulfinstitute.org/ > > > > _______________________________________________ > netcdf-java mailing list > netcdf-java@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ > -- -------------------------------------------------------------- Dr Jon Blower Tel: +44 118 378 5213 (direct line) Technical Director Tel: +44 118 378 8741 (ESSC) Reading e-Science Centre Fax: +44 118 378 6413 ESSC Email: jdb@xxxxxxxxxxxxxxxxxxxx University of Reading 3 Earley Gate Reading RG6 6AL, UK -------------------------------------------------------------- _______________________________________________ netcdf-java mailing list netcdf-java@xxxxxxxxxxxxxxxx For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
-- Sachin Kumar Bhate, Research Associate MSU-High Performance Computing Collaboratory, NGI John C. Stennis Space Center, MS 39529 http://www.northerngulfinstitute.org/
netcdf-java archives: