CHANGES for Netcdf-Java library Version 4.2.32 Build Date = 2012-05-09 21:49:36 Changes in version 4.2: 33: (2012-05-08) Nids update (dual pole) from Yuan RotatedPole fix (from Robert Schmunk) SliceReader broken when rank > 1 32: (2012-04-16) nexrad level 2 had wrong time coordinate unit (since start instead of since midnight) nexrad level 2 time coordinate not dealing with midnight crossover 31: (2012-03-30) new nexrad station JoinNew has bug when groups are present because non-agg vars are not getting proxied. Grads: A change so users can set the units of the vertical dimension if they want. Grib1PDS ecmwf perturbation octet (from jitka) 30: (2012-01-27) Nexrad AR2V0007 29: (2011-12-27) Fix when earth_radius comes in km instead of meters. NetcdfFile unzip was broken 28: (2011-12-01) fix set proxy nexrad radar station update - add LGX south polar stereographic has wrong scale 27: (2011-11-04) netcdfDataset.openOrAcquireFile(): check on http before ncml grib2: scale factor in pds is signed grib2 gaussian (gds 40): calc deltaLon from (lo2-lo1)/(nx-1). opendap : 5 min timeout Mercator : handle "scale at standard parellel" correctly bug in CoordinateAxis1D.getNames() when the coordinate is of type char, like ensemble names. RandomAccessFile.readLine() not following javadoc. Replace with code copied from java.io. 26: (2011-07-25) Fix to Suomi coord sys builder NetcdfFile.makeValidCdmObjectName() - remove chars < 20. attributes in NcMLWrite must get filtered refactor all uses of Variable.getName() to getFullName and getShortName Variable.getName() always returns escaped version. Withdraw Variable.getEscapedName(), NetcdfFile.findTopVariable() NcDDS was assuming a dimension always has a name. Switch to backslash escaping for CDM objects. Remove superfluous unescaping. NcMLReader has incredible isVariableLength="false" bug, how did anything work? String.unescape() handle %x at the end of string, eg "/thredds/dodsC/testdods/coads_climatology.nc.dods?SST.SST%5" NetcdfFile.escape/unescape - standard % escaping using URLEncoder.encode/decode NetcdfFile.makeValidCdmObjectName() - dont allow trail.lead spaces or "/" Grib2 jpeg2000 decoder : bug when unsigned, nbits > 8 CF Radial (alpha) LambertConformal accessor methods added hdf5: vlen can have null heapId, indicates no data opendap escaping of "/." get earth radius from GRIB GDS (affects LC only) GRIB forecast time roundoff error InvCatalogs: no longer support variants of catalogs (eg 0.6 namespace) InvCatalogImpl : dont retain the InvCatalogFactory, Converter; just get a new one if needed GridCoordSys: 2d time coords may have null dates - skip them in makeTimes2D() Opendap: properly escape attribute and variable names. 25: (2011-05-20) DatasetCollectionManager: make sure scan triggers event MetadataManager delete was turned off GRIB1Write: downgrade messages to debug GRIB2 standard tables: 0-1-52 should be Total_precipitation_rate Array.reflectArrayCopyOut bug fix from Rob Weingruber InvAccess.getStandardName() no longer wraps with thredds: or cdmremote: use getWrappedName() intead 24: (2011-05-02) Improve Grads, Gempak Iosp (dmurray) GridCoordSys.getLatLon() reversed the indices when calling CoordinateAxis2D.getCoordValue(). Ncml writing String attributes when n>1 Opendap refactor 23: (2011-04-13) H5: bug in some subsets of chunked data. Grib2.PDS: process timeUnit=3 (month) 22: (2011-03-10) Attribute.isString() false if value is null. All maven versions are 4.2.22 Uspln iosp: ensure we are correctly interpreting GLN data as extended format. Nexrad2 iosp: hi-res bug Default TableAnalyser: turn into trajectory if there is trajectory metadata (record dimension case) igra iosp (alpha) H5header: remove symbolic links in same dir as regular object GradsBinaryGridServiceProvider (don) 21: (2011-03-02) various opendap problems are fixed ghcnm iosp (beta) 20: (2011-02-16) GRIB: Deal with zero length index better Default TableAnalyser: turn into trajectory if there is trajectory metadata DatasetCollectionManager is not activated if there is no scan N3header: check for one record variable, no-padding case. URLnaming.escapeQuery() reverted to old way of checking for query string CompositePointCollection.getDataVariables() : check for null ParsedSectionSpec: check for illegal ranges Variable: null unit string bug GridHorizCoordSys: allow gaussian lats NP != NY 19: (2011-02-01) Nexrad2: dual pole radar offset bug H4header: workaround for bad hdf4 HTTPClient refactor / framework opendap bison parser CSMConvention looks for date and time units H5header.showBytes() -> Misc GridPointWriter fix from Don ToolsUI: GribReportPanel NcStreamGrammer version 1 cdmRemote sequences, getStructureIterator IospHelper.makeArrayBB ncstream was relying on assert to be enabled (!) TableAnalyser CdmDirect added StructureData.convertScalarLong() added StructureDataComposite use member lookup by name (!) NCdumpW.printArray() had ArrayStructure before ArraySequence (!) DefaultConventions uses short name 18: (2010-12-23) N4: vlen not setting byteOrder ncss bug when time subsetting FMRC datasets - time_run axis was not getting subsetted. MetadataManager.closeAll() bug GridDatasetInv looks for run time inside file if needed. TdsMonitor : put dns cache directory under home/tdsMonitor; add reverse hostname to access/user table fix close bug in MetadataManager (bdb closing) GridCoordSys.getDateRange() now works for 2D time coord. get deprecated. mavenized: combine common, unidataCommon; split bufr, bufrTables FMRC bug in constant offset datasets when offset didnt start at 0. protobuf-2.3.0 UI: CatalogCopier, GribReportPanel ucar.ma2.ArrayScalar ucar.nc2.iosp.noaa.Ghcnm IOSP: StructureDataAscii, improve TableParser 17: (2010-11-30) Attribute tolerate null value NcMLWriter check for null attribute H5: Off by 1 error in DoublingTable 16: (2010-11-24) Improve diagnostic on Aggregation / no datasets found H5/N4 non-string vlen bug. GridVariable ensemble coordinate - use perturbation number and enseble type Grib2 Template 11 bug reading forecast time 15: (2010-11-19) NetcdfCFWriter : set large file if needed FileWriter: allow largeFiles. also use much larger buffer H5header: message sizes are unsigned bug in BDB create permission 14: (2010-11-13) NetcdfFile.getDetailInfo() had a bug, showed up in BUFR cdmremote improvemnets NetcdfFile: unescape string in getRaf() (from erussell@ngs.org) H5header: clarify GlobalHeap termination condition 13: (2010-11-01) CoordinateAxis sets caching to true, VariableDS propagates to orgVar. remove e.printStackTrace() at CEEvaluator line 180 add Fmrc.close(), CollectionManager.close() to release resources manage bdb databases so that they can be deleted when needed to regenerate the metadata GribPds: getForecastTimeInterval(int wantUnit) convert to desired unit, for GFS_Global_2p5deg GRIB2: if end of interval = 0, use forecast time. GribGridRecord: getStatisticalProcessTypeName, getStatisticalProcessTypeNameShort ToolsUI: add IOSP/Cdmremote panel Grib: clarify units and udunits remove GridLookup.getFirstTimeRangeUnitName() Grib2ProductDefinitionSection needs refTime use CF.GRID_MAPPING_NAME where possible 12: (2010-10-26) LatLonProjection uses "latitude_longitude" for grid_mapping_name attribute ToolsUI: remove StationRadial (obs), add IOSP/CDMremote (rough) FileWriter: incorporate Steve Ansari's changes GridServiceProvider: ensemble dim order was wrong move ucar.utils.prefs into cdm, because used by tds (dqcRedirect) and delete prefsAll.jar 11: (2010-10-24) tdsMonitor: use prefs ComboBox to persist list of servers put tdsMonitor into ToolsUI, remove lucene reorganize ui packages GridVariable : remove check for same data on duplicate record - too expensive. 10: (2010-10-21) Grib2DataSection: handle all missing data case (DS error os=2 nbitsd=0) DatasetCollectionManager.enableMetadataManager(): allow bdb caching to be optional Grib2: add template 15 GribPds.getIntervalStatType() -> getStatisticalProcessType() AggOuterDimension.getNestedJoinRange() getting strides wrong sometimes. DatasetConstructor.transfer() (Aggregations) wasnt allowing unlimited dimensions, not sure why, but passes unit tests without it. FmrcInvLite: change linear search - CPU bottleneck 9: (2010-10-12) UI: CoordSys tab "show values" shows irregular bounds as a pair UI: add GridInv tab in FeatureType/Grid Grib2: distinguish genProcType from genProcId. Use genProcType in variable name when its type "error". Fmrc.makeFmrcInv(): on io error, skip file. Grib2Pds30: use superclass getForecastDate() ToolsUI: Grib2RawPanel can save individual records to a file Grib2Input: track start, end file pos of records nexrad : bug fix for dual pole radars. 8: (2010-09-30) move ucar.unidata.util completely to unidataCommon for now Grib2: new way to calculate intervals, works even for NDFD Grib2 PDS: add template 6, 10 GridHorizCoordSys, Polar Stereographic : look for GridDefRecord.LAD, set scale accordingly. remove error messsage in GribPDSParamTable; return false on file not found opendap performance improvements (~36% on opening dataset) Grib1: add table 7-4-128 Grib1 Time Range Indicator 123, 124 ToolsUI PointFeature: remove structure hashCode in column 0 7: (2010-09-27) ToolsUI : Grib1 Tables Viewer special hack for avhrr OI-V2 files, need zlev to be recognized as HEIGHT coord cdmremote works for full netcdf-4 data model FeatureDataset inner and outer Ncml Set cacheSize for coordinate variable == 40K 6: (2010-09-21) Remove Date from GridIndexToNC "history" attribute. ToolsUI : allow http: argument to directly open a dods URL FmrcInvLite.GridInventory now checks for intervals correctly CoordinateAxis1D.makeBounds() - check for ordering, dont accept equality 5: (2010-09-17) ArrayStructureBB.setOffsets() also calls setStructureSize(), does all nested members. BUFR MessageUncompressedDataReader: bug in deeply nested structures FeatureCollectionConfig.ProtoChoice : add Run, with param = run hour Calendar.HOUR should be Calendar.HOUR_OF_DAY everywhere GribIOSP: raf was not setting endianess Fmrc : integrate time intervals M3IO : add projection types 6,7,8,9,10 (from plessel.todd@epa.gov) ToolsUI: WMO grib code tables UI NetcdfDataset.wrap() creates real wrapper when enhancement needed - to not munge cached file GridDataset always uses default enhance mode. CFPoint: bug when feature id was a char type dual pole radar changes Remove Indexing cruft IOSP: Replace Index with IndexLong to deal with very large files Fix problem with WMS ignoring _FillValue (see JIRA TDS-108). Clean up some NetcdfDataset and GridDataset enhance mode stuff. refactor grib, deal with time intervals iosp.IndexChunker needs to use a special version of Index that uses longs, since the source can be a file > 2^32 4: (2010-07-26) FMRCDataset process ncml in ProtoDataset element FMRCDataset check if 2D dataset exists, ncml in ProtoDataset CoordAxis1D.makeBounds() check for numeric GridCoordSys.getRangesFromLatLonRect() uses LatLonPointImpl.lonNormalFrom() for latlon bounding box HDF5 bug: group btree entries were the wrong size when offsets = 4 bytes (!) Change GRIB projection name "rotated_lat_lon" to "rotated_latlon_grib" WRFConventions: stereographic projection use centralLat ("CEN_LAT") when available HDF5 bug: group btree entries were the wrong size when offsets = 4 bytes (!) MADIS/AWIPS station profile improvements to plugin GRIB adds attribute _CoordinateTransforms = "" to lat/lon Coord sys variable; no longer need weird code in CoordSysBuilder.VarProcess() hdf5, nc3: attributes with no value add StereographicAzimuthalProjection, TransverseMercatorProjection elliptical earth (Heiko.Klein@met.no) FeatureCollectionConfig.Config: use collection name if it exists CollectionSpecParser, FeatureCollectionConfig: trim the spec string bugs in CoordinateAxis1D.findCoordElement - remove longitude special case, rely on fixing monotonicity. VariableDS copy constructor - dont apply scale/offset twice hdf5: variable length array of strings, not incrementing counter (Steve Sullivan) hdf5: bug in CHAR type when compressed (Steve Sullivan) grib refactor - time intervals ToolsUI - IOSP/GRIB2 tab Comparators using longs were wrong, eg DiskCache (bug fix from Steve Ansari) CF1 use super.getAxisType() CF1 break apart convention list DODS parser: fix alias command in the DAS Integrate Java-OPeNDAP into thredds under name opendap and parallel to eg cdm. 3:(2010-05-27) NetcdfFile.readSection(String) should unescape names DODNetcdfDataset.getDetailInfo() shows DAS, DDS. CoordinateAxis1D non-monotonic lon coordinates bug VariableDS() constructor must distinguish between copy() and CoordinateAxis() VariableDS.needConvert() for StructureDS.convert() VariableDS: hasCachedDataRecurse() checks orgVar.hasCachedData(), used by FmrcDataset clean up VariableDS cache checking AWIPS CoordSysBuilder improvements H5header: workaround for natts > 8 hdf5 bug N3header: deal with possible vsize overflow. FMRC xml cache lastModified needs to ignore msecs. GridDatatype was adding duplicate time dimension when had coordsys runtime(time), time(time) eg FMRC best CF 1.5 point obs fixes remove dependency on gnu-regexp 2: (2010-05-04) ThreddsDataFactory does not open NetcdfDataset in enhance mode. use opendap-2.2.jar (new parser) DAS parser is now assumed to handle escaping and remove "" from strings Length check is now zero based for Grib(1|2)PDSVariables Grib now uses CF-1.4, other CF global attributes were modified Fmrc: parameterized best dataset DatasetCollectionManager delete old metadata entries FmrcInvLite, time coord variants monotonize Lon coordinate axis Christian Ward-Garrison improvements on FileWriter NetcdfFileWritable: check for record variable on rewrite, unlimited dimension must be first Aggregation.closeDataset() : dont null out the dataset list. StringUtil.split() refactor Fmrc implementation to eliminate retained fmrcInv, orgVar.. ucar.nc2.util.NamedAnything CoordinateAxis1D: defer constructing edge, bounds arrays. GridCoordSys: defer constructing times, levels arrays. 1: (2010-03-29) try to get IndexConstant right Section.reduce() EnumTypedef is immutable Variable.getCachedData() convenience NetcdfFile.getDetailInfo : show cached variables, proxyReader ToolsUI deferred construction, remove DQCchooser tab DLCrawler improvememts ComboBox - context menu to delete items refactor Fmrc implementation Changes in version 4.1: 15: VariableDS.getEnhanceMode is public GridCoordSys accepts runtime or time bug in N3header when nvars = 0. 14: fix union aggregation when there's Groups DPA added to NIDS Nowrad iosp clarify MSGnavigation parameters improved Cosmic1Convention NsslRadarMosaic Convention: dont use the NcML, test for variable names Grib: Curvilinear Orthogonal projection now works add Aggregation.FIRST as option for choosing prototype earth_radius attribute on grid_mapping variable must be in meters 13: new visadNoDods.jar N3iosp, N3header, RAF: simplify flushing, file.setLength() N3 writing: bug in redefine mode. HDF4, HDF-EOS dimensions: replace " " with "_" HDF4: deal with compress.compress_type == TagEnum.COMP_CODE_NONE Switch to use unidataCommon.jar. hawaii station name changed GRIB: Made modification for partial use of Pressure coordinates for Hybrid variables MSGnavigation must override latLonToProjBB(LatLonRect rect) 12: HDF5 IOSP now sets the Attribute Array unsigned flag; added a convenience routine Attribute.isUnsigned() BUFR: add ecmwf, fnmoc, clean up table names Cosmic CSB improvements McIdasGrid.isValid() Gempak logging WRF v3 lat/lon grids. works, but ignores coordsys lat/lon time dependency (is there any?) UTM projection bug NcML write: fix possible problems with '%' in string values BUFR: process 2-1-Y "change width" for enums and chars, ignore 2-6-0 (fnmoc uses as cancel) BUFR: compressed data: permutation wrong when ndims > 2, ie more than one nested structure BUFR: use new WMO CodeFlagTable 11: Grib1BinaryDataSection get data array dimension from Grib1Data when bit-map size not available CEDRIC radar coordSysBuilder BUFR can read JUSA41 isGridCoordSys: improve performance. should optimize for 2D coords, maybe wait for swath datatype? NcMLReader bug when no reference dataset BUFR tables refactor UI tabs for examining BUFR tables B and D refactor BUFR tables bug in NetcdfDatasetConvert on converting enums in nested structures 10: multiple profiles in the FslRaob plugin StationProfileFeature.getTimes(), getProfileByDate() Aggregation bug with netcdf elements with no referenced dataset Remove nested structures from PointFeature where needed ProfileFeature now has a nominal time Default Conventions look for altitude in description DateUnit checks for NaN various improvements to Bufr IOSP UI 9: NetdfDataset, VariableDS: do not share dimensions with orgVar, make new copy. messes up NcML rename dimension with file caching BUFR fixes - combine countBits and read Variable.getShapeAll() Array.reshapeNoCopy() ArrayStructureMA.factoryMA() add Point viewing widget CF doesnt require feature type - will default to point point datasets: NLDN, USPLN, Madis-ACARS add SequenceDS extends StructureDS Add GEMPAK sounding IOSP NetcdfFile, IOSP getStructureIterator() for top-level sequences BUFR edition 4 had month+1 on reference time OPeNDAP scalars like NcSDInt32 were assuming Array.D0 8: Add -isLargeFile option to FileWriter to generate 64-bit offset files DefaultConventions: allow multiple forced time coords Grib: precipitation accumulation, hybrid sigma levels point datasets: CF-1.5, Suomi, FslWindProfiler, Cosmic, GempakSurface, Madis, UnidataObs, GempakSounding catch illegal bounds shapes in CoordAxis1D.makeBounds() UAMIV iosp not handling file seperator generally Handle missing values in CFPointObs, NestedTables. Allow missing_value processing of CHAR dataType. empty string = 0. StructurePseudo moved to ucar.nc2.dataset.StructurePseudoDS because we need missing values; ucar.nc2.StructurePseudo now deprecated 7: H5iosp: deal with compressed, chunked compound types ucar.nc2.iosp.LayoutBB now a subclass of Layout ucar.nc2.dt.point.StandardSection implementation, add to PointFeatureDatasetViewer HdfEos: look for XDim/YDim coordinates with GCTP_GEO projection, these are lat/lon CPpointObs: clarify the variants. Variable.getNameAndDimensions() had bug when theres a '%' in the name. UAMIV iosp calls finish() not sync() NetcdfDataset.enhance() now returns the CoordSysBuilder for debugging purposes fix NetdfDatasetInfo.getParseInfo() improve HDF-EOS processing. add earth_radius parameter to AlbersEqualArea, LambertAzimuthEqualArea, LambertConformalConic refactor Earth, EarthEllipsoidal. Use inverse flattening as parameter, consolidate constructors, use proj4 calculations of ellipse parameters add ucar.unidata.geoloc.projection.proj4 for jhlab java versions of proj4 code; AlbersEqualAreaEllipse, LambertConformalConicEllipse. 6: nasty bug in Index.scalarIndexImmutable redo NCdumpW.writeNcML to use Formatter fix sync() in all iosps we control ucar.nc2.ft.point.standard.Evaluator() uses equals() instead of == add pointstream error handling; add NcStream.Error message aggExisting caching now works correctly cdmremote point: get DateRange filter working optionally add time resolution in server config CdmRemoteFeatureDataset gets bb, dateRange from capabilities doc cdmRemote req=form ncstream anon dimensions cdmRemote east/west flipped 5: AggFmrc: 2D time coordinate switch to double (when numeric) to avoid round-off. URLDumpPane, TdsReadLogs : escape query string NppConvention for NPOESS 4: ui/iosp: Grib2 Table Array.createView was not propagating isUnsigned() ui.StructureTable also shows nominal time if possible PointFeatureImpl checks if nominal time == 0, if so use obsTime. workaround for protobuf lask of NaN default refactor station point classes CdmRemoteFeatureDatset is not a CdmRemote optimise hashmap in StructureMembers GridTableLookup.Curvilinear requires external 2D lat/lon 3: qunhe@unc.edu minor changes for UTM ToolsUI: ncml prototypes in NcmlEditor pane hack enhance back into ncml scan element, needed for WRF and anyone who needs CoordSysBuilder intervention more GRIB ensemble work cdmremote working again composite stations work correctly 2: StationPointFeature, StationCollection.flatten catalog2 refactor 1: GRIB library 8.0 Back out DateUnit changes - too many small glitches - revert to 4.0 code base UAMIVServiceProvider looks for camxproj.txt for config info ScaleMissingEnhance: assume if forVar isUnsigned, data conversion should use unsigned data, regardless of orgVar Contiguous Table : dont add startRaggedArray variable, keep internal GridCoordSys.convertUnits(): convert to DOUBLE to avoid loss of precision NetcdfDataset calls CoordSysBuilder.augmentDataset(), then processes scale/missing, then calls buildCoordSys() Variable.slice() : ok on record dimension size = 0 FeatureDatasetPointXML for creating capabilities document for FeatureDataset Try to make nested Table configuration XML NcStreamRemote -> CdmRemote 0: GridDataset: getVariables() only returns the grids AggExistingOne : make time coord into a String DODNetcdfFile: workaround for Grid HDF4 wierdness (Float64 Longitude[fakeDim2 = 8192]) InvDatasetImpl : check if parent is null VariableDS only sets orgDataType when its wrapping a variable. AggregationOuterDimension tolerates different types between main variable and nested AggregationOuterDimension.cacheVar may keep data as different type, make sure to convert to correct type in read() MAMath.convert - utility routine for converting Arrays to different types. Cosmic data - FeatureType plug AggregationOuterDimension.rebuildDatasets(): bug on updating datasets, assuming agg vars are 1D (!) HDF4 images use unshared dimensions ToolsUI/IOSP/HDF4 CFConventions look for standard name attribute for latitude / longitude GridCoordSys: 2D Coordinate Systems now can find index (i,j) from (lat, lon) coord CoordinateAxis2D: do not cache data, messes up strided data access NcML: values can be specified by fromAttribute filesystem caching in Aggregation, PointCollection DateUnit refactor - use udunit.conveCoordSysrt() instead of TimeUnit as base time udunits numeric timezone bug Changes in version 4.0: 48: WRF Conventions looks for GRIDTYPE if it cant find DYN_OPT improve M3IO CoordSysBuilder, from Qun He Stereographic Projection, from the "natural origin" and the tangent point, calculating the scale factor. ToolsUI: Viewer: compare two files, global attributes table ToolsUI: NcML Aggregation tab: check for homogeneity. AggFmrc: ragged, set missing values to NaN DConnect2: allowSessions now defaults to true, so "X-Accept-Session: true" header is sent (can use ucar.nc2.dods.DODSNetcdfFile.setAllowSessions() to change) 47: NcMLReader.merge -> mergeNcML : create new Dataset setValues.setValues (NcmlReader) : scalar Array need to be reshaped new IOSP: ucar.nc2.iosp.mcidas.McIDASGridServiceProvider CF1Convention.getZisPositive() - use in WriterCFPointObsDataset atmosphere_hybrid_sigma_pressure_coordinate: ap(z) + b(z)*surfacePressure(x,y) 46: udunit fixes for logarithmic units StructurePsuedo: bug in _read(section), returning wrong ArrayStructure.getShape(). PointTable: refactor and document, esp for CF Structure.select(), StructurePseudo() : use member names, not Variable object NetcdfWriteable: check for valid DataType, improve error messages. was allowing addVariable(.., DataType.STRING) causing invalid vsize. netcdf-java read ok, c library doesnt. NcML: logical view (not done) NCMLReader: allow use of alternate CrawlableDataset impls in NcML aggregation. LambertConformal: crossSeam() true when distance > 20,000. ProjectionImpl.latLonToProjBB: dont check crosssSeam() ProjectionImpl: now create a new object when needed in latlonToProj(), projToLatlon() - too susceptible to error otherwise. NetcdfCFWriter: max 2 GB files. NetcdfCFWriter: check if timeAxis is null when dateRange is passed in. IOSP and NetcdfFile new methods: .getFileTypeId(), getFileTypeVersion(), getFileTypeDescription() 45: ToolsUI: fine tune point obs display ToolsUI: eliminate multiple combobox events by waiting 10 msecs dont rely on dim.isUnlimited() to test if theres a record structure NetcdfDataset.setDefaultEnhanceMode(Set mode); GEMPAK bug workaround (from Don) Nexrad2,3 coord caching COARDSConvention: withdraw "SimpleUnit.isCompatible("m", unit), must have "positive" attibute ncstream uses URLEncode for passing UTF-8 var names. N3iosp.sync() checks for numrec < 0 (isStreaming) Fmrc: remove global atttributes taken from random prototype dataset fix readToByteChannel() problems refactor ncstream, another pass at protobuf messsages GempakCdm point adapter now does stationAsPoint. JoinArray is parameterized Table.extraJoins now is a list. Stucture.Iterator now handles general rank case. StructureDS.Itertaor is removed AvhrrConvention bug CoordinateAxis.copyNoCache() must preserve state. caused grid.NetcdfCFWriter to fail when x,y units != km. 44: TDSRadarDatasetCollection improvements AvhrrSatellite Convention - start of Image datatype ArrayChar.makeString() needs to check string len CFpointObs: station unlimited dimension case Array.arrayCopy() now deals with src not in canonical order. mostly affected aggregation of datasets which do sectioning in memory, such as GINI. TDS: remote station data add List StationCollection.getStations(String[] names); CdmRemoteDatasetFactory now handles any type of cdmRemote dataset, including Grid. 43: add Variable.setCachedData check for correct array type WRF, ADAS CoordSysBuilder makeSoilCoordintae had wrong datatype ToolsUI Viewer: write local file NetcdfFileWriteable: change att names to legal and give warning rather than exception 42: Fmrc must enhance datasets when acquiring. Use FeatureType.ANY instead of FeatureType.NONE upgrade protobuf library to 2.1.0 grib index performance enhancements ucar.units enhancement nexrad2 bug fixes ucar.nc2.dataset.grid is removed, since it can no longer be made to work. VerticalTransformCT no longer keeps copy of VerticalTransform, since different coordSys may need different functions. Now the VerticalTransform is kept in GridCoordSys, which is the only place that it was used. VerticalTransformCT is now immutable, so it can be shared safely. FmrcImpl: dont remove _CoordinateAxes ncml: check namespace is correct enhanceProxy: throw exception if coordSystem is null cdmremote replaces ncstream point: add calcBounds() to iterator, collections, datasets 40: CF 1.4 multidim case when time is 1 dimensional overflow in IndexChunker when writing large files DateType was accepting invalid ISO strings. grib indexing: binary indexes dont return unknown params grib1 param table reading: make thread safe for TDS new visad.jar Bufr: was not getting missing data correct when compressed add: GridCoordSystem.findXYindexFromLatLonBounded(), findXYindexFromCoordBounded() 39: add uamiv iosp, from Barron Henderson (UNC Chapel Hill) HdfEos distinguish between swath and profiles GridCoordSys.isGridCoordSys() checks for {x,y} having at least 2 dimensions. to exclude point data add CoordinateSystem.makeDomain() add NetcdfFile.findGroup() Array.getFastIterator is deprecated; remove use in MAMath which fails on section add NestedPointFeatureCollection.subset(latLonBB) vertical transforms OceanS modified, OceanSG1 and OceanSG2 added, thanks to Sachin Bhate and Rich Signell. TdsMonitor application in UI IO.readContents(InputStream is, String charset); UI: DateFormatMArk in Units panel AlbersEqualArea : falseEasting, falseNorthing (with help from R.Schmunk) UI: URLDump handles compressed body (shows compressed ratio) and charset. NetcdfDataset.sync() now actually works. WRF sets MAP_PROJ=0 to flat earth for non-georeferencing runs NetcdfDataset.wrap() ToolsUI: Fmrc, Aggregation Panels improved Fmrc : only use Grids with time coordinate NetcdfDataset.enhance() uses Set, not EnumSet in order to get unmodifiable set. Not sure if this breaks the API. GridDataset always enhances NetcdfDataset with coordSystem, if not already done NetcdfDataset.setEnhanceProcessed, isEnhanceProcessed, setCoordSysWereAdded removed VariableDS, StuctureDS.clearCoordinateSystems() caleld from NetcdfDataset.clearCoordinateSystems() NcML enhance attribute is deprecated - and is now ignored. 38: FmrcImpl - add coordinate systems to component datasets CoordinateAxis1DTime.NamedObjecct - do not retain, used only by UI. NetcdfFile.finalize() makes TDS run out of memory when heavily loaded SimpleUnit gets new UnitFormatManager each time, rather than using a static that becomes threading bottleneck GridServiceProvider.readXY() : dont catch Exception Fmrc: put "coordinates" attribute back on grid variables. new visad.jar (3/20/09) GeoGrid.subset() deal with 6 dimensions, 2D time dimension. refactor grib iosp: GribGridServiceProvider. binary indexes. ToolsUI: added Fmrc/Inventory, NcML/Aggregation Tabs 37: CoordinateTimeAxis1D.findTimeIndexFromDate() now returns 0 when date < first. N3iosp.createValidNetcdf3ObjectName() change to new allowable names ToolsUI: IOSP/HDF5 H5header fractal heap. Netxrad2, NetcdfFile.makeCompressed - use FileLock to prevent clobbering between threads FileCache cleanup scheduling - make sure only one at a time make sure NetcdfFile closes raf on Exception in spi.open() reinstate NetcdfFile.finalize() grib now does filtering on variable names to keep them consistent with nj22. joinExisting timeUnitsChange - time variable must be Double, else fractional values get lost. 36: NcMLReader was mistakenly throwing exception when there was no variable shape attribute. That is allowed, but deprecated. NcMLWriter now always adds that attribute. ucar.units hashCode consistent with equals() Gempak IOSP now uses its own Convention, not CF. Add PointFeature processing. InvDatasetFmrc prevents FmrcImpl from being closed, with FileCacheNOP CoordAxis1D.findIndex() off-by-one bug for irregular descending case. FileCache now has name and nicer log messages; log to 'cacheLogger' NetcdfFile.isClosed didnt work for cache 35: NcML: allows elements to modify each dataset in the aggregation, before it is aggregated. this allows datasets to be made into grids for fmrc, for example. Some improvements in NcML error messages. CFpointObs improved, handles gempak structures. JoinArray added, Join interface clarified Table.getStructureDataIterator(Cursor cursor, ...) add _CoordinateSystemFor to allow wholesale assignment of coord systems to variables. Gempak IOSP now standard cleanup NetcdfFile.isClosed(). Make sure you get error if you call read() after close, even in cache/ 34: Fmrc.makeDatasets must clear coordinate system. GINI polar stereographic bug grid/grib/gempak refactor ToolsUI/FeatureTypes/WMS is rudimentory WMS client FeatureDataset extends FileCacheable, so they can be cached directly 33: refactor ThreddsDataFactory add Radial datasets to FeatureDatasetFactoryManager; TypedDatasetFactory now obsolete experimental PointDatasetFromNcStream fysat iosp grib refactor 32: Stereographic false_east/north implemented CoordinateAxis.section() uses copy() not copying all fields. Move CoordinateAxis.POSITIVE_XX to ucar.nc2.constants.CF GDV Conventions: dont check for meters but COARDS has to CFPointObs station ragged put thredds/wcs back into netdf.jar for now NetcdfDataset.main() now is a cover to FileWriter. CoordinateAxis1D - dont convert misssing values to NaNs, missing values not alllowed so likely a user metadata error ArrayChar.makeFromString() CFPointObWriter more bugs N3streamWriter takes an optional numrec, if numrec are know in advance to avoid new streaming format NetcdfDataset : disambiguate bare http URL requests, use HttpClient 31: UnidataPointObs : allow POINT type even if really a STATION. NSSLRadarConvention checks no Convention is set. COARDS : look for units of meters for vertical coords (again), when all else fails. DefaultCoordBuilder no longer extends COARDS, to avoid such nonsense. ncStream uses protocol buffers (experimental) nimbus NPE BufrTable UI: dump distinct DDS RAF: fix writeToStream Array.factory(ByteBuffer) bug in PointFeatureSubset iterator. move Station, EarthLocation to ucar.unidata.geoloc 30: Modis Satellite CoordSysBuilder Cosmic profiles HDF5 fletcher32 jpeg2000Unpacking Exception Mercator false east/northing Nimbus/RAF Convention: try to guess more coordinates add ucar.ma2.StructuresDataComposite fix CFPointObWriter to correctly handle CHAR add FeatureType selection to PointFeature tab context menu on FeatureScan FeatureDatasetFactory was not threadsafe, isMine() returns "analysis object" PointFeatures: standard package refactor; add Table.Type.Top for top-level variables PointFeatureDatasetViewer: works with all types except Section. add 100 obs limit for speed Bug in JoinExisting aggs with > 1 dataset, when taking a section of a cached variable. Try to reduce memory usage in CrawlableDataset. Grib iosp: dont swallow exception on data reading error NetcdfFile : remove finalize() for now N3iosp.close() : check for null raf, set raf to null on close. remove VariableDS.isCoordinateVariable() - must use classic sense of coordinate variable add Aggegation info to NetcdfDataset.getDetailInfo() refactor NcML Aggregation MetadataExtractor.extractDateRange() - deal with no time axis at all. 29: Rotated Lat/lon projection JoinExisting caching now handles nested aggregations, by using the id attribute to identify them. GridDataset, RadialDataset: use Formatter instead of StringBuilder VariableIF use Formatter instead of StringBuilder CoordTransBuilder use Formatter instead of StringBuilder add getCatalogRef(InvCatalogRef dd) to CatalogGrawler.Listener CoordAxis1D.findCoordElement() bug for Longitude MAMath.convert2packed() utility for scale/offset writing Change _unsigned to _Unsigned; look for _Unsigned but ignore case. 28: EarthLocation.isMissing() Improvements to CoordAxis1D.findCoordElement(coord) Start to fix CoordSysBuilder to deal with groups. DefaultConventions - split from GDV. Nested Tables implementation FeatureType (ucar.nc2.ft) framework. mostly POINT, some STATION Bugs and performance improvements for StructureDS TDWR into NIDS iosp Grib2 tolerate missing parameters 27: Start implementing CF point data spec in FeatureType (ucar.nc2.ft) framework Point data: RecordHelper using wrong StructureMembers - always need to get from curent StructureData/Array object! Range, Section: use Range.VLEN for size < 0. H5iosp: structure member reading had wrong byteOrder VariableDS.lookupEnum must defer to orgVar H5header: redo String, char attributes; _FillValue attribute vs Message H5iosp: exception if unknown filter. HDF5 shuffle filter was inverted HDF5 char attributes are Strings (UTF-8) DataType.getClass is deprecated - use DataType.getPrimitiveClass Array.isUnsigned(): ArrayByte, ArrayShort, ArrayInt all check when doing conversions; NetcdfFile.readData() sets this based on Variable.isUnsigned(). DataType.OPAQUE : use ByteBuffers wrapped by ArrayObject Array.set() made public for efficiency, eg in Attributes bugs in enhance handling and NcML JNI interface to netcdf4 C library, for testing 26: ucar.nc2.ft.point.writer.CFPointObWriter, WriterCFPointObsDataset for writig point obs data in proposed CF format. EnhancedScaleMissingImpl: deal with unsigned data and missing values Universal Format Radar IOSP NetcdfDatasetInfo must reopen dataset, capture CoordSysBuilder info, so that normal case doesnt retain unneeded objects 25: opendap parsing bug with signed bytes ncml agg fixes grib thread-safety Calendar.clear() to eliminate random millisecs in a new Date NIDS: raster, DPA read bug BUFR tables cleanup add Grib GDS type 87 NCDumpW -ncml -vall now works cache cleanup only worked once (!) agg files were being opened twice under some circumstances FileCache hard limit 24: cleanup error messages when catref or show catalog service fails to open remote catalog. HTTPRandomAccessFile tolerates lack of "Accept-Ranges" header bug in IO.copyUrlB buffering - was very slow, prone to hang start to reduce memory consumption based on TDS profiling: - remove ucar.ma2.Index.set/getName() since its never used - NetcdfDataset does not retain NetcdfDatasetInfo - remove GribServiceProvider.parseInfo - DateUnit does not retain DateFormatter or timeUnitString - LatLonPointImpl does not preallocate StringBuilders 23: new visad.jar (9/12/08) more BUFR fixes DateRange now moves when start or end date = "present" NetcdfFile.canOpen() GeoCoordSys: warn, dont fail, if x, y axis units not compatible with "km" mcidas area iosp 22: improved BUFR reading 21: Improve nexrad3 iosp GeoTiff: new projections 20: StructureDS: get nested convert to work Mercator projection: remove lat0; grib parameters were interpreted wrong Allow opening GRIB files where we dont know the grid type, and so cant add coordinates HDF5 bug: filtered data (eg compressed) wasnt checking for byte order. add 'nodods':, 'slurp:' as aliases for http: enums return values, enhance will convert to Strings redo NetcdfDataset.EnhanceMode, use EnumSet ENUM1, ENUM2, ENUM4 enum types BUFR code tables (still experimental) BUFR "associated variables" VariableDS extends EnhanceScaleMissing, StructureDS and VariableEnhanced does not 19: BUFR reading mostly works turn off debugging StructureDS.convert nested structures 18: fix large file writing in agg, fix CacheVar, joinExisting persistance. move NetcdfDataset.makeArray to Array.makeArray, add new variant UnidataPointObsDataset does not require bounding box in global attributes NetcdfFileWriteable will allow object names to start with a digit ToolsUI : allow setting file caching add NcdumpW.printArray for vanilla array printing CoordSysBuilder allows coordinate axes to have coordinate systems move McIDASAreaProjection, McIDASAreaTransformBuilder to ucar.nc2.iosp.mcidas change AxisType, DataType, FeatureType, TransformType into a java 1.5 enum WRFConvention tolerates not having GLAT,GLON fields bug fixes in BUFR NcMLWriter.writeXMLexplicit puts in NcML fix bug in NCdumpW.print sequences 17: HTTPRandomAccessFile allows content-length to be long fix NcML aggregation unit tests; add new ones integrate aggregation with FileCache, experimental concurrency fix bug with relative URLs with blanks in them (URLnaming) finish new agg features : promoteGlobalAttribute, timeUnitsChange 16: remove direct use of ucar.units, except from ucar.nc2.units rewrite NetcdfFileCache, remove NetcdfDatasetCache EnhanceScaleMissingImpl uses Misc.closeEnough for missing value comparisions. default VariableDS.useNaNs = true ucar.nc2.util.cache.FileCache uses FileCacheable, FileFactory interfaces; implement NetcdfFileCache, HTTP file cache with FileCache. start adding net.jcip.annotations to document thread safetyAgg convertScaleOffsetMissing() methods back into VariableEnhanced clean up Aggregation, make sure docs are correct clean up NetcdfDataset.EnhanceMode, add deferred option 15: change StringBuffer to StringBuilder for efficiency clean up ArrayStructure/StructureData API, clarify Member use StructureData.getScalarXXX(name) must be correct type cleanup javadoc, improve reference docs NetcdfFileWriteable - redefine mode, large file support, extra header bytes library upgrades : JDOM 1.1 NetcdfFile.readSection(secSpec) used in NcDumpPanel, replaces old readNestedData. 14: NetcdfDataset: unsigned always gets converted Netcdf-3 files: truncate Strings at first null byte. complete rework of point datatypes, now in package ucar.nc2.ft (feature types) many improvements to Structure, ArrayStructure etc rewrite BUFR and supporting library, improve tables, etc make sure compiles under 1.5 13: hdf4 attribute arrays 12: DODS preload put back HttpClientManager, RuntimeConfigParser temporarily (testing with IDV) hdf4 bug fixes experiment with ucar.nc2.dt2 iosp.NmcObsLegacy DataType Sequence 11: start to fix name/shortName, ie deal with Groups systematically Structure/ArrayStructure/StructureData refactor 10: hdf4 bug fix - no data tag, use missing data Structure.subset(), nc3 optimizes for it hdf5, ArrayStructureBB : set byteOrder ArrayStructureBB puts Strings on "heap" rearrange packages to separate modules cdm build redone - xxxComplete.jar added fix problem preventing 1.5 compilation (ternary op returns array) 09: hdf5 byteOrder must be set when chunked (tiledLayout) --------------------- Overview of changes from 2.2 Data Model * Object names are Unicode Strings. No Restrictions on characters used. * DataType STRING are Unicode. In Netcdf3/4, stored as UTF-8 encoded. * DataType CHAR are uninterpreted bytes, default is to consider them ascii. * DataType ENUM : stored as 1,2,4 byte ints, map to Strings * DataType OPAQUE : byte[] cant be subsetted. * DataType SEQUENCE : Structure(*) only accessible through StructureDataIterator Features: netcdf * Structure.select(): create subset consisting only of selected member variables * StructureDS correctly gets enhanced * NetcdfFileWriteable supports large file writing, redefine mode, extra header bytes, preallocating file size, all for Netcdf-3 files. iosp * netcdf-4 * complete HDF5 access * complete HDF4 access * parsing of HDF-EOS/HDF5-EOS metadata to identify dimensions. * GEMPAK grids * NMC Office Note 29/143 * BUFR (edition 2-4) * Mcidas AREA * Universal Radar Format geoloc * Add RotatedLatLon projection * Add RotatedPole projection ft * Point Feature Types completely reworked, based on StructureData iterators NcML * Tiled aggregation * A JoinExisting type now accepts timeUnitsChange, ndicating time units change in the differnt files * A JoinExisting type now accepts dateFormatMark, if theres only one time step per file, to specify the agg coord value * Union accepts scan * Global attributes can be promoted to Variables. * Can specify variables to cache. * Can specify iosp, a string parameter to the iosp, and the RAF buffer size Libraries * JDOM 1.1 * updated visad 9/12/08