Hi Bob:
a few more comments below (more later)
Attached are 5 .nc point files created by me (all use the various "ragged array (contiguous) representations") and 1 file (epa) created by the SURA group. All are intended to follow the proposed CF Point Observations conventions at
https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions .
But since this is new to us, we may have made mistakes -- that could certainly be the source of the problems below. But I see nothing wrong with the 5 files created by me.
Basically, I couldn't read data from any file except a Point file (not attached).
I'm using a very recent version of netcdf-java 4.2.
(Some) Problems
* For Profile.nc, Station.nc, Trajectory.nc, (and TrajectoryProfile.nc and epa+seamap_04-08_WD23.nc are analogous), I see the same problem:
I use flatten() to get one PointFeatureCollection
(or the iterator to get each e.g., ProfileFeature, which is a PointFeatureCollection))
then I use the internal iterator to get the PointFeatures
and then pointFeature.getData().getMembers to get the members
then getArray to get the data.
But the size of each array varies.
For longitude, latitude, time, rowSize, (station), the size
is the number of (stations).
For all the actual data variables, the size is one.
I would have expected the sizes to all be 1.
* For TrajectoryProfile.nc (for a SectionFeatureCollection) my code uses layers of iterators, but is roughly similar to e.g, Profile.nc described above. * The first problem described above (different sized arrays) occurs here, too. * There are no members for the actual data variables. * member=bottle_posn getDataArray() always returns null! (For one of the other files, member=altitude getDataArray() always returns null.)
* The problems with epa+seamap_04-08_WD23.nc are similar to the problems with TrajectoryProfile.nc file.
* I never got flatten or subset to work with a dateRange (even with a huge range, so it should catch all data). Instead, using dateRange returns a null PointFeatureCollection. So I always use dateRange=null.
* The JavaDocs say EarthLocation always uses longitude +/-180!
Does that mean that the underlying data always uses longitude +/-180?
Does that mean that when I create files with longitude 0 - 360, that I should convert to +/-180?
What about trajectories that cross -180?
* For the StationProfile.nc file, FeatureDatasetFactoryManager.open() says "Must have a Station id variable with standard name station_id", but I think it does.
this has the station and profile dimensions mixed up. probably a bug.
* For SectionFeatureCollection and StationProfileFeatureCollection,
if I use flatten, then try to use the internal iterator's hasNext() to see the PointFeatures in the PointFeatureCollection, I get
Exception in thread "main" java.lang.UnsupportedOperationException: SectionCollectionImpl does not implement getPointFeatureCollectionIterat
or()
at ucar.nc2.ft.point.SectionCollectionImpl.getPointFeatureCollectionIterator(SectionCollectionImpl.java:60)
at ucar.nc2.ft.point.MultipleNestedPointCollectionImpl$NestedPointFeatureCollectionFlatten.getPointFeatureIterator(MultipleNestedPoi
ntCollectionImpl.java:108)
at ucar.nc2.ft.point.PointCollectionImpl.resetIteration(PointCollectionImpl.java:89)
at ucar.nc2.ft.point.PointCollectionImpl.hasNext(PointCollectionImpl.java:75)
at gov.noaa.pfel.erddap.dataset.EDDTableFromNcCFFiles.addDataFromPointFeatureCollection(EDDTableFromNcCFFiles.java:472)
...
* Most of the interfaces don't offer subsetting methods that I would expect, e.g., StationProfileFeatureCollection has no subset(latLonRect).
* There is little guidance in the JavaDocs: * I couldn't find any complete example for how to read (including subset) data files. It took a long time to put all the pieces together, especially since many of the important classes are just interfaces (and so not referenced in other java docs). * I open files with FeatureDatasetFactoryManager.open(). Is that the recommended way? * My way of going from FeatureCollection to a specific ...Collection interfaces is just to repeated test e.g., if (fc instance of ProfileFeatureCollection) {...} then cast to that interface. Is there a better way? * Somewhere it says it is imperative that I call hasNext()/next() to the end (or finish()). Is that true even if an exception if thrown? So do I need to use try/catch around everything? Won't things be garbage collected when they fall out of scope? * What is efficient? E.g., when is subset() or flatten() with a latLonRect or timeRange efficient vs. not? * Does resetIteration() for the internal iterator need to be called before initial use of the internal iterators? (I saw some sample code that showed this.)
I am probably making mistakes (either when creating the files or reading them). Sorry. But I'm stuck. I can't get anything to work. So any guidance you can provide is appreciated.
Thank you for looking into this.
P.S. If you have other test files for each of the cdm_data_types, can you please send them to me since I only have "ragged array (contiguous) representations" files? I want my code to be able to read all of the possible representations (e.g., multidimensional, ragged array (indexed), single, flattened, ...) of all of the feature types. (E.g., I have no examples of Radial or StationRadial datafiles.) Since flatten() doesn't work reliably, all of the cases don't just resolve to a couple of uses of flatten(). I have to write a little code for each feature type to iterate through various levels down to PointFeatureCollections. It would be great if flatten (with latLonRange and dateRange) worked reliably and efficiently.
Thank you very much.
On 2/1/2011 1:10 PM, John Caron wrote:Hi Bob:
On 1/31/2011 10:26 AM, Bob Simons wrote:Please ignore my previous email.
If I use NestedPointFeatureCollection npfc = (NestedPointFeatureCollection)stationProfileFeatureCollection; PointFeatureCollection pfc = npfc.flatten(latLonRect, dateRange);
A call to pfc.resetIteration() throws
Exception in thread "main" java.lang.UnsupportedOperationException:
StationProfileFeatureCollection does not implement getPointFeatureCollec
tionIterator()
Why not?
That seems to be the best (only?) way to specify a latLonRect and dateRange, and I *want* flattened data.
Im not sure why thats failing, it may be the implementation is not complete or something deeper. Can you send me a sample file?
Perhaps I'm making a mistake.
Just curious (I have to ask, because one time, for other code, the answer was 'no'), is all of the Point Feature Collection code finished and ready to be used?
Im thinking the iteration should work the first time through, ie without calling resetIteration(), just start calling pfc.hasNext() / next() ??
Sincerely,
Bob Simons IT Specialist Environmental Research Division NOAA Southwest Fisheries Science Center 1352 Lighthouse Ave Pacific Grove, CA 93950-2079 Phone: (831)658-3205 Fax: (831)648-8440 Email: address@hidden
The contents of this message are mine personally and do not necessarily reflect any position of the Government or the National Oceanic and Atmospheric Administration. <>< <>< <>< <>< <>< <>< <>< <>< <><
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.