Re: [IDV #SNE-677532]: My NcML has stopped working in the 2.7 released today

[Don Murray <dmurray@xxxxxxxxxxxxxxxx>]

Hi Keith-

Sorry for this slipping in at the last moment.  We didn't have this in 
our test cases so it slipped through.  The change was necessary so we 
can support more of the WRF products.  Glad you were able to modify the 
NcML to get it working. Perhaps we can touch on the conventions issue 
when we meet on Wednesday.

Don

Keith Searight wrote:
> Thanks,  I was able to use NcML to load the file without having to 
> regenerate it.  FYI, I used <attribute name="DYN_OPT" type="short" 
> value="2" />.
>
> BTW, I completely agree with your recommendations on using CF 
> conventions and I expect that to happen as these data products are moved 
> from research to operational deployment.  IDV may help spur this more 
> quickly.
>
> Keith
>
> Keith Searigh
> NCAR/RAL
> searight@xxxxxxxx
> 303.497.8416
>
>
>
> Unidata netCDF Java Support wrote:
> > add DYN_OPT= 2, assuming this is the ARW core (im pretty sure it is), 
> > but better is to preserve all the global attributes from the original 
> > files, since i dont know what others will be needed in the future.
> >
> > BTW: "you could also lobby the WRF group to publish and follow some 
> > conventions in their output" -> "you could also lobby the RAL group to 
> > publish and follow some conventions in their output"
> >
> >
> >  
> > > OK....  Can you please tell me what value the DYN_OPT attribute needs to
> > > be set to so that we can start using the new IDV release?
> > >
> > > Keith
> > >
> > > Keith Searight
> > > NCAR/RAL
> > > searight@xxxxxxxx
> > > 303.497.8416
> > >
> > >
> > > Unidata netCDF Java Support wrote:
> > >    
> > > > If you are generating it, why not use CF Convention?
> > > >
> > > > The reason it broke is because it differs from WRF global attributes 
> > > > (no DYN_OPT attribute). If you put that back in, it will work.
> > > >
> > > > The fact that it used to work is actually a bug that got fixed, in 
> > > > order to distinguish which dynamic core is being used.
> > > >
> > > > We dont have an incoming ftp server. Youll have to put it on your 
> > > > own ftp or web server.
> > > >
> > > >
> > > >      
> > > > > The 2 files you have are both different from this one.  This is a RAL
> > > > > internally generated format that is not from the WRF group.  We could
> > > > > certainly change it, but it did load in IDV v2.6u2 and v2.7b2, meaning
> > > > > something must have changed in the code since then.  Can you give me
> > > > > instructions on where I can FTP a sample to Unidata?
> > > > >
> > > > > Thanks,
> > > > > Keith
> > > > >
> > > > > Keith Searight
> > > > > NCAR/RAL
> > > > > searight@xxxxxxxx
> > > > > 303.497.8416
> > > > >
> > > > >
> > > > >
> > > > > Unidata netCDF Java Support wrote:
> > > > >
> > > > >        
> > > > > > I currently have std_d01_09-01_09-30_00z_1h.nc and 
> > > > > > std_d01_09-01_09-30_01z_1h.nc; not sure where these come from, but 
> > > > > > the two are different enough from each other to cause trouble.
> > > > > >
> > > > > > i see that your ncml references different names 
> > > > > > (max_d01_09-01_09-30_14z_1h.nc). send me a sample if these are 
> > > > > > different (put on web or ftp server)
> > > > > >
> > > > > > it would be helpful to know what program is writing these files. 
> > > > > > you could also lobby the WRF group to publish and follow some 
> > > > > > conventions in their output.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >          
> > > > > > > I think the output is similar, but it's not the same as WRF 
> > > > > > > netCDF.  I
> > > > > > > can send you a file if you'd like a sample (it's 16MB, too big 
> > > > > > > for an
> > > > > > > e-mail attachment).
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Keith
> > > > > > >
> > > > > > > Keith Searight
> > > > > > > NCAR/RAL
> > > > > > > searight@xxxxxxxx
> > > > > > > 303.497.8416
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Unidata netCDF Java Support wrote:
> > > > > > >
> > > > > > >
> > > > > > >            
> > > > > > > > Is this the output of the WRF model itself, or some postprocessor?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >              
> > > > > > > > > Hi Kieth-
> > > > > > > > >
> > > > > > > > > The IDV can't even read one of your files now, thus the 
> > > > > > > > > aggregation fails.  Something changed in the underlying netCDF 
> > > > > > > > > layer so these aren't recognized  as grids.  I'm transferring 
> > > > > > > > > this to the netcdf-Java support group so they can look into it.
> > > > > > > > >
> > > > > > > > > Don
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >                
> > > > > > > > > > Full Name: Keith Searight
> > > > > > > > > > Email Address: searight@xxxxxxxx
> > > > > > > > > > Organization: NCAR/RAL
> > > > > > > > > > Package Version: 2.7 build date:2009-06-25 07:08 UTC
> > > > > > > > > > Operating System: Windows XP
> > > > > > > > > > Hardware: Java: home: C:\Program Files\IDV_2.7\jre version: 
> > > > > > > > > > 1.6.0_11 j3d:1.5.2 fcs (build4)
> > > > > > > > > > Description of problem: There was an error loading the data:
> > > > > > > > > > Error creating data source:file.grid with: 
> > > > > > > > > > D:\SHOM\IDV\max_d1.ncml
> > > > > > > > > > Grid data source failed making data set: D:\SHOM\IDV\max_d1.ncml
> > > > > > > > > > AggregationExisting: no coordinate variable for agg dimension= 
> > > > > > > > > > Time
> > > > > > > > > >
> > > > > > > > > > See attached NcML file.  This loads fine in 2.6u2 and 2.7b2, 
> > > > > > > > > > so something in the new 2.7 release code has changed.
> > > > > > > > > >
> > > > > > > > > > ******************
> > > > > > > > > > Stack trace:
> > > > > > > > > > java.lang.IllegalArgumentException: AggregationExisting: no 
> > > > > > > > > > coordinate variable for agg dimension= Time
> > > > > > > > > > at 
> > > > > > > > > > ucar.nc2.ncml.AggregationExisting.buildNetcdfDataset(AggregationExisting.java:81) 
> > > > > > > > > >
> > > > > > > > > > at ucar.nc2.ncml.Aggregation.finish(Aggregation.java:295)
> > > > > > > > > > at ucar.nc2.ncml.NcMLReader.readNetcdf(NcMLReader.java:466)
> > > > > > > > > > at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:413)
> > > > > > > > > > at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:234)
> > > > > > > > > > at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:184)
> > > > > > > > > > at 
> > > > > > > > > > ucar.nc2.dataset.NetcdfDataset.acquireNcml(NetcdfDataset.java:788) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:664) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:434) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.nc2.dataset.NetcdfDataset.acquireDataset(NetcdfDataset.java:514) 
> > > > > > > > > >
> > > > > > > > > > at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:88)
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:1052) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:1074) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:443) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.data.DataManager.createDataSource(DataManager.java:1490) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1966) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1883) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1817) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1790) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:907) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.chooser.FileChooser.selectFilesInner(FileChooser.java:648) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.chooser.FileChooser.selectFiles(FileChooser.java:517) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > ucar.unidata.idv.chooser.FileChooser.doLoadInThread(FileChooser.java:539) 
> > > > > > > > > >
> > > > > > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > > > > > > > at 
> > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
> > > > > > > > > >
> > > > > > > > > > at 
> > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
> > > > > > > > > >
> > > > > > > > > > at java.lang.reflect.Method.invoke(Method.java:597)
> > > > > > > > > > at ucar.unidata.util.Misc$2.run(Misc.java:1033)
> > > > > > > > > > at ucar.unidata.util.Misc$3.run(Misc.java:1061)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >                   
> > > > > > > > Ticket Details
> > > > > > > > ===================
> > > > > > > > Ticket ID: SNE-677532
> > > > > > > > Department: Support netCDF Java
> > > > > > > > Priority: Normal
> > > > > > > > Status: Open
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >               
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: SNE-677532
> > > > > > Department: Support netCDF Java
> > > > > > Priority: Normal
> > > > > > Status: Open
> > > > > >
> > > > > >
> > > > > >           
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: SNE-677532
> > > > Department: Support netCDF Java
> > > > Priority: Normal
> > > > Status: Open
> > > >
> > > >       
> > >     
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: SNE-677532
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Closed
> >   

--
*************************************************************
Don Murray                               UCAR Unidata Program
dmurray@xxxxxxxxxxxxxxxx                        P.O. Box 3000
(303) 497-8628                              Boulder, CO 80307
http://www.unidata.ucar.edu/staff/donm
*************************************************************