Re: [netcdf-java] Reading sequences from BUFR with NetcdfDataset

Update:

This is how you can read the values from the sequence. Replace the
NCdumpW.printStructureData(new PrintWriter(System.out), nestedData);
with this
for (Member member : nestedData.getMembers()) {
        String name = member.getName();
        switch (member.getDataType()) {
        case INT:
                Integer value1 = nestedData.getScalarInt(member);
                System.out.println(name + " (Integer) = " + value1);
                break;
[... other types...]
        case CHAR:
        case STRING:
                String value2 = nestedData.getScalarString(member);
                System.out.println(name + " (String) = " + value2);
                break;
        default:
                System.out.println(name + " unknown");
        }
}

Netty

> Gesendet: Dienstag, 03. September 2019 um 09:20 Uhr
> Von: netty.jawn@xxxxxx
> An: netcdf-java@xxxxxxxxxxxxxxxx
> Betreff: Re: [netcdf-java] Reading sequences from BUFR with NetcdfDataset
>
> Hi John,
> 
> many thanks for your help! You saved my day!
> 
> All read data looks very well. 
> 
> BTW: 
> The exception, which I found also happens in ToolsUI:
> - Open toolsUI-5.0.0.jar
> - Click Viewer
> - Enable Enhance
> - Open my BUFR file
> - Click on Show data as shown in the screenshot
> 
> The full exception in the window is:
> java.lang.ClassCastException: ucar.ma2.ArrayObject$D1 cannot be cast to 
> ucar.ma2.ArrayStructure
>       at ucar.nc2.dataset.StructureDS.convert(StructureDS.java:268)
>       at ucar.nc2.dataset.SequenceDS.read(SequenceDS.java:84)
>       at ucar.nc2.ui.op.DatasetViewer.showData(DatasetViewer.java:566)
>       at ucar.nc2.ui.op.DatasetViewer.access$1000(DatasetViewer.java:59)
>       at 
> ucar.nc2.ui.op.DatasetViewer$NestedTable$3.actionPerformed(DatasetViewer.java:416)
>       at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
>       at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
>       at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
>       at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
>       at javax.swing.AbstractButton.doClick(Unknown Source)
>       at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
>       at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown 
> Source)
>       at java.awt.Component.processMouseEvent(Unknown Source)
>       at javax.swing.JComponent.processMouseEvent(Unknown Source)
>       at java.awt.Component.processEvent(Unknown Source)
>       at java.awt.Container.processEvent(Unknown Source)
>       at java.awt.Component.dispatchEventImpl(Unknown Source)
>       at java.awt.Container.dispatchEventImpl(Unknown Source)
>       at java.awt.Component.dispatchEvent(Unknown Source)
>       at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
>       at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
>       at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
>       at java.awt.Container.dispatchEventImpl(Unknown Source)
>       at java.awt.Window.dispatchEventImpl(Unknown Source)
>       at java.awt.Component.dispatchEvent(Unknown Source)
>       at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
>       at java.awt.EventQueue.access$500(Unknown Source)
>       at java.awt.EventQueue$3.run(Unknown Source)
>       at java.awt.EventQueue$3.run(Unknown Source)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at 
> java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
>  Source)
>       at 
> java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
>  Source)
>       at java.awt.EventQueue$4.run(Unknown Source)
>       at java.awt.EventQueue$4.run(Unknown Source)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at 
> java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
>  Source)
>       at java.awt.EventQueue.dispatchEvent(Unknown Source)
>       at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
>       at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
>       at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
>       at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
>       at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
>       at java.awt.EventDispatchThread.run(Unknown Source)
> 
> Kind regards
> 
> Netty 
> 
> Gesendet: Dienstag, 03. September 2019 um 03:55 Uhr
> Von: "John Caron" <jcaron1129@xxxxxxxxx>
> An: netty.jawn@xxxxxx
> Cc: "NetCDF-Java community" <netcdf-java@xxxxxxxxxxxxxxxx>
> Betreff: Re: [netcdf-java] Reading sequences from BUFR with NetcdfDataset
> 
> Hi netty:
>  
> Sorry for the crappy documents and confusion. Sequences are strange, they 
> dont behave like normal Arrays. As sort-of described here:
>  
> https://www.unidata.ucar.edu/software/netcdf-java/current/reference/Cookbook.html#readSequences
>  
> you cant call read(), you can only call getStructureIterator() and iterate, 
> because sequences dont have a fixed size. You have nested sequences: a BUFR 
> file consists of a variable number of BUFR messages (top level sequence), and 
> each BUFR message has a variable number of Long_time_period_or_displacement 
> fields (nested sequence). Heres some code that can get you started:
>  
> public void testReadSequenceEnhanced() throws IOException {try (NetcdfDataset 
> ncd = NetcdfDataset.openDataset("yourFilename")) {SequenceDS record = 
> (SequenceDS) ncd.findVariable("obs");
> try (StructureDataIterator iter = record.getStructureIterator()) {int 
> recordCount = 0;while (iter.hasNext()) {StructureData sdata = 
> iter.next();Assert.assertNotNull(sdata); recordCount++;
> ArraySequence nestedSequence = 
> sdata.getArraySequence("Long_time_period_or_displacement");try 
> (StructureDataIterator nestedIter = 
> nestedSequence.getStructureDataIterator()) {int nestedCount = 0;while 
> (nestedIter.hasNext()) {StructureData nestedData = 
> nestedIter.next();Assert.assertNotNull(nestedData);NCdumpW.printStructureData(new
>  PrintWriter(System.out), nestedData); nestedCount++; 
> }Assert.assertEquals(1685, nestedCount); } }Assert.assertEquals(1, 
> recordCount); } } }
>  
> Theres also this doc that may be useful:
> https://www.unidata.ucar.edu/software/netcdf-java/current/reference/StructureData.html[https://www.unidata.ucar.edu/software/netcdf-java/current/reference/StructureData.html]
> Good luck! 
> 
> On Mon, Sep 2, 2019 at 8:18 AM <netty.jawn@xxxxxx[mailto:netty.jawn@xxxxxx]> 
> wrote:Hi,
> 
> I have to read the attached BUFR-File with netcdf-java and it works a little 
> bit.
> 
> I opened my BUFR-File with ToolsUI to examine the content. Here was the first 
> surprise: The icon 
> https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/images/netcdf-java/tutorial/cdmdatasets/enhanceButton.jpg[https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/images/netcdf-java/tutorial/cdmdatasets/enhanceButton.jpg]
> shows "add Coordinates..." as tooltip. But it is actually the enable/disable 
> of the enhance-mode, as stated here
> https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html[https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html]
> The tooltip should be changed.
> 
> Reading variables was easy thanks to the enhance-mode:
> try (NetcdfDataset ncFile = NetcdfDataset.openDataset(filename);) {
>   VariableDS variable = (VariableDS) 
> ncFile.findVariable("obs.Software_identification_and_version_number");
>   System.out.println(variable.readScalarString());
>   }
> Float and all other types are correctly enhanced.
> 
> Naively I thought reading sequences is easy too. I have to read the sequence 
> "obs.Long_time_period_or_displacement" (as an example). Aunt Google showed me 
> the function "showNestedSequence" from 
> https://www.unidata.ucar.edu/software/netcdf-java/current/reference/Cookbook.html[https://www.unidata.ucar.edu/software/netcdf-java/current/reference/Cookbook.html]
>  (can't find it in the current documentation). This function shows me all 
> values, but not enhanced. Furthermore 2 nested loops were used. I would 
> prefer ONE loop (if possible).
> 
> After (I think) thousands of failed attempts, I came to this
> try (NetcdfDataset ncFile = NetcdfDataset.openDataset(filename);) {
>   VariableDS variable = (VariableDS) 
> ncFile.findVariable("obs.Long_time_period_or_displacement");
>   System.out.println("Type: " + variable.getDataType());
>   Array value = variable.read();
>   System.out.println("THIS LINE IS NOT PRINTED");
>   }
> The following exception is thrown:
> java.lang.ClassCastException: class ucar.ma2.ArrayObject$D1 cannot be cast to 
> class ucar.ma2.ArrayStructure (ucar.ma2.ArrayObject$D1 and 
> ucar.ma2.ArrayStructure are in unnamed module of loader 
> org.apache.catalina.loader.ParallelWebappClassLoader @ebbfb89)
>         at ucar.nc2.dataset.StructureDS.convert(StructureDS.java:268)
>         at ucar.nc2.dataset.SequenceDS.read(SequenceDS.java:84)
>         at de.sample.BUFRTest.showNestedSequenceNew(BUFRTest.java:88)
> Line 88 is the "variable.read();".
> I called your netcdf-lib from a servlet, therefore the "org.apache.catalina", 
> but this should not be the problem.
> 
> So it seems this sentence is not correct:
> https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html[https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html]
> "Note that NetcdfDataset is a subclass of NetcdfFile, and so can be used 
> wherever a NetcdfFile is used."
> 
> Someone suggested to have a look at NCdumpW, which I also did, but this 
> program does not enhance any values.
> 
> Since I don't want to open my file as NetcdfDataset and again as NetcdfFile, 
> I would try to open it as NetcdfFile and manually enhance the values. But no 
> idea how to do this. Any good solution for my problem is also appreciated.
> 
> Thanks
> 
> Kind regards
> 
> Netty_______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> 
> 
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx[mailto:netcdf-java@xxxxxxxxxxxxxxxx]
> For list information or to unsubscribe, visit: 
> https://www.unidata.ucar.edu/mailing_lists/[https://www.unidata.ucar.edu/mailing_lists/]_______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> 
> 
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit: 
> https://www.unidata.ucar.edu/mailing_lists/ 
>


  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: