Re: [netcdf-java] Questions

Sebastian, please keep replies on-list so all can benefit.

What are the values in your time variable? Are they all zero? I like to use ncdump to see raw values to check their sanity.

rlat and rlon are for rotated coordinates, I think. Eastings and northings should be x and y. Do your tools support multidimensional coordinate variables? If not, you will likely have to interpolate your data onto your new coordinate grid. What are your source and target coordinate reference system? Can you get your data in another CRS? Can you use an external tool like gdalwarp?

Kind regards,
Ben.

On 20/02/18 22:41, Sebastian Raubach wrote:
Hi Ben,

Yes, I realized my mistake about an hour after sending the email. Sorry about 
that. And thank you again for the great feedback so far. It's really helping me 
understand the format.
I managed to create an output file now that has some data in it.

I've been using Panoply before to look at the files and I can see the values in 
the data matrix for the new file as well. However, Panoply won't plot the data 
(plot is all grey).
Because of some requirements that I cannot do anything about, I've been asked to use eastings and 
northings as the geographic dimensions. I've kept their names as "rlat0" and 
"rlon0", because I wasn't sure if I can simply rename them. I'm wondering if this is 
causing Panoply to not be able to plot the data. I can't find anything about using 
eastings/northings for  longitude/latitude dimensions in the documentation 
(https://www.unidata.ucar.edu/software/netcdf/docs/netcdf_data_set_components.html) so I'm not sure 
if it's technically supported.
However, the plotting in Panoply isn't really an issue. What's more concerning is that if I look at 
the temperature variable, I can see the 2d array for eastings northings with the actual value as 
the cell for each point in time (out of 53970), but while Panoply for the original file always 
tells me the correct date for each point in time, it always shows "1970-01-01 00:00 - 
1970-01-01 00:00". I assume the date is taken from the "time" variable:

     float time(time=53970);
       :bounds = "time_bnd0";
       :units = "days since 1970-01-01 00:00:00";
       :standard_name = "time";
       :calendar = "360_day";
       :axis = "T";
       :_ChunkSizes = 2048; // int
       :_CoordinateAxisType = "Time";

But why isn't it able to show the correct date anymore when browsing through 
the dates. I mean, I don't need it to work in Panoply specifically, but this 
makes me wonder if the file has actually been created correctly.

Any ideas and suggestions would greatly be appreciated!

Many thanks,
Sebastian

-----Original Message-----
From: Ben Caradoc-Davies [mailto:ben@xxxxxxxxxxxx]
Sent: 19 February 2018 20:31
To: Sebastian Raubach <Sebastian.Raubach@xxxxxxxxxxxx>
Cc: netcdf-java@xxxxxxxxxxxxxxxx
Subject: Re: [netcdf-java] Questions

Sebastian,

how do you obtain the Variable in your call to NetcdfFileWriter#write(Variable, 
int[], Array)? The error suggests that you are using the NetCDF-3 Variable 
obtained from the original file, not the newly-created NetCDF-4 from the target 
file, obtained with findVariable or by saving the result of addVariable.

Kind regards,
Ben.

On 19/02/18 23:31, Sebastian Raubach wrote:
Hi Ben,

Thank you, that helped. I can now create a new file that has all the same 
variables and dimensions. I had to move to NetCDF4 using the C library, but 
that appeared to be working fine until I actually tried adding some data to the 
new file using NetcdfFileWriter#write(Variable, int[], Array).
I now get the following exception:

Exception in thread "main" java.lang.ClassCastException: 
ucar.nc2.iosp.netcdf3.N3header$Vinfo cannot be cast to ucar.nc2.jni.netcdf.Nc4Iosp$Vinfo
        at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2799)
        at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:958)

Which makes me think that there is a version 3 <-> 4 conflict/issue here, but I 
can't seem to figure out what's wrong. The file I'm reading from is a v3 file and I'm 
trying to write to v4.
I tried googling for the error, but couldn't find much.

Do you have any idea what I need to do here?

Many thanks,
Sebastian

-----Original Message-----
From: Ben Caradoc-Davies [mailto:ben@xxxxxxxxxxxx]
Sent: 15 February 2018 20:06
To: Sebastian Raubach <Sebastian.Raubach@xxxxxxxxxxxx>;
netcdf-java@xxxxxxxxxxxxxxxx
Subject: Re: [netcdf-java] Questions

Sebastian,

please try using NetcdfFileWriter.addDimension to create each dimension and add 
it to the target file, after testing that it does not already exist. You can 
use NetcdfFileWriter.findDimension to locate dimensions in the target file. In 
your code below, your new Dimension instances have not been yet been added to 
the target file so cannot be used to define variables.

Kind regards,
Ben.

On 16/02/18 03:02, Sebastian Raubach wrote:
Hello,

I've been tasked with running some data manipulation across a netcdf file. It's 
supposed to work off an existing file, but create a new one that has the same 
structure (although dimensions will have different lengths) and then change 
some of the cell values by applying a formula.

I'm currently trying to replicate the structure of the existing file in the new 
file. I can copy the global attributes just fine and wanted to re-create the 
variables and dimensions next. To do so, I wrote this code:

                   @Override
                   public void process(NetcdfFile original, NetcdfFileWriter 
target, Config config) throws IOException
                   {
                                   for(Attribute att :
original.getGlobalAttributes())
target.addGroupAttribute(att.getGroup(), att);

                                   for(Variable var : original.getVariables())
                                   {
                                                   Group group = 
target.addGroup(null, var.getGroup().getShortName());
                                                   List<Dimension>
dimensions = new ArrayList<>();

                                                   for(Dimension dim : 
var.getDimensions())
                                                   {
                                                                   
dimensions.add(new Dimension(dim.getShortName(), dim.getLength(), 
dim.isShared(), dim.isUnlimited(), dim.isVariableLength()));
//                                                            
dimensions.add(new Dimension(dim.getShortName(), dim));
                                                   }

                                                   target.addVariable(group, 
var.getShortName(), var.getDataType(), dimensions);
                                   }
                   }

I ran into several issues. The code as it stands throws this exception:

Exception in thread "main" java.lang.IllegalStateException: unknown Dimension 
== time = UNLIMITED;   // (53970 currently)
                   at 
ucar.nc2.iosp.netcdf3.N3header.findDimensionIndex(N3header.java:970)
                   at 
ucar.nc2.iosp.netcdf3.N3header.writeVars(N3header.java:910)
                   at 
ucar.nc2.iosp.netcdf3.N3header.writeHeader(N3header.java:666)
                   at ucar.nc2.iosp.netcdf3.N3header.create(N3header.java:602)
                   at ucar.nc2.iosp.netcdf3.N3iosp.create(N3iosp.java:683)
                   at 
ucar.nc2.NetcdfFileWriter.create(NetcdfFileWriter.java:804)
                   at jhi.netcdf.Main.main(Main.java:58)

I can't really find any information online about how to best approach this 
scenario or what exactly the exception means. If anyone can give me any clues 
or has experience with this kind of thing, that'd really be appreciated.
Maybe there is a more convenient way to re-create an existing file structure 
(not the actual data) that I'm not aware of.

Regards,
Sebastian Raubach


The James Hutton Institute is a Scottish charitable company limited by 
guarantee.
Registered in Scotland No. SC374831
Registered Office: The James Hutton Institute, Invergowrie Dundee DD2 5DA.
Charity No. SC041796



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they do
not want to be made public.


netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/


--
Ben Caradoc-Davies <ben@xxxxxxxxxxxx>
Director
Transient Software Limited <https://transient.nz/> New Zealand


The James Hutton Institute is a Scottish charitable company limited by 
guarantee.
Registered in Scotland No. SC374831
Registered Office: The James Hutton Institute, Invergowrie Dundee DD2 5DA.
Charity No. SC041796


--
Ben Caradoc-Davies <ben@xxxxxxxxxxxx>
Director
Transient Software Limited <https://transient.nz/> New Zealand


The James Hutton Institute is a Scottish charitable company limited by 
guarantee.
Registered in Scotland No. SC374831
Registered Office: The James Hutton Institute, Invergowrie Dundee DD2 5DA.
Charity No. SC041796




  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: