Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
If im calculating correctly, this file would be about 1.8 TBytes. We do have a "large file format" that could theoretically handle this, but the nj22 does not yet support large file writing. That feature will be added soon, but I'm not sure exactly when. The C library does support this now. But I wonder if your OS/ disk system can handle that large of a file, and if you've thought about breakint it up into smaller files? If you wrote one file for each time step, you would still have 8Gb size files. Also with a dimension structure of (time,east,north,ens), you are optimizing to look at all the ensemble results at a single point, then looping over points, then over time. With files this large, you will have performance differences of 1000 or more depending on your read acccess pattern. Gabriella Turek wrote:
Hi John, I can provide the code (attached), it's a single class, but the input files are huge. The final size of the array is: (time, east,north,ens) = (241, 3668,3551,50), but I am trying to write it in chucks of arrays of size (241, 1,3551,50) at starting index (0, ieast,0,0). Time is always defined as unlimited in all our files. You will see that in the real code I am actually constructing 4 arrays of this shape, but the code always fails at the very first attempt of writing the first array when ens >= 14. If you think running the code would be really crucial I can put the the input files on my ftp server. Many thanx! Gaby John Caron wrote:Hi Gabriella: These kinds of errors are usually due to a misunderstanding of what shape the array has to be. Can you send a small test program that reproduces the problem? Gabriella Turek wrote:I have investigated this more and it turns out that I can run the code for nens =< 13. I wonder if I am running into some kind of array size limitation? I would not think so since in javadocs it says that the arrays can be as big as 2*10^9. But perhaps there's a cumulative effect? Thanx gaby------------------------------------------------------------------------ import java.io.IOException; import java.io.FileNotFoundException; import java.io.InputStream; import java.io.FileInputStream; import java.io.File; import java.io.FileWriter; import java.util.List; import java.util.ArrayList; import java.util.Properties; import java.awt.Point; import java.text.SimpleDateFormat; import java.util.Calendar; import java.util.GregorianCalendar; import java.util.Date; import ucar.ma2.InvalidRangeException; import ucar.nc2.NetcdfFile; import ucar.nc2.NetcdfFileWriteable; import ucar.nc2.Dimension; import ucar.nc2.Attribute; import ucar.nc2.Variable; import ucar.ma2.ArrayInt; import ucar.ma2.ArrayDouble; import ucar.ma2.DataType; /** * @author g.turek */ public class LandslideModel { private static String landslideSpatial; private static String topnetOutput; private static String topnetSpatial; private static String landslideAncillary; private static String outputFile; private static String logFile; private static String configFile = "LandslideModel.config"; private static final double DTOR = Math.PI/180.0; private static final double UW = 9810.0; private static final String OS = System.getProperty("os.name"); //Input data from ancillary file private static ArrayInt.D2 nrchm; // Model NRCH's private static int nrchm_fv; private static ArrayDouble.D2 awi; // Averageѕ wetness index for each point on the grid private static double awi_fv; private static double aw; //Input data from landslide spatial file private static ArrayDouble.D1 nzmge; // easting private static String eunit; private static String eaxis; private static ArrayDouble.D1 nzmgn; // northing private static String nunit; private static String naxis; private static ArrayDouble.D2 weti; // wetness index private static double weti_fv; private static double wetin; private static ArrayInt.D2 dtsp; // depth to shear plane (cm!) private static int dtsp_fv; private static int dt; private static ArrayInt.D2 duw; // dry soil unit weight private static int duw_fv; private static int du; private static ArrayInt.D2 slope_angle; // slope angle at each grid point (degrees!) private static int slope_angle_fv; private static int sa; private static ArrayInt.D2 sat_soil; // percent soil saturated water content private static int sat_soil_fv; private static int ss; private static ArrayInt.D3 cohesion; // cohesion private static int cohesion_fv; private static int co0,co1; private static ArrayInt.D3 frangle; // friction angle (degrees!) private static int frangle_fv; private static int fr0,fr1; //Input data from topnet files private static ArrayDouble.D1 time; // time private static String sname; private static String unit; private static String axis; private static String calendar; private static double time_fv; private static ArrayDouble.D3 zbarh2o; // depth of saturated zone (meters) private static double zbarh2o_fv; private static double zb; private static ArrayDouble.D3 soilh2o; // water in soil (meters) private static double soilh2o_fv; private static double sl; private static ArrayDouble.D1 fparam; // topmodel f parameter private static double fparam_fv; private static double fp; //Output data private static ArrayInt.D4 z; // depth private static ArrayInt.D4 smin; // min strength private static ArrayInt.D4 smax; // min strength private static ArrayInt.D4 strn; // strain private static ArrayInt.D4 fail_prob; //failure probability private static int fpb0; private static FileWriter fw; private static double d, w, strain, smn, smx; private static int nnzmge; private static int nnzmgn; private static int nv; private static int nrch; private static int nens; private static int nt; private static int tstart; private static int tend; private static double minSlope; private static NetcdfFileWriteable outFile; private static boolean debug = false; private static boolean interactive = false; public static void main(String[] args) { parseArgs(args); readConfiguration();if (interactive) {calculate_ls(); System.exit(0); }fw = openOutput(logFile); log("Beginning time step = " + tstart);log("Ending time step = " + tend); long start = System.currentTimeMillis(); readAncillaryFile(); readTopnetSpatial(); readTopnetOutput(); readLandslideSpatial(); writeOutputMetadata(); log("Start loop"); int rchindx; try { outFile.write("time", time); time = null; outFile.write("nzmge", nzmge);nzmge = null; outFile.write("nzmgn", nzmgn); nzmgn = null; for (int east = 0; east < nnzmge; east++) // loop over nzmge{ //z = new ArrayInt.D4(tend-tstart+1,1, nnzmgn, nens); smin = new ArrayInt.D4(tend-tstart+1,1, nnzmgn, nens); smax = new ArrayInt.D4(tend-tstart+1,1, nnzmgn, nens); strn = new ArrayInt.D4(tend-tstart+1,1, nnzmgn, nens); fail_prob = new ArrayInt.D4(tend-tstart+1,1, nnzmgn,nens); for (int north = 0; north < nnzmgn; north++) // loop over nzmgn { sa = slope_angle.get(east,north); aw = awi.get(east,north); du = duw.get(east,north); dt = dtsp.get(east,north); ss = sat_soil.get(east,north); wetin = weti.get(east,north); co0 = 0; //co0 = cohesion.get(east,north,0); co1 = cohesion.get(east,north,1); fr0 = frangle.get(east,north,0); fr1 = frangle.get(east,north,1); rchindx = nrchm.get(east,north); if (rchindx == nrchm_fv || aw == awi_fv || wetin == weti_fv || sa == slope_angle_fv || du == duw_fv || dt == dtsp_fv || ss == sat_soil_fv || co0 == cohesion_fv || co1 == cohesion_fv || fr0 == frangle_fv || fr1 == frangle_fv ) { // reach or a constant parameter is nodata - set all output to nodata int tcntr = 0; for (int t = tstart; t <= tend; t++) // loop over time { for (int ens = 0; ens < nens; ens++) // loop over ensambles { //z.set(tcntr,0,north,ens,-9999); smin.set(tcntr,0,north,ens,-9999); smax.set(tcntr,0,north,ens,-9999); strn.set(tcntr,0,north,ens,-9999); fail_prob.set(tcntr,0,north,ens,-9999); } tcntr++; } } else if (sa < minSlope) { // slope to low - fp = 0 int tcntr = 0; for (int t = tstart; t <= tend; t++) // loop over time { for (int ens = 0; ens < nens; ens++) // loop over ensambles { //z.set(tcntr,0,north,ens,-9999); smin.set(tcntr,0,north,ens,-9999); smax.set(tcntr,0,north,ens,-9999); strn.set(tcntr,0,north,ens,-9999); fail_prob.set(tcntr,0,north,ens,0); } tcntr++; } } else { // all spatial inputs have valid values - go thru time/ens loops fp = fparam.get(rchindx); int tcntr = 0; for (int t = tstart; t <= tend; t++) // loop over time { for (int ens = 0; ens < nens; ens++) // loop over ensambles { zb = zbarh2o.get(t,rchindx,ens); sl = soilh2o.get(t,rchindx,ens); if (zb == zbarh2o_fv || sl == soilh2o_fv){ // a TopNet output is nodata - cannot calculate depth to GW and weight! //z.set(tcntr,0,north,ens,-9999); smin.set(tcntr,0,north,ens,-9999); // no GW - so cannot calculate strength smax.set(tcntr,0,north,ens,-9999); strn.set(tcntr,0,north,ens,-9999); // no GW - so cannot calculate strain fail_prob.set(tcntr,0,north,ens,-9999);} else { // we have TopNet output - calculate! calculate_ls(); //z.set(tcntr,0,north,ens,(int)d); strn.set(tcntr,0,north,ens,(int)strain); smin.set(tcntr,0,north,ens,(int)smn); smax.set(tcntr,0,north,ens,(int)smx); fail_prob.set(tcntr,0,north,ens,fpb0); } } tcntr++; } } } //outFile.write("z", new int[] {0,east,0,0}, z); outFile.write("smin", new int[] {0,east,0,0}, smin); outFile.write("smax", new int[] {0,east,0,0}, smax); outFile.write("strain", new int[] {0,east,0,0}, strn); outFile.write("failure_probability", new int[] {0,east,0,0}, fail_prob); } //To disk outFile.flush(); long end = System.currentTimeMillis(); double elapsed = (double)(end - start)/1000.; log("Time elapsed : " + elapsed + " seconds"); fw.close(); System.exit(0); } catch (InvalidRangeException ire) { log("Invalid range exception: " + ire.getMessage()); } catch (IOException ioe) { log("Unable to write to output file: " + ioe.getMessage()); } finally { try { fw.close(); System.exit(1); } catch (IOException ioe) {} } } private static void calculate_ls() { if (interactive) { writeTTYLine("Enter slope angle [deg] (sa)"); sa = (int)readTTYLine(); writeTTYLine("Enter average wetness (aw)"); aw = readTTYLine(); writeTTYLine("Enter average wetness (wetin)"); wetin = readTTYLine(); writeTTYLine("Enter dry soil unit weight [N/m3] (du)"); du = (int)readTTYLine(); writeTTYLine("Enter depth to shear plane [cm](dt)"); dt = (int)readTTYLine(); writeTTYLine("Enter saturated soilwater content [%] (ss)"); ss = (int)readTTYLine(); writeTTYLine("Enter cohesion 0 [N/m**2](co0)"); co0 = (int)readTTYLine(); writeTTYLine("Enter cohesion 1 [N/m**2] (co1)"); co1 = (int)readTTYLine(); writeTTYLine("Enter friction angle 0 (fr0)"); fr0 = (int)readTTYLine(); writeTTYLine("Enter friction angle 1 (fr1)"); fr1 = (int)readTTYLine(); writeTTYLine("Enter fparam (fp)"); fp = readTTYLine(); writeTTYLine("Enter depth of saturated zone [m] (zb)"); zb = readTTYLine(); writeTTYLine("Enter water in soil [m] (sl)"); sl = readTTYLine(); }if (fr0< 0.0 || fr1<0) log("Negative friction angle!!! " + w); // depth to saturated zone [m]d = zb +(aw-wetin)/fp; writeTTYLine("Depth to saturated zone : " + d); // corrected depth to saturated zone d = Math.max(Math.min(dt/100.0,d),0.0); // limit to [0,dt] for later calc writeTTYLine("Corrected depth to saturated zone : " + d); // weight of soil + water above shear plain (must convert DTSP in m) w = du*(dt/100.0) + UW*Math.min(sl,d) + UW*ss/100.0*((dt/100.0)-d); writeTTYLine("Weight of soil + water above shear plain : " + w); if (w < 0.0) log("Negative weight!!! " + w); // shear strain in downslope direction strain = w*Math.sin(sa*DTOR); writeTTYLine("Shear strain in downslope direction : " + strain); if (strain < 0.0) log("Negative strain!!! " + strain); // shear strength in downslope direction smn = co0+Math.max((w-UW*((dt/100.0)-d)),0.0)* Math.cos(sa*DTOR)*Math.tan(fr0*DTOR); smx = co1+Math.max((w-UW*((dt/100.0)-d)),0.0)* Math.cos(sa*DTOR)*Math.tan(fr1*DTOR); if (smn < 0.0) log("Negative strength!!! " + smn); //if (smn<strain) log("Failure " + (strain - smn)/(smx - smn) ); writeTTYLine("Min shear strength in downslope direction : " + smn); writeTTYLine("Max shear strength in downslope direction : " + smx); // failure probability fpb0 = (int)(100.0*((strain - smn)/(smx - smn))); fpb0 = Math.max(Math.min(100,fpb0),0); // limit to [0,100] //if (fpb0>0) System.out.println("Calc Failure " + fpb0); //log("calculate_ls failure probability : " + fpb0); writeTTYLine("Failure probability : " + fpb0); } private static void writeOutputMetadata() { try { outFile = NetcdfFileWriteable.createNew(outputFile, false); outFile.addGlobalAttribute("title", "landslide model output file"); outFile.addGlobalAttribute("file_creation_time", getDate()); //define mode Dimension timedim = outFile.addUnlimitedDimension("nt"); Dimension edim = outFile.addDimension("nzmge", nnzmge); Dimension ndim = outFile.addDimension("nzmgn", nnzmgn); Dimension ensdim = outFile.addDimension("nens", nens); //addVariable(java.lang.String varName, DataType dataType, Dimension[] dims) outFile.addVariable("time", DataType.DOUBLE, new Dimension[] {timedim}); outFile.addVariableAttribute("time", "standard_name", sname); outFile.addVariableAttribute("time", "unit", unit); outFile.addVariableAttribute("time", "axis", axis); outFile.addVariableAttribute("time", "calendar", calendar); outFile.addVariableAttribute("time", "_FillValue", time_fv); outFile.addVariable("nzmge", DataType.DOUBLE, new Dimension[] {edim}); outFile.addVariableAttribute("nzmge", "unit", eunit); outFile.addVariableAttribute("nzmge", "axis", eaxis); outFile.addVariable("nzmgn", DataType.DOUBLE, new Dimension[] {ndim}); outFile.addVariableAttribute("nzmge", "unit", nunit); outFile.addVariableAttribute("nzmge", "axis", naxis); /* outFile.addVariable("z", DataType.INT, new Dimension[] {timedim, edim, ndim, ensdim}); outFile.addVariableAttribute("z", "long_name", "depth of saturated zone"); outFile.addVariableAttribute("z", "unit", "cm"); outFile.addVariableAttribute("z", "_FillValue", -9999); */ outFile.addVariable("smin", DataType.INT, new Dimension[] {timedim, edim, ndim, ensdim}); outFile.addVariableAttribute("smin", "long_name", "minimum strength"); outFile.addVariableAttribute("smin", "_FillValue", -9999); outFile.addVariable("smax", DataType.INT, new Dimension[] {timedim, edim, ndim, ensdim}); outFile.addVariableAttribute("smax", "long_name", "maximum strength"); outFile.addVariableAttribute("smax", "_FillValue", -9999); outFile.addVariable("strain", DataType.INT, new Dimension[] {timedim, edim, ndim, ensdim}); outFile.addVariableAttribute("strain", "long_name", "strain"); outFile.addVariableAttribute("strain", "_FillValue", -9999); outFile.addVariable("failure_probability", DataType.INT, new Dimension[] {timedim, edim, ndim, ensdim}); outFile.addVariableAttribute("failure_probability", "long_name", "failure probability"); outFile.addVariableAttribute("failure_probability", "unit", "percent"); outFile.addVariableAttribute("failure_probability", "valid_min", 0); outFile.addVariableAttribute("failure_probability", "valid_max", 100); outFile.addVariableAttribute("failure_probability", "_FillValue", -9999); //finish define mode outFile.create(); } catch (IOException ioe) { log("Unable to create output file: " + ioe.getMessage()); } } /*** Read topnet output file */ private static void readTopnetOutput(){log("****** Opening " + topnetOutput); try { NetcdfFile f = NetcdfFile.open(topnetOutput);//listDimensions(f); //listVariables(f); Dimension d = (Dimension)f.findDimension("nens"); nens = d.getLength(); log("nens = " + nens); nens = 13; d = (Dimension)f.findDimension("nrch"); nrch = d.getLength(); log("nrch = " + nrch); d = (Dimension)f.getUnlimitedDimension(); nt = d.getLength(); log("nt = " + nt);Variable v = (Variable)f.findVariable("time");time = (ArrayDouble.D1)v.read(new int[] {tstart}, new int[] {tend-tstart+1}); int[] dims = time.getShape();log("time rank = " + v.getRank() + ", shape = " + dims[0]); Attribute a = v.findAttribute("standard_name");sname= (String)a.getStringValue(); a = v.findAttribute("units"); unit = (String)a.getStringValue(); a = v.findAttribute("axis"); axis = (String)a.getStringValue(); a = v.findAttribute("calendar"); calendar = (String)a.getStringValue(); a = v.findAttribute("_FillValue");time_fv = ((Double)a.getNumericValue()).doubleValue();v = (Variable)f.findVariable("zbarh2o"); zbarh2o = (ArrayDouble.D3)v.read(); dims = zbarh2o.getShape();log("zbarh2o rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1] + "," + dims[2]); a = v.findAttribute("_FillValue"); zbarh2o_fv = ((Double)a.getNumericValue()).doubleValue();v = ((Variable)f.findVariable("soilh2o")); soilh2o = (ArrayDouble.D3)v.read(); dims = soilh2o.getShape();log("soilh2o rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1] + "," + dims[2]); a = v.findAttribute("_FillValue"); soilh2o_fv = ((Double)a.getNumericValue()).doubleValue(); f.close();} catch (InvalidRangeException ire) { log("Error reading time vector: " + ire.getMessage()); System.exit(1); } catch (IOException ioe) { log("Unable to open topnet output file: " + ioe.getMessage()); System.exit(1); } }/** * Read topnet spatial file */ private static void readTopnetSpatial(){log("****** Opening " + topnetSpatial); try { NetcdfFile f = NetcdfFile.open(topnetSpatial);//listDimensions(f); //listVariables(f); Variable v = ((Variable)f.findVariable("topmodf"));log("fparam rank = " + v.getRank()); fparam = (ArrayDouble.D1)v.read();Attribute a = v.findAttribute("_FillValue");fparam_fv = ((Double)a.getNumericValue()).doubleValue(); f.close();} catch (IOException ioe) { log("Unable to open topnet spatial file: " + ioe.getMessage()); System.exit(1); } } /*** Read ancillary file */ private static void readAncillaryFile(){log("****** Opening " + landslideAncillary); try { NetcdfFile f = NetcdfFile.open(landslideAncillary);//listDimensions(f); //listVariables(f); Dimension d = (Dimension)f.findDimension("nzmge"); nnzmge = d.getLength();log("nnzmge = " + nnzmge);d = (Dimension)f.findDimension("nzmgn"); nnzmgn = d.getLength();log("nnzmgn = " + nnzmgn);Variable v = ((Variable)f.findVariable("nrch_model")); nrchm = (ArrayInt.D2)v.read(); int[] dims = nrchm.getShape(); log("nrchm rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); Attribute a = v.findAttribute("_FillValue");nrchm_fv = ((Integer)a.getNumericValue()).intValue();v = ((Variable)f.findVariable("avg_wetness_index")); awi = (ArrayDouble.D2)v.read(); dims = awi.getShape(); log("awi rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); a = v.findAttribute("_FillValue");awi_fv = ((Double)a.getNumericValue()).doubleValue(); f.close();} catch (IOException ioe) { log("Unable to open ancillary file: " + ioe.getMessage()); System.exit(1); } } /*** Read landslide spatial file */ private static void readLandslideSpatial(){log("****** Opening " + landslideSpatial); try { NetcdfFile f = NetcdfFile.open(landslideSpatial);//listDimensions(f); //listVariables(f); Dimension d = (Dimension)f.findDimension("nzmge"); int nne = d.getLength(); d = (Dimension)f.findDimension("nzmgn"); int nnn = d.getLength(); if (nnzmge != nne || nnzmgn != nnn) { log("Mismatched grid sizes, something is wrong!"); System.exit(1); } d = (Dimension)f.findDimension("nv"); nv = d.getLength(); log("nv = " + nv); Variable v = ((Variable)f.findVariable("nzmge")); nzmge = (ArrayDouble.D1)v.read(); int[] dims = nzmge.getShape(); log("nzgme rank = " + v.getRank() + ", shape = " + dims[0]); Attribute a = v.findAttribute("unit"); eunit = (String)a.getStringValue(); a = v.findAttribute("axis"); eaxis = (String)a.getStringValue(); v = ((Variable)f.findVariable("nzmgn")); nzmgn = (ArrayDouble.D1)v.read(); dims = nzmgn.getShape(); log("nzgmn rank = " + v.getRank() + ", shape = " + dims[0]); a = v.findAttribute("unit"); nunit = (String)a.getStringValue(); a = v.findAttribute("axis"); naxis = (String)a.getStringValue(); v = ((Variable)f.findVariable("depth_to_shearplane")); dtsp = (ArrayInt.D2)v.read(); dims = dtsp.getShape(); log("dtsp rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); a = v.findAttribute("_FillValue");dtsp_fv = ((Integer)a.getNumericValue()).intValue();v = ((Variable)f.findVariable("wetness_index")); weti = (ArrayDouble.D2)v.read(); dims = weti.getShape(); log("weti rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); a = v.findAttribute("_FillValue");weti_fv = ((Double)a.getNumericValue()).doubleValue();v = ((Variable)f.findVariable("dry_soil_unit_weight")); duw = (ArrayInt.D2)v.read(); dims = duw.getShape(); log("duw rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); a = v.findAttribute("_FillValue");duw_fv = ((Integer)a.getNumericValue()).intValue();v = ((Variable)f.findVariable("slope_angle")); slope_angle = (ArrayInt.D2)v.read(); dims = slope_angle.getShape(); log("slope_angle rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); a = v.findAttribute("_FillValue");slope_angle_fv = ((Integer)a.getNumericValue()).intValue();v = ((Variable)f.findVariable("saturated_soilwater_content")); sat_soil = (ArrayInt.D2)v.read(); dims = sat_soil.getShape(); log("sat_soil rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1]); a = v.findAttribute("_FillValue");sat_soil_fv = ((Integer)a.getNumericValue()).intValue();v = ((Variable)f.findVariable("cohesion")); cohesion = (ArrayInt.D3)v.read(); dims = cohesion.getShape(); log("cohesion rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1] + "," + dims[2]); a = v.findAttribute("_FillValue");cohesion_fv = ((Integer)a.getNumericValue()).intValue();v = ((Variable)f.findVariable("friction_angle"));log("friction_angle rank = " + v.getRank()); frangle = (ArrayInt.D3)v.read();dims = frangle.getShape(); log("frangle rank = " + v.getRank() + ", shape = " + dims[0] + "," + dims[1] + "," + dims[2]); a = v.findAttribute("_FillValue");frangle_fv = ((Integer)a.getNumericValue()).intValue();f.close(); } catch (IOException ioe) { log("Unable to open landslide spatial file: " + ioe.getMessage()); System.exit(1); } } /** * Dimension information */ private static void listDimensions(NetcdfFile nf) { Dimension d; List l = nf.getDimensions(); int nd = l.size(); log("Found " + nd + " dimensions:"); for (int i = 0; i < nd; i++) { d = (Dimension)l.get(i); log(" " + d.getName() + " = " + d.getLength()); } } /** * Variables information */ private static void listVariables(NetcdfFile nf) { List l = nf.getVariables(); int nd = l.size(); log("Found " + nd + " variables:"); Dimension d; Variable v; String s; List l1; int nl1; for (int i = 0; i < nd; i++) { v = (Variable)l.get(i); l1 = v.getDimensions(); nl1 = l1.size(); s = " " + v.getName() + ": size = " + v.getSize() + ", " + nl1 + " dimensions = "; for (int j = 0; j < nl1; j++) { d = (Dimension)l1.get(j); s += v.getDimension(j).getLength() + " "; } s += ", data type = " + v.getDataType().toString() + ", description = " + v.getDescription(); log(s); } } /** * Read a configuration file into a Properties object from either the local disc or a jar file */ public static void readConfiguration() { Properties p = new Properties(); try { File f = new File(configFile);if (f.exists()) {//Try reading from local disc FileInputStream fs = new FileInputStream(f.getAbsolutePath()); p.load(fs); if (fs != null) fs.close(); } else { //Reading from a jar file if (ClassLoader.getSystemResource(f.getAbsolutePath()) != null) { InputStream is = ClassLoader.getSystemResource(f.getAbsolutePath()).openStream(); p.load(is); is.close(); } else { //File really does not exist throw new FileNotFoundException("Unable to find configuration file " + f.getAbsolutePath()); } } tstart = Integer.parseInt((String)p.get("begTimestep")); tend = Integer.parseInt((String)p.get("endTimestep")); minSlope = Double.parseDouble((String)p.get("minSlope")); landslideSpatial = (String)p.get("landslideSpatial"); topnetOutput = (String)p.get("topnetOutput") ; topnetSpatial = (String)p.get("topnetSpatial"); landslideAncillary = (String)p.get("landslideAncillary"); outputFile = (String)p.get("outputFile"); logFile = (String)p.get("logFile"); } catch (FileNotFoundException fnfe) { log("Unable to find file: " + fnfe.getMessage()); System.exit(1); } catch (IOException ioe) { log("Unable to open file: " + ioe.getMessage()); System.exit(1); } } private static void log(String s) { try { fw.write(s + "\n"); if (debug) System.out.println(s); } catch (IOException e) { System.out.println("Couldn't write to log file: " + e.getMessage()); } } private static final void writeTTYLine(String s) { if (interactive) System.out.println(s); } /** * Gets input from the command line * Note: must look for '\r' for it to work properly in DOS * @return tty input */ private static final double readTTYLine() { StringBuffer buf = new StringBuffer(80); int c = 0; try { while ((c = System.in.read()) != -1) { char ch = (char)c; if (ch == '\r') { // This is a temporary fix. XXXX We really need lookahead. if (OS.equalsIgnoreCase ("macos")) break; else { System.in.read(); break; } } else if (ch == '\n') break; // Unix flavors. buf.append(ch); } } catch (IOException e) { System.err.println(e.getMessage()); } return Double.parseDouble(buf.toString()); } /** * Open output file * @param fileName the output file */ private static FileWriter openOutput(String fileName) { FileWriter file = null; try { file = new FileWriter(fileName); } catch (IOException e) { System.out.println("Couldn't write log file: " + e.getMessage()); System.exit(1); } return file; } /*** Parse arguments */private static void parseArgs(String[] args) { int numArgs = args.length; int index = 0; while (index < numArgs) { if (args[index].equals("-h")) { usage(); } else if (args[index].equals("-i")) { interactive = true; index++; } else if (args[index].equals("-d")) { debug = true; index++; } else if (args[index].equals("-c")) { configFile = args[++index]; index++; } else { usage(); } } } /** * Command line switches */ private static void usage() { System.out.println("**** LandslideModel optional invocation flags ****"); System.out.println(" -c <configfile> : config file"); System.out.println(" -h : this listing"); System.out.println(" -i : interactive mode"); System.out.println(" -d : turn debug on listing"); System.exit(0); } /** * Get current date in dd/mm/yyyy hh:mm:ss.SSS format * @return date string */ private static String getDate() { SimpleDateFormat format = new SimpleDateFormat("dd'/'MM'/'yyyy HH:mm:ss.SSS"); format.setLenient(false); Calendar cal = new GregorianCalendar(); Date d = cal.getTime(); return format.format(d); } }
netcdf-java archives: