ucar.nc2.iosp.netcdf3
Class N3iosp

java.lang.Object
  ucar.nc2.iosp.AbstractIOServiceProvider
      ucar.nc2.iosp.netcdf3.N3iosp

All Implemented Interfaces:

IOServiceProvider, IOServiceProviderWriter

Direct Known Subclasses:

N3raf

public abstract class N3iosp
extends AbstractIOServiceProvider
implements IOServiceProviderWriter

IOServiceProvider implementation abstract base class to read/write "version 3" netcdf files. AKA "file format version 1" files.

Author:

caron

See Also:

concrete class

Field Summary

protected boolean	debug
protected boolean	debugRecord
protected boolean	debugSize
protected boolean	debugSPIO
protected boolean	debugSync
protected java.util.HashMap	dimHash
protected long	fileUsed
protected boolean	fill
static java.lang.String	FillValue
protected N3header	headerParser
protected long	lastModified
static int	MAX_NUMRECS The maximum number of records is 2^32-1.
static long	MAX_VARSIZE Each fixed-size variable and the data for one record's worth of a single record variable are limited to a little less than 4 GiB.
static byte	NC_FILL_BYTE
static char	NC_FILL_CHAR
static double	NC_FILL_DOUBLE
static float	NC_FILL_FLOAT
static int	NC_FILL_INT
static long	NC_FILL_LONG
static short	NC_FILL_SHORT
protected NetcdfFile	ncfile
protected int	numrecs
protected boolean	readonly
protected long	recsize
protected long	recStart
protected boolean	showHeaderBytes
protected boolean	useRecordStructure

Fields inherited from class ucar.nc2.iosp.AbstractIOServiceProvider

raf

Constructor Summary

N3iosp()

Method Summary

protected abstract void	_create(RandomAccessFile raf)
protected abstract void	_open(RandomAccessFile raf)
void	close() Close the file.
protected static char[]	convertByteToCharUTF(byte[] byteArray)
protected static byte[]	convertCharToByteUTF(char[] from)
void	create(java.lang.String filename, NetcdfFile ncfile, int extra, long preallocateSize, boolean largeFile) Create new file, populate it from the objects in ncfile.
static java.lang.String	createValidNetcdf3ObjectName(java.lang.String name) Convert a name to a legal netcdf name.
protected void	fillNonRecordVariables()
protected void	fillRecordVariables(int recStart, int recEnd)
void	flush() Flush all data buffers to disk.
java.lang.String	getDetailInfo() Show debug / underlying implementation details
java.lang.String	getFileTypeDescription() Get a human-readable description for this file type.
java.lang.String	getFileTypeId() Get a unique id for this file type.
static java.util.regex.Pattern	getValidNetcdf3ObjectNamePattern() Valid Netcdf Object name as a regular expression.
boolean	isValidFile(RandomAccessFile raf) Check if this is a valid file for this IOServiceProvider.
static boolean	isValidNetcdf3ObjectName(java.lang.String name) Determine if the given name can be used for a Dimension, Attribute, or Variable name.
static java.lang.String	makeValidNetcdfObjectName(java.lang.String name) Convert a name to a legal netcdf-3 name.
static java.lang.String	makeValidNetcdfObjectNameOld(java.lang.String name)
void	open(RandomAccessFile raf, NetcdfFile ncfile, CancelTask cancelTask) Open existing file, and populate ncfile with it.
protected abstract java.lang.Object	readData(Layout index, DataType dataType) Read data subset from file for a variable, create primitive array.
protected abstract long	readData(Layout index, DataType dataType, java.nio.channels.WritableByteChannel out)
Array	readData(Variable v2, Section section) Read data from a top level Variable and return a memory resident Array.
Array	readNestedData(Variable v2, Section section)
long	readToByteChannel(Variable v2, Section section, java.nio.channels.WritableByteChannel channel) Read data from a top level Variable and send data to a WritableByteChannel.
boolean	rewriteHeader(boolean largeFile)
java.lang.Object	sendIospMessage(java.lang.Object message) A way to communicate arbitrary information to an iosp.
void	setFill(boolean fill) Set the fill flag.
protected void	setNumrecs(int n)
static void	setProperty(java.lang.String name, java.lang.String value) Set a static property.
boolean	sync() Update the metadata in the NetcdfFile if the underlying dataset has changed.
boolean	syncExtend() Extend the NetcdfFile if the underlying dataset has changed in a way that is compatible with the current metadata.
java.lang.String	toStringDebug(java.lang.Object o) Debug info for this object.
void	updateAttribute(Variable v2, Attribute att) Update the value of an existing attribute.
protected abstract void	writeData(Array aa, Layout index, DataType dataType) Write data subset to file for a variable, create primitive array.
void	writeData(Variable v2, Section section, Array values) Write data into a variable.

Methods inherited from class ucar.nc2.iosp.AbstractIOServiceProvider

copyToByteChannel, createValidNetcdfObjectName, getFileTypeVersion, getStructureIterator, readSection

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ucar.nc2.iosp.IOServiceProvider

getFileTypeVersion, readSection

Field Detail

NC_FILL_BYTE

public static final byte NC_FILL_BYTE

See Also:

Constant Field Values

NC_FILL_CHAR

public static final char NC_FILL_CHAR

See Also:

Constant Field Values

NC_FILL_SHORT

public static final short NC_FILL_SHORT

See Also:

Constant Field Values

NC_FILL_INT

public static final int NC_FILL_INT

See Also:

Constant Field Values

NC_FILL_LONG

public static final long NC_FILL_LONG

See Also:

Constant Field Values

NC_FILL_FLOAT

public static final float NC_FILL_FLOAT

See Also:

Constant Field Values

NC_FILL_DOUBLE

public static final double NC_FILL_DOUBLE

See Also:

Constant Field Values

FillValue

public static final java.lang.String FillValue

See Also:

Constant Field Values

MAX_VARSIZE

public static final long MAX_VARSIZE

Each fixed-size variable and the data for one record's worth of a single record variable are limited to a little less than 4 GiB.

See Also:

Constant Field Values

MAX_NUMRECS

public static final int MAX_NUMRECS

The maximum number of records is 2^32-1.

See Also:

Constant Field Values

ncfile

protected NetcdfFile ncfile

readonly

protected boolean readonly

headerParser

protected N3header headerParser

numrecs

protected int numrecs

recsize

protected long recsize

lastModified

protected long lastModified

fileUsed

protected long fileUsed

recStart

protected long recStart

debug

protected boolean debug

debugSize

protected boolean debugSize

debugSPIO

protected boolean debugSPIO

debugRecord

protected boolean debugRecord

debugSync

protected boolean debugSync

showHeaderBytes

protected boolean showHeaderBytes

useRecordStructure

protected boolean useRecordStructure

fill

protected boolean fill

dimHash

protected java.util.HashMap dimHash

Constructor Detail

N3iosp

public N3iosp()

Method Detail

setProperty

public static void setProperty(java.lang.String name,
                               java.lang.String value)

Set a static property. Supported static properties:

• syncExtendOnly = "true" : assume all file changes are syncExtend only.

Parameters:

name - property name

value - property value

makeValidNetcdfObjectName

public static java.lang.String makeValidNetcdfObjectName(java.lang.String name)

Convert a name to a legal netcdf-3 name. ([a-zA-Z0-9_]|{UTF8})([^\x00-\x1F\x7F/]|{UTF8})*

Parameters:

name - convert this name

Returns:

converted name

makeValidNetcdfObjectNameOld

public static java.lang.String makeValidNetcdfObjectNameOld(java.lang.String name)

isValidNetcdf3ObjectName

public static boolean isValidNetcdf3ObjectName(java.lang.String name)

Determine if the given name can be used for a Dimension, Attribute, or Variable name.

Parameters:

name - test this.

Returns:

true if valid name.

getValidNetcdf3ObjectNamePattern

public static java.util.regex.Pattern getValidNetcdf3ObjectNamePattern()

Valid Netcdf Object name as a regular expression.

Returns:

regular expression pattern describing valid Netcdf Object names.

createValidNetcdf3ObjectName

public static java.lang.String createValidNetcdf3ObjectName(java.lang.String name)

Convert a name to a legal netcdf name. From the user manual: "The names of dimensions, variables and attributes consist of arbitrary sequences of alphanumeric characters (as well as underscore '_' and hyphen '-'), beginning with a letter or underscore. (However names commencing with underscore are reserved for system use.) Case is significant in netCDF names."

Algorithm:

1. leading character: if alpha or underscore, ok; if digit, prepend "N"; otherwise discard
2. other characters: if space, change to underscore; other delete.

Parameters:

name - convert this name

Returns:

converted name

isValidFile

public boolean isValidFile(RandomAccessFile raf)
                    throws java.io.IOException

Description copied from interface: IOServiceProvider

Check if this is a valid file for this IOServiceProvider. You must make this method thread safe, ie dont keep any state.

Specified by:

isValidFile in interface IOServiceProvider

Parameters:

raf - RandomAccessFile

Returns:

true if valid.

Throws:

java.io.IOException - if read error

getDetailInfo

public java.lang.String getDetailInfo()

Description copied from interface: IOServiceProvider

Show debug / underlying implementation details

Specified by:

getDetailInfo in interface IOServiceProvider

Overrides:

getDetailInfo in class AbstractIOServiceProvider

Returns:

debug info

open

public void open(RandomAccessFile raf,
                 NetcdfFile ncfile,
                 CancelTask cancelTask)
          throws java.io.IOException

Description copied from interface: IOServiceProvider

Open existing file, and populate ncfile with it. This method is only called by the NetcdfFile constructor on itself. The provided NetcdfFile object will be empty except for the location String and the IOServiceProvider associated with this NetcdfFile object.

Specified by:

open in interface IOServiceProvider

Parameters:

raf - the file to work on, it has already passed the isValidFile() test.

ncfile - add objects to this empty NetcdfFile

cancelTask - used to monitor user cancellation; may be null.

Throws:

java.io.IOException - if read error

setFill

public void setFill(boolean fill)

Description copied from interface: IOServiceProviderWriter

Set the fill flag. For new files, set in the create() method. This method is to set fill for existing files that you want to write. If true, the data is first written with fill values. Leave false if you expect to write all data values, set to true if you want to be sure that unwritten data values have the fill value in it.

Specified by:

setFill in interface IOServiceProviderWriter

Parameters:

fill - set fill mode true or false

readData

public Array readData(Variable v2,
                      Section section)
               throws java.io.IOException,
                      InvalidRangeException

Description copied from interface: IOServiceProvider

Read data from a top level Variable and return a memory resident Array. This Array has the same element type as the Variable, and the requested shape.

Specified by:

readData in interface IOServiceProvider

Parameters:

v2 - a top-level Variable

section - the section of data to read. There must be a Range for each Dimension in the variable, in order. Note: no nulls allowed. IOSP may not modify.

Returns:

the requested data in a memory-resident Array

Throws:

java.io.IOException - if read error

InvalidRangeException - if invalid section

See Also:

Range

readNestedData

public Array readNestedData(Variable v2,
                            Section section)
                     throws java.io.IOException,
                            InvalidRangeException

Throws:

java.io.IOException

InvalidRangeException

readToByteChannel

public long readToByteChannel(Variable v2,
                              Section section,
                              java.nio.channels.WritableByteChannel channel)
                       throws java.io.IOException,
                              InvalidRangeException

Description copied from interface: IOServiceProvider

Read data from a top level Variable and send data to a WritableByteChannel. Must be in big-endian order.

Specified by:

readToByteChannel in interface IOServiceProvider

Overrides:

readToByteChannel in class AbstractIOServiceProvider

Parameters:

v2 - a top-level Variable

section - the section of data to read. There must be a Range for each Dimension in the variable, in order. Note: no nulls allowed. IOSP may not modify.

channel - write data to this WritableByteChannel

Returns:

the number of bytes written to the channel

Throws:

java.io.IOException - if read error

InvalidRangeException - if invalid section

convertByteToCharUTF

protected static char[] convertByteToCharUTF(byte[] byteArray)

convertCharToByteUTF

protected static byte[] convertCharToByteUTF(char[] from)

create

public void create(java.lang.String filename,
                   NetcdfFile ncfile,
                   int extra,
                   long preallocateSize,
                   boolean largeFile)
            throws java.io.IOException

Description copied from interface: IOServiceProviderWriter

Create new file, populate it from the objects in ncfile.

Specified by:

create in interface IOServiceProviderWriter

Parameters:

filename - name of file to create.

ncfile - get dimensions, attributes, and variables from here.

extra - if > 0, pad header with extra bytes

preallocateSize - if > 0, set length of file to this upon creation - this (usually) pre-allocates contiguous storage.

largeFile - if want large file format

Throws:

java.io.IOException - if I/O error

rewriteHeader

public boolean rewriteHeader(boolean largeFile)
                      throws java.io.IOException

Specified by:

rewriteHeader in interface IOServiceProviderWriter

Throws:

java.io.IOException

writeData

public void writeData(Variable v2,
                      Section section,
                      Array values)
               throws java.io.IOException,
                      InvalidRangeException

Description copied from interface: IOServiceProviderWriter

Write data into a variable.

Specified by:

writeData in interface IOServiceProviderWriter

Parameters:

v2 - variable to write; must already exist.

section - the section of data to write. There must be a Range for each Dimension in the variable, in order. The shape must match the shape of values. The origin and stride indicate where the data is placed into the stored Variable array.

values - data to write. The shape must match section.getShape().

Throws:

java.io.IOException - if I/O error

InvalidRangeException - if invalid section

setNumrecs

protected void setNumrecs(int n)
                   throws java.io.IOException,
                          InvalidRangeException

Throws:

java.io.IOException

InvalidRangeException

updateAttribute

public void updateAttribute(Variable v2,
                            Attribute att)
                     throws java.io.IOException

Update the value of an existing attribute. Attribute is found by name, which must match exactly. You cannot make an attribute longer, or change the number of values. For strings: truncate if longer, zero fill if shorter. Strings are padded to 4 byte boundaries, ok to use padding if it exists. For numerics: must have same number of values.

Specified by:

updateAttribute in interface IOServiceProviderWriter

Parameters:

v2 - variable, or null for fglobal attribute

att - replace with this value

Throws:

java.io.IOException

fillNonRecordVariables

protected void fillNonRecordVariables()
                               throws java.io.IOException

Throws:

java.io.IOException

fillRecordVariables

protected void fillRecordVariables(int recStart,
                                   int recEnd)
                            throws java.io.IOException,
                                   InvalidRangeException

Throws:

java.io.IOException

InvalidRangeException

syncExtend

public boolean syncExtend()
                   throws java.io.IOException

Description copied from interface: IOServiceProvider

Extend the NetcdfFile if the underlying dataset has changed in a way that is compatible with the current metadata. For example, if the unlimited dimension has grown.

Specified by:

syncExtend in interface IOServiceProvider

Overrides:

syncExtend in class AbstractIOServiceProvider

Returns:

true if the NetcdfFile was extended.

Throws:

java.io.IOException - if a read error occured when accessing the underlying dataset.

sync

public boolean sync()
             throws java.io.IOException

Description copied from interface: IOServiceProvider

Update the metadata in the NetcdfFile if the underlying dataset has changed.

Specified by:

sync in interface IOServiceProvider

Overrides:

sync in class AbstractIOServiceProvider

Returns:

true if the NetcdfFile was changed.

Throws:

java.io.IOException - if a read error occured when accessing the underlying dataset.

flush

public void flush()
           throws java.io.IOException

Description copied from interface: IOServiceProviderWriter

Flush all data buffers to disk.

Specified by:

flush in interface IOServiceProviderWriter

Throws:

java.io.IOException - if I/O error

close

public void close()
           throws java.io.IOException

Description copied from interface: IOServiceProvider

Close the file. It is the IOServiceProvider's job to close the file (even though it didnt open it), and to free any other resources it has used.

Specified by:

close in interface IOServiceProvider

Overrides:

close in class AbstractIOServiceProvider

Throws:

java.io.IOException - if read error

toStringDebug

public java.lang.String toStringDebug(java.lang.Object o)

Debug info for this object.

Specified by:

toStringDebug in interface IOServiceProvider

Overrides:

toStringDebug in class AbstractIOServiceProvider

Parameters:

o - which object

Returns:

debug info for this object

sendIospMessage

public java.lang.Object sendIospMessage(java.lang.Object message)

Description copied from interface: IOServiceProvider

A way to communicate arbitrary information to an iosp.

Specified by:

sendIospMessage in interface IOServiceProvider

Overrides:

sendIospMessage in class AbstractIOServiceProvider

Parameters:

message - opaque message.

Returns:

opaque return, may be null.

getFileTypeId

public java.lang.String getFileTypeId()

Description copied from interface: IOServiceProvider

Get a unique id for this file type.

Specified by:

getFileTypeId in interface IOServiceProvider

Returns:

registered id of the file type

See Also:

"http://www.unidata.ucar.edu/software/netcdf-java/formats/FileTypes.html"

getFileTypeDescription

public java.lang.String getFileTypeDescription()

Description copied from interface: IOServiceProvider

Get a human-readable description for this file type.

Specified by:

getFileTypeDescription in interface IOServiceProvider

Returns:

description of the file type

See Also:

"http://www.unidata.ucar.edu/software/netcdf-java/formats/FileTypes.html"

readData

protected abstract java.lang.Object readData(Layout index,
                                             DataType dataType)
                                      throws java.io.IOException

Read data subset from file for a variable, create primitive array.

Parameters:

index - handles skipping around in the file.

dataType - dataType of the variable

Returns:

primitive array with data read in

Throws:

java.io.IOException - on error

readData

protected abstract long readData(Layout index,
                                 DataType dataType,
                                 java.nio.channels.WritableByteChannel out)
                          throws java.io.IOException

Throws:

java.io.IOException

writeData

protected abstract void writeData(Array aa,
                                  Layout index,
                                  DataType dataType)
                           throws java.io.IOException

Write data subset to file for a variable, create primitive array.

Parameters:

aa - write data in this Array.

index - handles skipping around in the file.

dataType - dataType of the variable

Throws:

java.io.IOException - on error

_open

protected abstract void _open(RandomAccessFile raf)
                       throws java.io.IOException

Throws:

java.io.IOException

_create

protected abstract void _create(RandomAccessFile raf)
                         throws java.io.IOException

Throws:

java.io.IOException