Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Hi Ethan,
> Well, from what I've read, VisAD is very useful in displaying data in
> different formats such as netCDF. But is it possible for VisAD to visualize
> realtime data?
>
> Let me elaborate.
> I'm dealing with financial data here, and i've got historical data stored in
> a .nc file. I'll need to visualize that, as well as new data records that
> I'm collecting every minute.
>
> A straight forward method is simply to write the new data to the .nc file
> and load it again. But is there an alternative where I can update the data
> in memory and write out to disk at hourly intervals instead? (the historical
> data can be huge and new data may arrive seconds apart).
Yes, this should not be difficult. In fact, there are a
number of different ways to do this, depending on what you
want. For example, the visad.aune.ShallowFluid program
(distributed with VisAD, but only runs on Sparc Solaris)
creates displays of the state of a 2-D atmosphere that
evolves over time. However, it only keeps the latest
state, rather than a time history.
So I'll take a guess at the way you might want to manage
and display your data. I'll assume that your data are a
time series of records, where each record is a collection
("Tuple" in VisAD language) of numerical values. So lets
say the MathType of the time series is:
(RealType.Time -> (value1, value2, ..., valueN))
where value1, ..., valueN are RealTypes. They might be
share prices, exchange rates, or whatever magic you guys
are working with.
Then you read a .nc file with this data by:
Plain plain = new Plain();
FlatField data = plain.open("filename.nc");
Now you want to extend the set of times in this data to
make room for your real-time data. I'll assume that your
data are sampled on equal time intervals so getDomainSet()
will return a Linear1DSet (the alternative would be a
Gridded1DDoubleSet for unevenly spaced time samples). So
here's how to extend the time sampling:
// construct a new Linear1DSet of time samples that
// extends current set up to "new_last" time in future
Linear1DSet set = data.getDomainSet();
double first = set.getFirst(); // first time
double step = set.getStep(); // time step between samples
double last = set.getLast(); // this could be the last time
MathType set_type = set.getType();
double new_last = ... // new last time
int new_length = 1 + (int) ((new_last - first) / step);
Linear1DSet new_set
new Linear1DSet(set_type, first, new_last, new_length);
// construct new_data that combines time series from .nc
// file with space for real-time records
//
// copy any range Units - this should not be this complex
Unit[][] long_units = data.getRangeUnits();
Unit[] units = new Unit[long_units.length];
for (int i=0; i<long_units.length; i++) units[i] = long_units[i][0];
// get MathType of data
FunctionType time_series_type = (FunctionType) data.getType();
// construct new FlatField
FlatField new_data
new FlatField(time_series_type, new_set,
null, null, // no RangeCoordinateSystems from Plain
field.getRangeSets(), units);
// now copy "range" dependent values and fill with missing
float[][] values = data.getFloats(false); // don't copy
int dimension = values.length;
int length = values[0].length;
float[][] new_values = new float[dimension][new_length];
for (int d=0; d<dimension; d++) {
System.arraycopy(values[d], 0, new_values[d], 0, length);
for (int i=length; i<new_length; i++) {
new_values[d][i] = Float.NaN; // mark "future" values as missing
}
}
new_data.setSamples(new_values, false); // don't copy
Now you can construct a display and link it to new_data, something
like (I'll assume that you'll use Java3D graphics):
DisplayImplJ3D display = new DisplayImplJ3D("financial data");
display.addMap(new ScalarMap(RealType.Time, Display.XAxis));
// construct maps of various financial values to YAxis, ZAxis,
// Red, etc
display.addMap(new ScalarMap(value1, ...));
// construct a DataReference to link new_data to display
DataReferenceImpl ref = new DataReference("time series");
ref.setData(new_data);
display.addReference(ref);
JPanel panel = ... // construct a JFrame and GUI
panel.add(display.getComponent()); // add display to GUI
Now, as your time series data come in, you can add them to
new_data as follows:
// read values for next record from real-time data
double val1 = ...
double val2 = ...
...
double valN = ...
double[] values = {val1, val2, ..., valN};
// get MathType of record
RealTupleType record_type
(RealTupleType) time_series_type.getRange();
// add record to new_data
RealTuple record = new RealTuple(record_type, values);
int index = ... // ordinal index of current time sample
// in new_set
new_data.setSample(index, record);
Each time you call setSample() this will trigger the
display to update and add the latest record.
Note that the time associated with "index" will be:
first + (index - 1) * step
where first is in seconds since some epoch (default is
1970-01-01 00:00:00Z) and step is in seconds. You can
use the DateTime class to help with time calculations.
Also note that you may eventually overflow the time span
of new_data, when you get real-time records for times
later than new_last. If that happens, you can of course
extend the time sampling of new_data in just the same way
that new_data extends the time sampling of data.
Finally, after you have collected a number of real-time
records and you want to save the combined history and
real-time data to a file, you can do:
// note "true" is to replace any existing file named
// "newfilename.nc"
plain.save("newfilename.nc", new_data, true);
Please let me know if you have any questions or problems.
Cheers,
Bill
----------------------------------------------------------
Bill Hibbard, SSEC, 1225 W. Dayton St., Madison, WI 53706
hibbard@xxxxxxxxxxxxxxxxx 608-263-4427 fax: 608-263-6738
http://www.ssec.wisc.edu/~billh/vis.html
visad archives: