Re: Extreme memory usage (Code actually attatched)

Whoops, forgot to attatch the code.

Here it is.


On Fri, 30 May 2003, Ben Monnahan wrote:

> Hi, I'm having some problems with VisAD using seemingly way more memory
> than it should.  I'm loading a data file that is less than a megabyte, yet
> my program wants at least 300MB(it throws an OutOfMemoryError).  I figure
> I must be doing something wrong.  I'm not really sure what it could be
> because there just isn't that much data around.  In fact all the data
> into loaded into an array before the usage starts skyrocketing.  When I
> start creating all of the VisAD stuff the usage skyrockets.  I know people
> are using this for large data sets, so I must be doing something wrong.
> Can anyone see what might be causing the problem? (Code attatched, if you
> need clarification of something, or more of the code let me know)
>
> The reason I'm using the arrays for everything is that I have a 6 models
> that I want to plot data for, but there are a bunch of different views.
> There is a separate view for each datatype/forecast time/sample area.  I
> want to be able to switch between the views quickly so I'm loading all the
> data and adding it to the display, and then adding and removing references
> to only show the data that I want.  So this is much more data than I'm
> displaying but still much less than 1 MB total.
>
> Again if you have any questions I'd be happy to answer them.
>
> Thanks,
> Ben
>


/*
* Explanation of implementation:
*
*   _data               This is a Vector that is used to store the data sets.
*                       We only need one. 
*   _cMaps              This is where the ConstantMaps are stored. The first
*                       one is for the indep var, and the second is for the dep
*                       var.
*
*   // stuff for drawing lines
*   _lineTypes[]        This is where the RealType definitions are stored. The
*                       first one is the independent variable, the second one
*                       through the second from last are the dependent
*                       variables. The last one is the type to use when the
*                       axis's are combined.
*   _lineFuncs[]        This is where the FunctionType definitions are store.
*   _lineSets[]         This is where the sets of numbers used for the
*                       dependent variable values are stored.  We only need
*                       one.
*   _lineFlatFields[]   This is where we store the FlatFields.
*   _lineDataRefs[]     This is where we store DataReferenceImpl.
*   _lineSMaps[]        This is where the ScalarMaps are stored.  The 0'th one
*                       is for the indep var, and the 1'th through
*                       totalNumPlots for the dep var. totalNumPlots+1 is the
*                       generic type for all the dep vars.
*   _lineSMaps2[]       This is where the ScalarMaps used for selecting a
*                       display range are stored.
*
*   // stuff for drawing points
*   _pointTypes[]       This is where the RealTupleType definitions are stored.
*                       The first one is not used(indep type is RealType), the
*                       second one through the second from last are the
*                       dependent variables. The last one is the type to use
*                       when the axis's are combined.
*   _pointFuncs[]       This is where the FunctionType definitions are store.
*   _pointSets[]            This is where the sets of numbers used for the
*                       dependent variable values are stored.
*   _pointFlatFields[]  This is where we store the FlatFields.
*   _pointDataRefs[]    This is where we store DataReferenceImpl.
*
*
*   _plotCountTotal     This is the number of data sets being plotted.
*/

//initialize the point stuff
RealType index = RealType.getRealType("index", SI.second);
_pointSets[0] = (Set) _lineSets[0].cloneButType(index);
for(int i=1; i < _pointTypes.length; i++)
{
        _pointTypes[i] = new RealTupleType(_lineTypes[0], _lineTypes[i]);
}



// Create the maps for the independent variable
_lineSMaps[0] = new ScalarMap( _lineTypes[0], Display.XAxis );
_lineSMaps[0].addScalarMapListener(new TickMarkManager(new
                        DateTickMarkAlgorithm()));
_lineSMaps2[0] = new ScalarMap( _lineTypes[0], Display.SelectRange );
_display.addMap(_lineSMaps[0]);
_display.addMap(_lineSMaps2[0]);


//create the maps for the generic dep variable
_lineSMaps[_lineSMaps.length-1] = new ScalarMap( 
_lineTypes[_lineTypes.length-1], Display.YAxis);
_display.addMap(_lineSMaps[_lineSMaps.length-1]);



//do all the dependent variable stuff
for(int i=1; i <= _plotCountTotal; i++)
{

        System.out.println(" dep var: " + i);
        //- Set up all of the VisAD stuff -// 


        // Set up the ScalarMaps
        _lineSMaps[i] = new ScalarMap( _lineTypes[i], Display.YAxis );
        _lineSMaps2[i] = new ScalarMap( _lineTypes[i], Display.SelectRange );
        _display.addMap(_lineSMaps[i]);
        _display.addMap(_lineSMaps2[i]);

        // Set up the functions
        _lineFuncs[i-1] = new FunctionType( _lineTypes[0], _lineTypes[i] );

        _pointFuncs[i-1] = new FunctionType(index, _pointTypes[i] );


        // Set up the flatfields
        _lineFlatFields[i-1] = new FlatField( _lineFuncs[i-1], _lineSets[0] );
        _lineFlatFields[i-1].setSamples( (float[][]) _data.get(i-1) );

        _pointFlatFields[i-1] = new FlatField( _pointFuncs[i-1], _lineSets[0] );
        float[][] tmpData2 = (float[][]) _data.get(i-1);
        float[][] tmpData = new float[2][tmpData2.length];
        tmpData[1] = tmpData2[0];
        tmpData2 = _lineSets[0].getSamples();
        tmpData[0] = tmpData2[0];
        _pointFlatFields[i-1].setSamples( tmpData );
        System.out.println(" number of datapoints: " + tmpData.length +
                        "by " + tmpData[0].length);


        // Set up the data references
        _lineDataRefs[i-1] = new DataReferenceImpl("line_data"+(i-1)+"_ref_" + 
_graphID);
        _lineDataRefs[i-1].setData(_lineFlatFields[i-1]);

        _pointDataRefs[i-1] = new DataReferenceImpl("point_data"+(i-1)+"_ref_" 
+ _graphID);
        _pointDataRefs[i-1].setData(_pointFlatFields[i-1]);


        //- Add it to the display and set it to be shown  -//
        showPlot(i-1, true);

        //System.out.println(" #### tick");
}
System.out.println(" #### created and added dep smaps");