There are several features in the IDV that allow you to more efficiently view large datasets:
| 7.8.0 |
Temporal/Spatial Subset of Data
Subsetting the data before display reduces the memory and display time |
| 7.8.1 |
Memory allocation
Change the amount of memory allocated to the IDV |
| 7.8.2 |
Data Caching
Data caching uses more memory |
| 7.8.3 |
Caching to disk
Data source field caching |
| 7.8.4 |
Maximum grid/image size
Reducing the maximum size of a display can reduce the memory used |
| 7.8.5 |
Fast Rendering
Fast rendering reduces memory and time at the expense of accuracy |
| 7.8.6 |
Parallel Rendering and Data Reading
Parallel Rendering and Data Reading |
Properties menu of the data source (double click on the
Data Source in the Field Selector) or you can set
these for individual field using the tabs in the lower right corner
of the Field Selector. For more information, see
the Data Source Properties
section of the IDV User's Guide.
In addition, users can change the memory settings in the Edit→Preferences,
System tab. In order for these changes to take effect, the user will have to
restart the IDV.
In rare circumstances, the IDV start script cannot determine the optimal amount
of memory for the IDV. In this unusual case, the IDV start script allocates 512MB.
The user can still go to the Edit→Preferences, System tab and adjust memory
settings. Again, the user must restart the IDV in order for this to take effect.
In other exceptional situations, the user may still wish to override the automatic tuning mechanism. In the body of the runIDV script, there are instructions on how to achieve this change, although this should rarely be necessary.
Cache data in memory
checkbox on the System tab of
the user preferences (accessible from the Edit→Preferences menu).
The memory cache size is intially set at 30% of your maximum memory. This can be changed in the Edit→Preferences, System tab.
In a worst case scenario you could have a very long animation loop of imagery. In this case every time one of the images is displayed while animating its data needs to be accessed. If you have very large images or a very long loop then the images needed to display will be on disk and the time it takes to read them from disk for display will be quite noticable. In this case you can reduce the resolution of the images, reduce the number of times being displayed or increase the cache and/or overall memory size.
System tab of
the user preferences (accessible from the Edit→Preferences menu).
General tab of the user
preferences (Edit→Preferences menu)) is set, the IDV will not try
to account for the projection seams. If you are displaying data in
its native projection, this will result in faster rendering of the
data depiction. However, if you have several displays of data, each
from a different data source and on a different projection, you may see
anomalies in the displays (spurious lines, portions of images). At that
point, you can turn off fast rendering for a particular display using the
Edit→Properties menu of the Display Control for that display,
or set your system preference back to not use fast rendering.
There are 2 preferences in the Edit→Preferences, System tab.
One is the number of threads used for rendering. This defaults to the number
of processors on your machine. The second is the number of threads used for
data reading. This defaults to 4.
For rendering the IDV will render each time step in parallel. Note: since the rendering processes can allocate temporary memory it is possible to exhaust the available memory if too many threads are running concurrently. While we do not get linear speedup with the number of cores available for rendering (probably due to memory contention issues) we do see 40%-50% performance improvements for complex rendering tasks (e.g., contouring).
The second preference is used when reading individual time steps of data from remote ADDE and OpenDAP servers. This parallelization takes advantage of the multiple cores available on the remote server and somewhat the available bandwidth on the network. We do see a linear speed up in accessing remote data based on the number of cores on the remote server (ADDE or OpenDAP). However, we've seen that if you load the server too much your performance is degraded, probably due to file system issues.