[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #COT-972342]: Loading 3D Fields in IDV

Hi Josh-

> Full Name: Josh Gelman
> Email Address: address@hidden
> Organization: PSU
> Package Version:
> Operating System:
> Hardware:
> Description of problem: Hello,
> 
> Whenever I try to load a 3D field in IDV using a WRF ARW dataset, the time it 
> takes to load is extremely slow (up to a half hour sometimes).  For 2D fields 
> it's nearly instantaneous.  This only seems to happen with my WRF ARW 
> datasets.  The rest load pretty fast.  I am curious if there is any way to 
> resolve this issue?

When you say "only seems to happend with my WRF ARW datasets", are you 
comparing them to standard (e.g. GFS) grids or WRF NMM datasets?  Are you
loading single levels or are you loading in the entire volume and trying
to do cross sections or isosurfaces?

The problem with the WRF netCDF files (NMM and ARW) is that they
grids are staggered.  To handle this, we need to hold both the 
native grid domain and a transform to pressure in memory.  So
at the outset, just loading in the grid is a double wammy.  Your
performance is going to depend on the size of the grid and number
of timesteps as well.  What are the dimensions of your grid?
Any of the derived parameters have an added overhead of resampling
the data from one point of the domain to another.  For example,
wind speed or wind vectors require resampling the V to the U domain.
Those are grids that I've seen really bad performance on and the
culprit is the resampling.  Straight parameters like U wind or
vapor pressure shouldn't take much longer than a standard unstaggered
high resolution NAM grid.

In general, we suggest that users postprocess the WRF netCDF data
using WRF-POST or the Vis5D converter.  These destagger the grids
and output them on pressure levels as well as compute standard
parameters like  sea level pressure.

There are also some steps you can take to improve performance
listed at:

http://www.unidata.ucar.edu/software/idv/docs/userguide/misc/PerformanceTuning.html

> The computers I'm running IDV on have 16 GB of memory and 3 GHz processor 
> speed.

When the data is loading slowly, is the memory monitor showing that
you are maxing out on memory?  Have you increased the default
amount of memory that the IDV gets allocated?  If not, see the
performance tips above.  Depending on your operating system, you'll
only be able to use a subset of the total memory.

> Thanks for any suggestions.

I'm guessing that the slowdown is due to resampling which is painful
for the staggered grids.  But if you are seeing really slow loading
on one of the main fields in the grid, then there is some other
issue.

Don Murray

Ticket Details
===================
Ticket ID: COT-972342
Department: Support IDV
Priority: Normal
Status: Open