[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #FDK-698636]: IDV v. Panoply OPeNDAP access



> 
> On Apr 5, 2011, at 5:01 PM, Unidata IDV Support wrote:
> 
> >> First, let me stipulate that I know IDV does many things Panoply doesn't, 
> >> and I use and proselytize it regularly.  But lately i have run into a 
> >> couple discrepancies where Panoply can handle data through OPeNDAP that 
> >> IDV struggles with due to performance reasons.  I wonder if there is 
> >> anything you can do to catch up...
> >>
> >> Case #1:  MODIS Level 3 data in OPeNDAP, hyrax variety (please do not 
> >> advertise this URL, though, it is intended primarily for machine use by 
> >> our Giovanni app):
> >>
> >> http://disc1.gsfc.nasa.gov/opendap/Aqua_MODIS_Level3/MYD08_D3.051/2011/MYD08_D3.A2011001.051.2011005093254.pscs_000500527176.hdf
> >>
> >> This is a very large file with many, many variables.  That said, Panoply 
> >> takes about 20-25 seconds to inventory the variables in the file, and 
> >> another 5-10 secs to display one.  IDV takes ~6 minutes to inventory the 
> >> data file on a MacBook Pro, but then about the same time as Panoply to 
> >> display one variable. (BTW, this is actually a subset of the original 
> >> MODIS L3 file, which is even larger!)
> >>
> >
> >
> > I tried this url and it took about 15 seconds, so it really depends on the 
> > network traffic and loads of the server.
> 
> And yet, every time I have tried this URL on my laptop, it has taken at least 
> 5 minutes to inventory the file.  The server has > 90% idle (load average 
> about 2) during the transaction and usually serves very little data either to 
> the public or our internal machines, due to its less-popular holdings.  And 
> this does not explain why Panoply, when executed either shortly before or 
> shortly after, takes less than 30 seconds to inventory the same URL.
> 
> >
> >
> >> Case #2 Merged IR in OPeNDAP of the GrADS Data Server (GDS) variety: 
> >> http://disc2.nascom.nasa.gov:80/dods/MergedIR
> >>
> >> GDS presents the whole dataset as one huge "file".  Panoply can display 
> >> this after a few minutes of churning; however IDV eventually errors out 
> >> after several minutes with a message about Java heap space.
> >>
> >> Case #2 would be more useful for us to get fixed--we had a user asking us 
> >> how to read the Merged IR through IDV.
> >>
> >> Anything you can do?
> >
> > My IDV has 4G memory, and it has no problem to load the data. So the 
> > solution for your user is to increase the memory.
> 
> What do you mean your "IDV has 4G memory"?  Is there some way of allocating 
> more memory to that particular app on a Mac laptop?  (I know there was on old 
> Mac operating systems, but I cannot find the mechanism on Snow Leopard.)  Or 
> do you mean I  have to add more physical memory to my laptop?  It actually 
> currently has 4 GB.  Also, since many of our users come from the developing 
> world, or small institutions, we cannot always suggest to them to upgrade 
> their hardware.


Yes, you can allocate more memory to the IDV, see the FAQ here: 
http://www.unidata.ucar.edu/software/IDV/docs/userguide/Faq.html.

And you still need to subset the time dimension of the dataset when creating 
the display.  The idea of input the url like this 
http://disc2.nascom.nasa.gov/dods/MergedIR.ch4[0:5][0:0][0:3297][0:9895]lon[0:9895]lat[0:3297]lev[0:0].dds

is bad, you can subset after loading the dataset 
http://disc2.nascom.nasa.gov/dods/MergedIR, and then create the display.



Yuan
> 
> >
> >
> >
> > Yuan
> >> --
> >> Dr. Christopher Lynnes     NASA/GSFC, Code 610.2    phone: 301-614-5185
> >>
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: FDK-698636
> > Department: Support IDV
> > Priority: Normal
> > Status: Open
> >
> 
> --
> Dr. Christopher Lynnes     NASA/GSFC, Code 610.2    phone: 301-614-5185
> 
> 
> 


Ticket Details
===================
Ticket ID: FDK-698636
Department: Support IDV
Priority: Normal
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.