Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [netcdfgroup] read performance slow compared to netCDF on other systems

  • To: "Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]" <matthew.thompson@xxxxxxxx>
  • Subject: Re: [netcdfgroup] read performance slow compared to netCDF on other systems
  • From: Liam Forbes <loforbes@xxxxxxxxxx>
  • Date: Fri, 11 Nov 2016 07:56:59 -0900
On Fri, Nov 11, 2016 at 5:22 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] <matthew.thompson@xxxxxxxx> wrote:

> A question: are you using the same version of NCO in both cases? A while
> back (year or two) I encountered a performance issue that was due to NCO
> and not netCDF (I tried many a combination). Charlie fixed whatever was
> causing the slowdown a while back, so if you just built NCO it's probably
> not that, but I wanted to make sure.
>

Matt,

I've built and am using NCO 4.6.1 on our new cluster. The researchers who
reported the problem aren't using NCO commands, they are using NetCDF from
within their applications. I hadn't considered that NCO might be skewing my
test case though. Is there another way to do whatever ncks is doing when it
reads in a file and writes out a new one with no options specified on the
command line? I can see that it's not just an nccopy because the resulting
file is different. I guess this is where my ignorance of what these
packages do starts to show.

Our comparison case, the Cray, has NCO 4.1.0.

-- 
Regards,
-liam

-There are uncountably more irrational fears than rational ones. -P. Dolan
Liam Forbes  loforbes@xxxxxxxxxx  ph: 907-450-8618 fax: 907-450-8601
UAF Research Computing Systems Senior HPC Engineer  LPIC1, CISSP
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: