[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #OOT-688821]: 1) Netcdf on Redhat Linux 6 has glitches 2) Polite questions



> The addendum:
> 
> Of course: If you have somewhere binaries for Redhat Linux, INTEL(!), IA32, 
> for netCDF I take of course THEM,
> that is: for Fortran and C as said.

Unidata doesn't maintain or support binary distributions of recent versions of 
netCDF.
We only support a source release for the current version, except for platforms 
that 
are difficult to build from source, such as Windows Visual Studio environments.

If you want a binaries for an older version 4.1.1 for RHEL, you can use the 
"yum"
command, but I think it only provides binaries for i686 or x86_64 platforms.

> I only want to run it with CESM from NCAR to add geological things into CESM, 
> that is: Reading and writing
> data-files from NCAR.
> 
> I "built" them only because the manual says so, e.g. I do not want it.

Unidata is not NCAR.  We develop and support open source software for the earth
sciences community for use in education and research, but we don't work directly
with the CESM developers, and we are much smaller than NCAR (14 developers).

> ***************
> 
> Answer to the email of address@hidden, now sent from a Windows system (and 
> not the Unix system yesterday).
> 
> I am the same(!) person.
> 
> There we misunderstood:
> 
> I _had_ to use the su command to avoid the error I showed.
> 
> I only wondered: Where did the script put the files? - as I could not find 
> them (= an additional(!) issue).
> 
> I found the files - so it made them: But(!) with su.
> 
> So: The additional things (files found) is solved. The "su" issue remains.
> I think that the "mkdir command" - when issued from a script, requires su 
> permissions (under the Perl)
> in contrast to a "manual" mkdir.
> I had a similar thing with an "older" CESM, e.g. the first orso CESM NCAR 
> released.
>
> So: mkdir "by hand" works anywhere.
> 
> mkdir "by perl-script" might require su (formulated as question) or different 
> rights-settings which I am not
> aware of (I focus on Fortran and Geology and not on scripts, e.g. I take 
> scripts "as is").


I don't understand why you mention perl scripts.  There are no perl scripts in 
the
netCDF distribution.  The configure script is a shell script.  The mkdir command
issued from a shell script has the same permissions as the user running the 
shell
script has on the command line, "manually".

No super-user permissions are required to run the netCDF configure script (or 
to run
"make check").  Only "make install" requires root permissions, as is the case 
with
most third-party libraries.  Even "make install" doesn't require root 
permissions if
you have specified an install location for which you have write permissions.

> Regarding configure:
> 
> I followed the netCDF page "netcdf 4.2.1.1, Main Page", "Building Netcdf with 
> Classic Library Only, pag 1/1 and 1/2.
> 
> There(!) it says to run "configure" and make check install.
> 
> I can send you the page as pdf as I printed it out.

That's not necessary, I'm familiar with the install instructions here:
  http://www.unidata.ucar.edu/netcdf/docs/build_classic.html

Would it be clearer if I changed the line

  make check install

to

  make check
  sudo make install  # if the installation location requires root permissions

?

> To comply fully with NCAR-Standards I did as I was instructed - whether it 
> was right or wrong I do not know
> but it is written there and I did so.

I'm not sure what "NCAR-Standards" you are referring to.  Unidata doesn't follow
any NCAR standards, we just develop the best software we can :-).

> Thankyou for your answer.
> 
> For the other things of the ticket(s) I wait.

So I thought I had answered your questions.  What question still needs a better
answer?  

Looking back at the questions and answers, I was trying to say that using

  --prefix=/PeterPaul/netcdf42

instead of 

  --prefix=/home/PeterPaul/netcdf42

(or where ever your $HOME directory is) was the source of the problems you saw.

The configure script can't tell you that it is an error to specify an 
installation
location that you don't have permissions to create, because it can't know 
whether
the "make install" command that is run later will be run as root, which would
permit that installation location.

--Russ

> -----Ursprüngliche Nachricht-----
> Von: Unidata netCDF Support [mailto:address@hidden]
> Gesendet: Dienstag, 12. Februar 2013 22:20
> An: address@hidden
> Cc: address@hidden
> Betreff: [netCDF #OOT-688821]: 1) Netcdf on Redhat Linux 6 has glitches
> 2) Polite questions
> 
> 
> Hi Dr. Smolka,
> 
> > to avoid wasting your time:
> >
> > The issue "where did the script place the files (below)?" is solved.
> >
> > The first run I did as "normal user", e.g. $HOME/netcdfinst/netcdf4.2.1.1
> >
> > There it had the problem.
> >
> > Then I ran it as "su": The script put the files into a /PeterPaul/netcdf42
> > -directory -as the script was told.
> >
> > The "user" is "PeterPaul" ($HOME) - same as the directory.
> 
> Right, it is not necessary to run the configure script or to run "make" or 
> "make check"
> as su, if you only want to install the executables and libraries in 
> directories where
> you have write permissions.  If you want to install the libraries and 
> executables in a
> directory where other users expect to find them, such as /usr/local/lib and 
> /usr/local/bin,
> then you need to either accept the default "/usr/local" location, or specify 
> a different
> location using the "--prefix=/where/ever" option to the configure script.
> 
> The only command you should run as su is "make install", after the libraries 
> are built and
> you have run "make check" to test that they work.
> 
> > ***************
> >
> > The compiler-question below:
> >
> > What I want is that it is compatible with the NCAR CESM1_1_1-scripts and
> > the compiler-flags there.
> >
> > How the text on p. 1/2 is to be understood accordingly I have to find out
> > (for the Intel 13.x Compiler).
> 
> Intel has published instructions for building the older netCDF 4.1.3 version
> with their compilers here:
> 
> http://software.intel.com/en-us/articles/performance-tools-for-software-developers-building-netcdf-with-the-intel-compilers
> 
> After version 4.1.3, we separated the Fortran software from the C software to 
> simplify
> maintenance and installation.  With version 4.2.1.1 that you have, only the C 
> libraries
> and utility programs like ncdump are built.  After version 4.2.1.1 is 
> installed, you can
> build the Fortran libraries separately from the netcdf-fortran-4.2 release, 
> telling its
> configure script where you installed the C libraries, as described here:
> 
> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-fortran-install.html
> 
> > CESM got "so script-laden" that I decided to "do it as NCAR does"
> > regarding the technology and focus on adding the paleoclimatology.
> >
> > Therefore: The script-compatibility with NCAR.
> 
> I'm not very familiar with the CESM scripts or procedures, so I'm not sure
> what that means.  Nevertheless, I'll try to answer the questions I understand 
> below.
> 
> > ---------- Forwarded message ----------
> ...
> > Dear Members of the Netcdf-Group,
> >
> > to run cesm (the new) with netcdf (I downloaded 4.2.1.something) on a new 
> > and
> > fully standard (I got six physically shipped Workstation-CDs from Redhat to
> > make sure
> > "everything is the standard").
> >
> > The Redhat 6.x  I installed as IA32.
> >
> > The new Intel-Compiler (with "everything", e.g. "Cluster Pack or so") I
> > installed as IA32 as well.
> >
> > It is a machine that is dedicated to CESM and related paleoclimatology.
> >
> >
> > I ran your the as indicated under:
> >
> > netcdf: Building Netcdf with Classical Library.
> >
> > First part: Logged on to the system as "normal" user with the name 
> > "PeterPaul"
> > and the home-directory $HOME/PeterPaul
> >
> > I made a directory under /PeterPaul called netcdf42, e.g.
> > $HOME/PeterPaul/netcdf42
> >
> > This is the directory where netcdf42 should be after installation.
> >
> > The install-directory has the name .../netcdf42inst
> >
> > Being inside the (...)/netcdf42inst/netcdf4.2.1.1 -directory:
> >
> > The glitches:
> >
> > using
> >
> > ./configure --prefix=/PeterPaul/netcdf42 -disable-dap
> >
> > 1) The script noted an "old" system date (2011, intentionally) and stopped.
> >
> > OK: Changed to today.
> >
> > 2) same: The script "complained" about a missing (something) (directory,
> > library etc.) and recommended to disaple netcdf4.
> >
> > 3) Done, e.g. dap disabled, netcdf-4 disable as written by you.
> >
> > 4) Result
> >
> > It ran quite far and passed many tests.
> >
> > Then he (the script) said (see below near "the end"):
> >
> > while creating a directory: "permission denied".
> >
> > (continuing below the stars ******* )
> >
> > netcdf error msg:
> >
> > PASS: do_comps.sh ================== All 2 tests passed ==================
> > make[3]: Leaving directory
> > `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples/CDL' make[2]: Leaving
> > directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples/CDL' 
> > make[2]:
> > Entering directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples'
> > make[2]: Nothing to be done for `check-am'. make[2]: Leaving directory
> > `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples' make[1]: Leaving
> > directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/examples' make[1]:
> > Entering directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1' make[1]:
> > Leaving directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1' Making 
> > install
> > in include make[1]: Entering directory
> > `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make[2]: Entering
> > directory `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make[2]:
> > Nothing to be done for `install-exec-am'. test -z 
> > "/PeterPaul/netcdf42/include"
> > || /bin/mkdir -p "/PeterPaul/netcdf42/include" /bin/mkdir: cannot create
> > directory `/PeterPaul': Permission denied make[2]: *** 
> > [install-includeHEADERS]
> > Error 1 make[2]: Leaving directory
> > `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make[1]: *** 
> > [install-am]
> > Error 2 make[1]: Leaving directory
> > `/home/PeterPaul/netcdf42inst/netcdf-4.2.1.1/include' make: ***
> > [install-recursive] Error 1
> 
> I think the errors were mostly from specifying --prefix=/PeterPaul/netcdf42 
> instead of
> /home/PeterPaul/netcdf42, in case that's what you intended.  Linux doesn't 
> permit
> normal users to create new directories at the top level.
> 
> > *********************
> >
> > 5) Bypass:
> >
> > Changed to su (set user, superuser, root)
> >
> > Restarted: It ran through and displayed congratulations.
> >
> > Thus it should work - except the questions:
> >
> > 6) Reset to "normal user".
> >
> > 7) In the /PeterPaul/netcdf42 -directory: Nothing found.
> >
> > 8) Then again changed back to "su".
> >
> > Found under "/usr/bin" quite a lot of files that look like netcdf (I know
> > netcdf since ccm3.6).
> >
> > So: Of course I can run the computer with "su" (administrator) but with 
> > Linux I
> > have really an uneasy feeling with it, except for short times (IT since 
> > 1979,
> > IBM-mainframe).
> 
> Right, as explained above you should only run "make install" as root and 
> everything,
> and then only if you are installing in directories that require root 
> permissions.
> 
> > Now the questions:
> >
> > Q1) Is it, as intended, a 32-Bit netcdf as I installed the Intel-Compiler
> > with IA32?
> 
> Yes, IA32 means you have installed 32-bit libraries.  That should be OK for 
> most uses,
> as you can still write and read netCDF classic files (with 32-bit file 
> offsets) and
> 64-bit offset netCDF files.
> 
> If it becomes necessary to write or read netCDF-4 files that are compressed 
> or chunked,
> or that have multiple variables larger than 4GB, you will need to rebuild 
> your libraries
> to use netCDF-4.
> 
> By the way, with RedHat Linux (or Fedora), you can get prebuilt binary 
> installations of
> netCDF, using the "yum" command, including netCDF-4 versions.
> 
> > Reason for IA32: All the "rest" is with IA32 and with 64-Bit all arrays are
> > "twice that large".
> 
> It's good to be compatible with other 32-bit libraries and programs that link 
> with shared
> netCDF libraries at run time.  However the on-disk arrays for netCDF-4 files 
> are typically
> about the same size as for netCDF-3 files, beyond a small fixed-size 
> overhead.  Only the
> slots in the file header that identify where variables start in the file are 
> 64 bits instead
> of 32-bits with 64-bit offset files.
> 
> > Q2) Is it correct that it installs (as it appears) in usr/bin?
> 
> The default installation location is /usr/local, so the libraries would be in 
> /usr/local/lib
> and the programs like ncdump, ncgen, nccopy, and nc-config would be in 
> /usr/local/bin.
> 
> > Or: How can I force it to $HOME/netcdf42? (and does it make sense in 
> > context of
> > CESM1_1_1?
> 
> It sounds like you figured out the answer to this one.
> 
> > Q3) Should I have written instead:
> >
> > ./configure --prefix=(the full directory path as root for PeterPaul, e.g.
> > /..../PeterPaul/netcdf42 starting at root (real root in the admin sense)?
> 
> Yes.
> 
> > I must admit: Running scripts as "root" creates a little bit an uneasy 
> > feeling.
> >
> > Is there a different way to enable scripts (applies also to CESM) to make
> > directories?
> 
> Yes, don't run scripts as root.  Only run "make install" as root if you need 
> to.
> 
> > Q4) The Fortran-part (on page 1/2) appears a little unclear.
> >
> > The C-Part was compiled (apparently) with the gcc of Redhat, which means:
> > The Intel C-Compiler was not(!) used.
> 
> The default is to use the first compiler found in your PATH, usually gcc.
> 
> You can force use of the Intel compiler by setting the CC environment variable
> before running the configure script, for example if you are using bash or sh 
> or
> ksh
> 
> CC=icc FC=ifort ./configure --prefix= ...
> 
> With csh or tcsh shells, you use a different syntax to set environment 
> variables.
> 
> Or you can just give the settings as arguments to configure at the end with 
> either
> kind of shell as in:
> 
> ./configure --prefix=... --disable-... CC=icc ...
> 
> > As I compiler normally Fortran with Intel (and as you have the CESM-group 
> > near
> > you): Will the Intel-Compiler be compatible with the settings of gcc?
> > (appears likely by the many settings you did, but I ask for experiences).
> 
> I think it's best to use icc and ifort together, or gcc and gfortran together.
> We don't test with mixed combinations.
> 
> > Into which directory should I place the netcdf-Fortran-Files (by NCARS
> > standard) to be compatible with the scripts?
> >
> > Email:
> > address@hidden (science)
> > address@hidden (private)
> 
> I don't know what NCAR's standard is.  I would use the same directory in
> --prefix=... for both C and Fortran libraries, or just accept the /usr/local
> default if you're willing to run "make install" as root.
> 
> > I want: Just to run it compatible to NCAR and after it: Adding 
> > paleoclimatology
> > and the respective things (already partially tested).
> >
> > The "technical" things (IA32, netcdf) should be "1:1 to the NCAR-standard".
> 
> To be sure you're compatible with the NCAR standard, I would ask someone from 
> the
> CESM group or the NCAR CISL consulting group.
> 
> > Thankyou very much for providing netcdf.
> 
> You're welcome, thank you for doing great science!
> 
> > Peter Smolka
> > (Dr. Peter P. Smolka)
> 
> --Russ
> 
> Russ Rew                                         UCAR Unidata Program
> address@hidden                      http://www.unidata.ucar.edu
> 
> 
> 
> Ticket Details
> ===================
> Ticket ID: OOT-688821
> Department: Support netCDF
> Priority: Normal
> Status: Closed
> 
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: OOT-688821
Department: Support netCDF
Priority: Normal
Status: Closed