Compiling OpenMPI with GCC-4.x.x

In order to compile OpenMPI, you will need a working compiler for the desired target languages. For the purposes of this post, I will assume the use of the GCC-4.2.1 or later, with target languages of C,C++, and GFORTRAN. See previous posts for information on compiling the GCC-4.x.x compilers and their prerequisites. From my personal experience as a C/C++/FORTRAN developer, it is usually best to avoid using the base compilers on most linux distros, especially when linking against existing scientific libraries, in order to avoid shared library versioning problems. So, for best results, build your own GCC compilers with a prefix directory similar to "/usr/local/gcc421". ALWAYS use "--enable-static --disable-shared --with-pic" flags when configuring your GCC compiler builds to avoid shared library compiler dependencies in your compiled code. Make sure you add the bin and lib directories to the front of your PATH and LD_LIBRARY_PATH variables in your shell environment (.bashrc or .profile). Using the above prefix directory as an example:

export PATH=/usr/local/gcc421/bin:$PATH
64-bit:
export LD_LIBRARY_PATH=/usr/local/gcc421/lib64:$LD_LIBRARY_PATH
32-bit:
export LD_LIBRARY_PATH=/usr/local/gcc421/lib:$LD_LIBRARY_PATH

Additionally, you will want to set the following environment variables to point to the proper compilers:

export CC=/usr/local/gcc421/bin/gcc
export CXX=/usr/local/gcc421/bin/g++
export F77=/usr/local/gcc421/bin/gfortran
export F90=/usr/local/gcc421/bin/gfortran
export FC=/usr/local/gcc421/bin/gfortran

Once you have the compilers installed, download a source tarball of the latest stable version of OpenMPI and untar it somewhere in your home directory. When configuring the openmpi source, use a prefix directory name that indicates what version of openmpi and gcc are being used. For example, if your base compiler is gcc-4.2.1 and your are compiling openmpi-1.4.2, your prefix could be as follows:

"--prefix=/usr/local/ompi142-gcc421

In my experience, correctly labeling of the prefix directory avoids version confusion and allows your to easily switch between compiler/openmpi versions in the future by changing your PATH and LD_LIBRARY_PATH environment variables as follows:

gccver="421"
if [[ $gccver == "421" ]]
then
export CC=/usr/local/gcc421/bin/gcc
export CXX=/usr/local/gcc421/bin/g++
export F77=/usr/local/gcc421/bin/gfortran
export F90=/usr/local/gcc421/bin/gfortran
export FC=/usr/local/gcc421/bin/gfortran
ompiver="142" #change to match your desired version
if [[ $ompiver == "142" ]]
then
export PATH=/usr/local/ompi142-gcc421/bin:/usr/local/gcc421/bin:$PATH
#For 64-bit
export LD_LIBRARY_PATH=/usr/local/ompi142-gcc421/lib:/usr/local/gcc421/lib64:$LD_LIBRARY_PATH
#For 32-bit, comment the line above and uncomment the below
#export LD_LIBRARY_PATH=/usr/local/ompi142-gcc421/lib:/usr/local/gcc421/lib:$LD_LIBRARY_PATH
elif [[ $ompiver=="???" ]] #other installed versions
....
fi

elif [[ $gccver == "???" ]] #other installed versions
then
....
fi

In general, I build openmpi as follows using the listed configuration flags:

./configure --prefix=/usr/local/ompi142-gcc421 --enable-static --disable-shared --with-pic
make 2>&1 | tee make.log

If your prefix is directory is located somewhere within a root owned directory, then do either

sudo make install

or

su root
make install

Otherwise, if you are installing in a user owned directory, just do

make install

If you've followed all these steps, you should now have a working OpenMPI installation. In order to automatically include the proper libraries and paths, remember to use the proper mpi compiler wrapper (mpicc, mpic++, mpif90) to compile your code. To see which compiler the wrapper is pointing to you can use the following to return the underlying compiler version information.

mpicc -v

I hope you have found this blog post helpful in you parallel computing endeavors.

Bootstrapping GCC-4.x.x (4.3.3): configure, build, install

In previous posts I covered some prerequisite information for compiling GCC-4.x.x, such as downloading the GCC-4.x.x package and downloading/compiling the proper GPM and MPFR libraries.

NOTE: DO NOT BUILD GCC IN THE SOURCE DIRECTORY!

The first thing to do after uncompressing the tarball is to make a build directory, generally in the same directory as the source directory. For example, if the source directory is gcc-4.x.x, create a build directory called gcc-4.x.xbuild.

The next step is to properly configure/make/make install GCC-4.x.x. For general users, the default settings work fine and the following commands are sufficient:

> cd /path/to/gcc-4.x.xbuild
> ../gcc-4.x.x/configure
> make bootstrap
> sudo make install

However, there are many times that the default setting are not sufficient and must be changed. Some examples where alternate setting are needed are:

• installing somewhere other than /usr/local


• changing the desired languages to compile


• changing the program suffix


• enabling/disabling shared/static libraries


• using gmp/mpfr from a different directory path

To see information on available configure options, type ./configure --help in the gcc-4.x.x source directory. Some configure options are platform dependent and if you have problems a good place to start is to look at the configure options used for the default system compiler by executing gcc -v to see a list of the configure options used. Some of the options that I use most are listed below:

• --prefix=/custom/install/path


• --mandir=/custom/man/path


• --infodir=/custom/info/path


• --enable-bootstrap


• --enable-static


• --disable-shared


• --with-pic


• --enable-languages=c,c++,fortran,java

Generally, when I am bootstrapping GCC-4.x.x for CFD work, I build only the c,c++, and fortran languages. In addition, I always perform bootstrap builds and perform the make check step to verify the build. Also, since I am a CFD researcher/developer and may need to run my code on a variety of platforms, I bootstrap generally bootstrap GCC with static enabled and shared disabled, but with PIC. This enables me to move the CFD executables to machines that do not have the same GCC libraries, while allowing me to link in shared object files as needed. Below is a typical bootstrap configuration and build process that I would use:


> cd path/to/gcc-4.3.3build
> ../gcc-4.3.3/configure --prefix=/usr/local/gcc433 --mandir=/usr/share/man --infodir=/usr/share/info --disable-shared --enable-static --with-pic --enable-threads=posix --disable-multilib --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,fortran --with-cpu=k8 --with-gmp=/usr/local/gmp422-mpfr231 --with-mpfr=/usr/local/gmp422-mpfr231 --program-suffix=-433
> make bootstrap 2>&1 | tee make.log
> make -k check 2>&1 | tee make.log
> sudo make install


Notice the custom install directory indicated by the prefix option, the custom gmp and mpfr versions and directories, and the use of the program suffix -433 to make sure that I always know the version of gcc I am using for compilation. The above worked well for a Linux CentOS-4.4 OS on a dual processor, dual core AMD opteron system.

Well, that completes a brief introduction to bootstrapping the GCC-4.x.x compiler series. If you have questions. comments, or corrections, please hop over to my website http://clusternut.net and click the clusternut forums link.

Compiling GMP and MPFR

GMP (Gnu Multi-Precision Arithmetic library) and MPFR (Multi-Precision Floating-Point with Rounding) libraries are prerequisites for bootstrapping the GCC-4.x.x compiler suite. Many modern Unix/Linux systems may have compatible GMP and MPFR library versions already installed; however, if the libraries are not present or the versions are too old, you will need to download either a binary package for your system or the source tarball and compile them yourself. When in doubt, download the sources and compile them yourself to ensure that you have the latest, fastest, and most bug-free libraries.

Once you've downloaded the source tarballs from the websites listed in my previous post and expanded them in a build directory, be sure to read the README and INSTALL text file for the latest requirements and instructions for building the libraries. Both libraries use an Autotools (autoconf, automake, etc.) generated configure script to generate a Makefile for compiling. Make sure to run "./configure --help" in a terminal to see all available configure options. Both GMP and MPFR have options for building shared and static libraries. If you decide to build static-only libraries or if you think you may want to explicitly link a static library into a shared binary, make sure you put "--with-pic" in the list of configure options. Also, you may want to make sure the proper compiler is used to build the libraries by setting the environment variables for them (CC,CXX,FC,CFLAGS,etc.) as described in the "./configure --help" output.

Below are the configure options I used on an AMD Opteron cluster running CentOS 4.4:

GMP:
> ./configure --prefix=/usr/local/gmp422-mpfr231 \
--disable-shared --enable-static --with-pic \
--infodir=/usr/share/info --mandir=/usr/share/man
> make check
> su root
> make install

In this configuration for GMP, I set the installation directory with the '--prefix=/usr/local/gmp422-mpfr231' option. This allows me to keep matching versions together while retaining other version on the system. Also, I disabled building shared libraries to simplify executing on other computers the programs that link against GMP. Using the '--with-pic' option allows the static GMP library to be linked into executables that also link against shared libraries. And finally I included the '--infodir=/usr/share/info --mandir=/usr/share/man' options to install the man and info files in the proper directories for my system.

Compiling MPFR is similar to GMP with the addition of the '--with-gmp=/usr/local/gmp422-mpfr231' option to point to the location of our new GMP library:

MPFR:
> ./configure --prefix=/usr/local/gmp422-mpfr231 --with-gmp=/usr/local/gmp422-mpfr231 --with-pic --disable-shared --enable-static
> make check
> su root
> make install

Congratulation! You are now ready to bootstrap GCC-4.x.x. There are a lot of issues to cover with bootstrapping GCC so I may have to cover that in several posts.

Have a Happy New Year!

Bootstrapping GCC-4.x.x: Prerequisites

In this post, I will be discussing the general process of compiling GCC, along with how to obtain the necessary source code for GCC and its prerequisites.

Bootstrapping a compiler is the process of building the compiler from source code that is written in the target computing language itself with minimal external resources. Therefore, when bootstrapping GCC-4.x.x, the system must containing a working C compiler and some pre-requisite libraries.

For the GCC-4.3.x series of compilers, the pre-requisite libraries are the GNU Multi-Precision Arithmetic library (GMP-4.1+) and the Multi-Precision Floating-Point with Rounding library (MPFR-2.3.0+). Earlier version of GCC-4.x.x only required these libraries when building the GCC GFORTRAN compiler; however, in all recent GCC versions, these libraries are required for building the compiler. Many modern UNIX/Linux systems come with these libraries pre-installed, however you must be sure to check the version of the included libraries against the version requirements for the specific GCC-4.x.x compiler you desire to bootstrap. Some of the older versions of these libraries contain bugs that can adversely affect the accuracy of the resulting bootstrapped compiler. The version numbers for the preinstalled prerequisite libraries can be found in the header files, gmp.h and mpfr.h, respectively.

GCC-4.x.x is an open source compiler collection know as the Gnu Compiler Collection (GCC), whose homepage is locate at http://gcc.gnu.org. The GCC homepage contains a wealth of information straight from the developer and user community. To download the desired version of GCC, just follow the appropriate links on the GCC homepage to one of the many ftp or http mirror sites. The source code comes in the form of a compressed tarball ending in ".tar.bz2". Once the source has been downloaded to your computer, put it in a build folder with a name such as "BuildDir" and expand the tarball with the following command: "tar -xjvf gcc_tarball_name.tar.bz2".

The GMP is another opensource project, whose homepage is located at http://gmplib.org. Just following the links on the homepage to download the desired GMP source code. Move the tarball to your "BuildDir" and expand the tarball. If the tarball ends with ".tar.gz", it can be expanded with the command: "tar -xzvf source_name.tar.gz". Similarly, MPFR is also an open source project, with it's homepage located at http://www.mpfr.org. Again, follow the links to download the appropriate source, move it to your "BuildDir", and expand the tarball.

Congratulations, you now have all the necessary source code to bootstrap GCC. In my next post, I will walk you through compiling and checking both the GMP and the MPFR libraries. I will also shared some of my experiences and tips for avoiding trouble down the road.

Until next time, God Bless and Have a great day!!!

Choosing a compiler - GCC-4.x.x, the GNU Compiler Collection

In order to compile or develop an HPC application, a solid compiler package is needed. From my experience, the GCC-4.x.x series of compilers offer the best balance of cost (FREE!!!), performance (competitive with commercial compilers from INTEL, IBM, PGI, PathScale, NAG, etc.), developer access (communication with developer community, rapid bug fixes, etc.), features, and standard compliance. In addition, the GCC-4.x.x compiler collection is the default compiler package on Linux, Unix, and Mac OS X operating systems. Commercial compilers from vendors such as those listed above offer their own unique advantages in certain areas; however, for my own personal (and professional) use the GCC-4.x.x is my compiler package of choice.

If the default compiler is sufficient (stability, features, performance, and relatively bug-free for your uses), then by all means use the default compiler on your system. However, if the system GCC installation is not sufficient for your needs (i.e. not a recent enough version , etc.), then your will need to either install a set of pre-built binaries or download the GCC source code of a more recent version and compile/bootstrap it yourself. In the near future, I plan to offer downloadable pre-built GCC binaries for a few select platforms (Mac OS X, RedHat Linux, Ubuntu Linux). In addition, future posts will descibe the process of downloading and compiling GCC-4.x.x along with its pre-requisite packages (GMP and MPFR). Once this is done, I will be discussing MPI package options and do a walk through of compiling and setting up OpenMPI. Until next time, may your clusters always crunch with minimal downtime! 

Intro to ClusterNut Notes

Hello and welcome to ClusterNut Notes! I am the ClusterNut and I will be writing on topics related to High Performance Computing (HPC), parallel computing clusters, MPI programming hints, and just about anything else related to scientific computing that crosses my mind. I am looking forward to sharing my knowledge and experience in the HPC field with you. Though I have quite a bit of experience in scientific computing and running clusters, I am by no means a guru and look forward to hearing your insight and suggestions as well.