CMake policies vs. minimum CMake version

CMake policies typically default to OLD behavior compatible with prior versions of CMake. However, cmake_minimum_required() automatically sets all the CMake policies to “new” through the minimum CMake version. For example, a project with cmake_minimum_required(VERSION 3.14) implicitly sets to NEW all CMake policies CMP0088 and older.

Tradespace

New versions of CMake don’t get the benefits of new behaviors until specifying the new behavior is OK, either implicitly via cmake_minimum_required() or explicitly via cmake_policy(). The downsides of a too-old cmake_minimum_required() include that you have to set if(CMAKE_VERSION VERSION_GREATER_EQUAL ...) statements around extra cmake_policy() statements.

Important policies

Here are a few CMake policies we generally use, typically implicitly by choosing at least CMake 3.13:

cmake_minimum_required(VERSION 3.13)
  • CMP0074 find_package uses PackageName_ROOT variables
  • CMP0076 target_sources() command converts relative paths to absolute. This allows specifying targets in the top-level CMakeLists.txt while having add_subdirectory() containing their sources.
  • CMP0077 option() honors normal variables. Does not manipulate variables when user has defined their value.

Deprecation

CMake gradually deprecates old policies, meaning the deprecated policies default to NEW behavior. For example, CMake 3.18 deprecates CMP0071 and older.

Make Clang environment for Windows

As with GCC on Windows it’s often useful to have separate development environments for each compiler. The Powershell script “~/clang.ps1” creates a Clang LLVM environment. We don’t permanently put Clang on our user or system PATH. Running “~/clang.ps1” in Powershell enables Clang until that Powershell window is closed.

$Env:CC="clang"
$Env:FC="flang"
$Env:CXX="clang++"
$Env:path += ";C:/Program Files/LLVM/bin"

If you need to use the MSVC CL-like clang driver clang-cl, create “~/clang-cl.ps1” and run it when desired.

$Env:CC="clang-cl"
$Env:FC="flang"
$Env:CXX="clang-cl"
$Env:path += ";C:/Program Files/LLVM/bin"

Benefits of CMake Object Libraries

Since CMake 2.8.8 CMake Object Libraries can be used to keep build directories less cluttered and speed up the build process. The traditional workflow for Makefiles was to create lots of object files (targets) with the relevant compile options, definitions and link flags. Then the final executable or library targets would be linked from all the object files.

CMake add_library() defaults to creating static libraries. The following stanza creates “libfoo.a” or similar:

add_library(foo src.f90)

With numerous targets, there are a lot of “lib*.a” that may be cluttering the build directory. Also, there is a finite time consumed in creating all those archive .a files from the object files. For projects that use Visual Studio, using Object Libraries can help avoid complications with exporting symbols for MSVC.

CMake object libraries

A cleaner option for most cases where the “lib*.a” artifact is not needed is to create CMake Object Libraries instead:

add_library(foo OBJECT src1.f90 src2.f90)
add_library(bar OBJECT src3.c src4.c)

add_executable(baz main.cpp)
target_link_libraries(baz foo bar)

instead of having files under build/ like “libfoo.a, libbar.a” there will be only “baz.exe”. The object files will be tucked away under build/CMakeFiles/foo/*.o and build/CMakeFiles/bar/*.o.

The INTERFACE target properties of object libraries propagate up to targets linking them as usual. You may need to modify the PUBLIC / PRIVATE / INTERFACE of object libraries or add them to the “target_link_libraries()” of the top-level targets when going from static or shared libraries to object libraries.

Alternative

The generator expressions expand to all the object files of “foo” and “bar” like a legacy Makefile.

add_executable(baz main.cpp $<TARGET_OBJECTS:foo> $<TARGET_OBJECTS:bar>)

Caveats

Using object libraries means that duplicate symbols can cause conflicts that need to be manually resolved. When linking non-object libraries, the linker typically just uses the first instance seen of a symbol.

Cygwin startup environment variables

Cygwin startup configuration can be easily modified. While advanced users might modify the Windows CYGWIN environment variable, typical needs can be handled from within Cygwin ~/.bashrc or from other Windows environment variables.

A typical need is to not populate Cygwin PATH from Windows PATH. This is something we typically do to avoid confusion from Windows programs being used instead of Cygwin programs. To do this, set Windows environment variable:

CYGWIN_NOWINPATH=1

This will result in a basic Cygwin PATH like:

/usr/local/bin:/usr/bin

More options

Look inside Cygwin “/etc/config” file for more environment variables that may be of interest.

Don’t populate WSL PATH from Windows PATH.

Git reword prior commit messages

Git history can be rewritten with certain caveats. One reason to rewrite history for feature requests is to have project-specified message formatting. Another is to add long comments, denoted by adding at least one blank line beneath the minimal summary message.

To edit messages of prior commits, use git rebase -i and then select “r” or “reword” for the desired commits. This is quicker and easier than “e” or “edit” that is used for editing code or breaking up prior commits.

Fix MPI_Z_LIBRARY CMake error

OpenMPI uses “libz” internally. For the past few years, using OpenMPI from Cygwin also requires:

setup-x86_64.exe -P libopenmpi-devel zlib-devel

Otherwise, when building with OpenMPI with CMake OpenMPI might not be found due to:

Missing MPI_Z_LIBRARY

This might occur on other platforms where libz development libraries are not available when building with libopenmpi.

Install PGI free Fortran compiler

The PGI Community Edition compilers are a set of no-cost, non-open-source, high performance compilers for:

  • C: pgcc (C11)
  • C++: pgc++ (C++17, Linux only)
  • Fortran: pgfortran (Fortran 2003)

Install

If you have an older PGI compiler installed, uninstall that first to avoid license server issues–even when using the no-cost Community Edition.

Download and install PGI Community Edition:

  • Linux: sudo not required, but must be on symbolic-link-aware drive (not ExFAT)
  • Windows: Requires Visual Studio
  • MacOS is no longer supported by PGI after version 19.10.

PGI compiler is over 2 GB download. CUDA install is optional.

Linux

We assume PGI was installed under “~/.local/pgi” – add to ~/.bashrc like:

export PATH="$PATH:$HOME/.local/pgi/linux86-64/2019/bin/"

Open a new terminal to use PGI pgfortran pgcc pgc++

Windows

use PGI command prompt–you may need to do

set FC=pgfortran
set CC=pgcc

Note that there is no pgc++ on Windows.

PGI on Windowss requires Visual Studio. If upgrading Visual Studio (even a minor update) after installing PGI, one may need to reinstall PGI compilers.

Usage

Typically we invoke compilers from a build system such as Meson or CMake.

Meson

Meson build system supports PGI and Flang compilers.

project('demo','fortran', meson_version: '>=0.52.0')

fc = meson.get_compiler('fortran')
if fc.get_id() == 'pgi'
  message('PGI compiler')
endif

CMake

CMake distinguishes between PGI and Flang compilers. Set compiler-specific options in CMakeLists.txt for various Fortran compilers like:

project(myproj Fortran)

if(CMAKE_Fortran_COMPILER_ID STREQUAL PGI)
  add_compile_options(-Mdclchk)
endif()

select the PGI compilers at CMake configure step:

FC=pgfortran CC=pgcc CXX=pgc++ cmake -B build

PGI Debugger

PGI 19.10 ended the “pgdbg” graphical debugger. The Java-based “pgdbg” graphical debugger was for Fortran, C and C++, including the no-cost Community Edition.

HDF5 Fortran autotools build

The HDF5 1.12 beta release seems to have broken CMake scripts. The CMake scripts for HDF5 invoke autotools anyway, so we decided to just use autotools to build HDF5 1.12:

./configure --enable-fortran --enable-build-mode=production --prefix=~/.local/hdf5-1.12/

make -j4 install
-j4
number of build threads (arbitrary)

This creates static (.a) and dynamic (.so) HDF5 library archives under the user install prefix:

hdf5/lib/libhdf5.a
hdf5/lib/libhdf5_fortran.a
hdf5/lib/libhdf5_hl.a
hdf5/lib/libhdf5_hl_fortran.a
hdf5/lib/libhdf5hl_fortran.a

The Fortran .mod files that need to be included are under

hdf5/include/

Usage

Tell CMake to use this new HDF5 1.12 from your project by:

cmake -B build -DHDF5_ROOT=~/.local/hdf5-1.12/hdf5

cmake --build build

GNU Make parallel build

GNU Make is an important legacy build system. Ninja build system is recommended for new and upgrading projects–Ninja works with CMake and Meson among others.

In general, build systems are used to:

  • save time on builds, only recompile parts of the code that changed
  • build in parallel (big compile time speedup)
  • avoid copy-paste mistakes with excessively long compile commands or piles of Bash scripts

In general, don’t build on FAT or ExFAT drives, since if the build script tries to make symbolic (soft) links, it will fail since ExFAT doesn’t allow soft links.

GNU Make parallel builds

By default, GNU Make only uses one thread. For most programs, speeding up compilation is desirable and accomplished via the -j or --jobs option:

make -j

The -j option alone uses all virtual CPU cores. On a typical laptop, 4 to 8 threads will thus be compiling simultaneously where possible. On some systems such as Windows Subsystem for Linux, the bare make -j -l2 may overwhelm the computer because WSL1 doesn’t seem to correctly report the load factor. Thus to do parallel builds with GNU Make on WSL1, manually set the number of parallel build threads perhaps equal to or less than the number of physical cores in your PC. For example an 8 physical core laptop would compile with make -j8.

Caveats

In certain special cases like older Raspberry Pi computers, the CPU may go into thermal cutback or suffer undervoltage spikes if using an inadequate power adapter. While the better solution is to provide adequate DC power and CPU cooling, one may choose to compromise by manually specifying the number of threads. For example, with the Raspberry Pi I often use 1 or 2 threads.

make -j1

# or

make -j2

RAM limitation

make -j does not consider system RAM, so your computer may become overwhelmed. Try make -j2 or so until the system memory isn’t over-consumed.

Keeping system responsive

A possible downside of building large programs with make -j is that the computer can become non-responsive to input including mouse and keyboard. A symptom of this is that the Caps Lock keyboard light is very slow to respond. To mitigate the problem automatically, disallow a new thread to start if the system becomes too heavily loaded. This is accomplished with the -l or --max-load option.

Linux system load of 0 to 1 means approximately that no tasks are waiting. So if you want to keep your computer responsible to user input (you can continue using your laptop while big program builds in the background), consider something like -l 1. You can set the -l factor higher to allow make to have increasingly higher priority over other tasks, including user input.

Notes

Instead of writing Makefiles, we use Meson on all projects involving compiled code. Meson comes with an integrated parallel-executing test suite, which works very well with continuous integration systems for automated self test. Continuous integration should be a part of all software projects of all sizes, from 50 line programs to operating systems. We recommend Azure Pipelines or GitHub Actions for CI in general.

  • GNU Make parallel docs

Cmake fundamentals

Gfortran -static avoids missing DLL errors

Compilers typically have an option to specify static linking of the compiler libraries. This can help mitigate “missing libgfortran” errors, particularly on Windows. This problem arises because the developer may switch compiler environments, or the end user may not have compiler libraries with a compatible ABI installed or on PATH / LD_LIBRARY_PATH. Build environment switching is especially common on Windows, and relatively few Windows users have compilers installed, so missing DLLs have long been a hallmark of distributing Windows programs in various programming languages.

GCC / Gfortran use the -static command line option to statically link the compiler libraries e.g. libgfortran into the final executable. This increases the size of the executable, but the extra space may be negligible compared to the nuisances avoided.

gfortran -static myprog.f90 mylib.f90 -o myprog.exe

Other compilers

clang -static
flang -static-flang-libs
ifort -static-intel

Caveats

  • We don’t universally apply static linking because some libraries may only be available as dynamic.
  • MacOS static linked executable may only work on your specific computer, it’s not intended for distribution. MacOS prefers dynamic linking.