Scientific Computing

CMake presets allow simple or complex JSON configurations for configuring, building, and testing projects. A project maintainer may with to enforce the use of presets, which may be accomplished by using sentinel environment variable:

Assuming the console shell supports color output, it can be pleasant for users to have color build system (Make) and compiler output. This is easily set by setting environment variable CMAKE_COLOR_DIAGNOSTICS.

Set environment variable CMAKE_COLOR_DIAGNOSTICS=1 to have CMake colorize the build system and compiler output.

CTest by design combines stdout and stderr into stdout for each test. For programs that emit a lot of text to stdout and put only diagnostic content to stderr, the combined text stream can be burdensome when debugging a test failure. There is a proposal to add a CTest property to make optional separate stdout and stderr, but it is not yet implemented as of this writing.

Example

WinGet can be used on Windows GitHub Actions runners where a Windows program needs to be installed. In this example, environment variable FFMPEG_ROOT tells Python where to find the ffmpeg.exe program. One could more generally append to the GITHUB_PATH environment variable.

    - name: Install winget
      if: runner.os == 'Windows'
      uses: Cyberboss/install-winget@v1
      with:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

    - name: install prereqs (Windows)
      if: runner.os == 'Windows'
      run: winget install ffmpeg --disable-interactivity --accept-source-agreements

    - name: FFMPEG_ROOT Windows
      run: echo "FFMPEG_ROOT=$env:LOCALAPPDATA/Microsoft/WinGet/Links/" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
      if: runner.os == 'Windows'

    - name: PyTest
      run: pytest

import functools
import shutil
import os


@functools.cache
def get_exe(name: str) -> str:

    for p in (os.environ.get("FFMPEG_ROOT"), None):
        if exe := shutil.which(name, path=p):
            return exe

    raise FileNotFoundError(name)

Many IDEs create per-project cache directories with metadata relevant to the IDE configuration for that project. Similar to user global .gitattributes, instead of editing the .gitignore file for each repository, ignore directories in Git for all repos a particular user has as follows, for any operating system.

git config --global core.excludesfile ~/.gitignore

Edit ~/.gitignore file to contain items like the following.

*.orig

git merge conflict files. To not generate git merge conflict .orig files:

git config --global mergetool.keepBackup false

.vs/

Visual Studio cache

.vscode/

Visual Studio Code cache

.idea/

PyCharm cache

pip-wheel-metadata/

pip cache

.mypy_cache/

mypy cache

.DS_Store

macOS Finder cache

*.pyc

Python compiled files

*.orig

.vs/
.vscode/
.idea/

CMake can print all cache variables using the cmake -LAH command. This will print all cache variables, including their help text.

cmake -LAH -B build

The -B build option is used to specify the build directory as usual.

cmake -LH omits the advanced variables that may be of less interest to general users.

cmake -L omits the help text.

Assuming the project has already been configured once, the cmake -N option added will load the existing cache without regenerating.

This command does not print non-cache variables, such as variables set in the CMakeLists.txt file or cmake -Dopt flags that are not cached.

Example CMakeLists.txt

project(ex LANGUAGES NONE)

option(my1 "option 1")

Running this command will print only “my1” value and not “my2” value because “my2” is not cached.

cmake -LAH -B build -Dmy1=on -Dmy2=on

On Windows with Python subprocess, an executable may be missing DLL from PATH. This environment error can be detected and a message given to the user.

The return code of the subprocess can be checked for the error code 3221225781, which corresponds to hex error code C0000135.

import subprocess
import os
import logging

ret = subprocess.run('myprog.exe')

if ret.returncode == 3221225781 and os.name == 'nt':
    # Windows 0xc0000135, missing DLL
    logging.error('missing DLL detected. Ensure DLL directory is in environment variable PATH')
    path = os.environ['PATH']
    print("PATH:", path)
    raise SystemExit(ret.returncode)

If putting DLLs on PATH is problematic, another possible approach is statically compiling the executable, but this may not be feasible for all programs.

Most shells (including Bash) don’t have a straightforward way to get the directory of the script being executed. Here’s a common one-liner to get the script directory:

sdir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

However, for some HPC batch systems, the above may not work as it copies this script to a temporary directory. This fallback put after the line above works if “qsub” batch command was run from this script’s directory–supposing a script file “gcc.sh” also exists in the same directory as the original script.

[[ ! -f ${sdir}/gcc.sh ]] && sdir=$PWD

Homebrew Cask packages GUI (graphical) programs. Many users install CMake without the GUI like

brew install cmake

but this does not have the “cmake-gui” program.

To install CMake-GUI do:

brew uninstall cmake

brew install --cask cmake

PROJ.4 / GEOS-based CartoPy downloads and caches shape files as needed, avoiding a large install up front. CartoPy is an alternative to Matplotlib basemap. CartoPy uses easily available / automatically & seamlessly installed prereqs.

conda install cartopy

Note the zorder option of Matplotlib elements such as contourf. Higher number zorder is higher priority (on top). See PlotPrecip.py for an example of zorder. An example using CartoPy follows:

import cartopy
import cartopy.feature as cpf
from matplotlib.pyplot import figure, show
import numpy.random as npr


proj = cartopy.crs.PlateCarree()

fig = figure()
ax = fig.add_subplot(projection=proj)

ax.add_feature(cpf.LAND)  # type: ignore
ax.add_feature(cpf.OCEAN)  # type: ignore
ax.add_feature(cpf.COASTLINE)  # type: ignore
ax.add_feature(cpf.BORDERS, linestyle=':')  # type: ignore
# ax.add_feature(cpf.LAKES,   alpha=0.5)
# ax.add_feature(cpf.RIVERS)

N = 10

lat = (npr.random(N) - 0.5) * 180
lon = (npr.random(N) - 0.5) * 360

ax.scatter(lon, lat, transform=proj)

show()

The #type: ignore is to avoid mypy lint as CartoPy didn’t yet have type hinting support.

CartoPy can even make an auroral oval.