Scientific Computing

Pytest stdout/stderr capture

Pytest captures stdout/stderr and sets stdin to os.devnull. When a test involves the use of Python subprocess, the “capfd” Pytest fixture is required to capture the subprocess output. “capsys” fixture only captures native Python prints, without capturing subprocess output. This is also true when testing a Python script intended to be used from the command line.

import subprocess
import sys

def test_script(capfd):
    """test that a particular message is printed to terminal"""
    ret = subprocess.run([sys.executable, "-m", "mymod"], text=True)
    assert ret.returncode == 0

    cap = capfd.readouterr()
    assert cap.out.strip() == "success"
    assert cap.err.strip() == ""

Mac mini Apple Silicon value

The Apple Silicon Mac Mini is several hundred dollars cheaper than Apple Silicon MacBooks, making the Mac Mini Apple Silicon a good value for developers. Homebrew has arm64 binaries. Most programs not specifically for arm64 work through Rosetta with nearly full performance.

Running benchmarks that take about ten minutes, the fan did not come on and the case was still cool to the touch. If building large projects, consider the amount of RAM to avoid needing to limit build parallelism to avoid running out of RAM on build.

The Mac mini has a modestly adequate internal speaker suitable for quiet offices. With macOS, the volume was not controllable in macOS itself with HDMI audio on the same monitor that volume control worked with Raspberry Pi–to be clear, this is an OS setting, not adjusting the monitor itself. It’s also disappointing that a microphone isn’t built in–this would have been useful for Siri at least. I suppose Apple may have felt people would try to use an internal microphone for conferencing, and then get disappointed if they weren’t close enough to it for good sound.

As compared to a cloud physical Mac service, the Mac Mini Apple Silicon pays for itself in a year, while adding value as a media center and convenience of having a local physical Mac.

CMake generate pkg-config .pc

CMake can generate pkg-config .pc files for packages. The .pc file can be used by many build systems. While we normally use the .cmake files for packages, we also include the .pc files for non-CMake users. Meson has a built-in function for generating pkg-config files, but CMake does not yet.

A good basic reference for pkg-config .pc syntax is helpful. We use a my_package.pc.in template with contents generated by CMake configure_file() and associated variables.

CMake NO_DEFAULT_PATH also disables hints and path suffixes

CMake “find_{file,library,package,path,program}” have a NO_DEFAULT_PATH parameter that disables ALL search paths, even the <NAME>_ROOT priority. The need for this and the workaround is best shown by example with a non-system compiler such as Intel oneAPI. The reason that we use NO_DEFAULT_PATH for non-system compilers is because CMake will still try to use the system libraries that may not be ABI compatible with the other compiler. NO_DEFAULT_PATH disables the CMake default PATH_SUFFIXES, so those need to be specified as well.

To make the command line and environment variable hints work again, do like:

find_library(ZLIB_LIBRARY
  NAMES z zlib
  NO_DEFAULT_PATH
  HINTS ${ZLIB_ROOT} ENV ZLIB_ROOT
  PATH_SUFFIXES lib)

find_path(ZLIB_INCLUDE_DIR
  NAMES zlib.h
  NO_DEFAULT_PATH
  HINTS ${ZLIB_ROOT} ENV ZLIB_ROOT
  PATH_SUFFIXES include)

Fortran generic interface procedure with optional arguments

Fortran 2018 added syntax useful for polymorphism including select rank, which allows rank polymorphism and select type, which allows class polymorphism. For reference, GCC ≥ 7 has select type and GCC ≥ 10 has select rank. Intel oneAPI supports both.

To be compatible with older compilers as well as handle cases where it’s simply more concise to use Fortran 2003 generic procedure interfaces, each procedure must be unambiguously distinguishable. Several criteria make a procedure distinguishable. Here we focus on having at least one non-optional argument that is TKR-distinct (Type, Kind, Rank).

Notice that this example has the first variable non-optional to make the procedures TKR-distinct.

module mod1

implicit none (type, external)

interface manyranks
  procedure m0,m1
end interface manyranks

private
public :: manyranks

contains

subroutine m0(val1, val2)
real, intent(in) :: val1
real, intent(in), optional :: val2
!! omitted code
end subroutine m0

subroutine m1(val1, val2)
real, intent(in) :: val1(:)
real, intent(in), optional :: val2(:)
!! omitted code
end subroutine m1

end module mod1

The code will compile as above. If you add optional to both “val1”, the compilation will fail like:

  • GCC: Ambiguous interfaces in generic interface 'manyranks' for 'm0' at (1) and 'm1' at (2)
  • Intel: #5286: Ambiguous generic interface MANYRANKS: previously declared specific procedure M0 is not distinguishable from this declaration. [M1]

Although this example used rank polymorphism, the same issue arises when using any of type, kind or rank (TKR) generic procedures–there must be an unambiguous resolution with at least one non-optional argument. This polymorphism is implemented at runtime, and so there is no guarantee of non-ambiguity when all arguments are optional.

Get list of CMake test names

CMake add_test() can be dynamically set by arbitrarily complex foreach(), if(), etc. logic. A subset of tests can be configured dynamically after the tests are declared. A list of tests enabled is retrieved by the TESTS directory property. The variable “test_names” contains all the test names previously added in the CMakeLists.txt via “add_test()” in a list in the DIRECTORY scope.

get_property(test_names DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY TESTS)

set_property(TEST ${test_names} PROPERTY TIMEOUT 60)

get_property(test_names GLOBAL PROPERTY TESTS) will return an empty list–DIRECTORY scope must be used.

CMake packaging with CPack example

CMake has a corresponding meta package system CPack, which generates configuration files for numerous packaging systems. Distributing code and binary executablees and libraries to users without requiring them to compile a project is done via these packages. CPack creates these binary packages like Windows .msi, Linux .deb/.rpm, macOS .dmg, etc. CPack also creates traditional source archives as are also generated by GitHub Releases, but with fine-grained control of the contents.

Assuming the PROJECT_BINARY_DIR is “build”, CPack generates build/CPackConfig.cmake for binary packages and build/CPackSourceConfig.cmake for source packages. CPackConfig.cmake is generated according to install() commands in the CMakeLists.txt files of the project.

Note that in general “install()” DESTINATION should always use relative paths. CPack ignores install() items with absolute DESTINATION.

CPackSourceConfig.cmake works the opposite way–it includes everything not excluded by CPACK_SOURCE_IGNORE_FILES, so we make a file cmake/.cpack_ignore with regex excluding non-source files. As a last step at the end of the main CMakeLists.txt after all install(), we include cmake/cpack.cmake:

As usual:

cmake -B build
cmake --build build

The distribution packaged .zip / .tar.gz files under build/package are generated by:

cpack --config build/CPackSourceConfig.cmake

cpack --config build/CPackConfig.cmake

These can be built by the CI system and uploaded for distribution on GitHub Releases, etc. by configuring the .github/workflows/ci.yml accordingly.

Check last error code across OS

In general, programs don’t usually print to console the integer return code from the main procedure. The program may well print some message indicating success or failure, but maybe not. When calling executables from a compiled or scripted language such as Fortran, C, Python or Matlab, it’s often vital to know the value of the integer return code as a signal that the program thought it was successful or not. Further, some program crashes do not emit any console text, and could make the user think the program was successful.

To help eliminate doubt, issue a command to print the last error code to console when working with command line programs. The method to print this integer code depends on the shell. From Terminal / Command Prompt, the return code from the last command is printed by:

  • Unix-like shell: echo $?
  • Windows command prompt: echo %errorlevel%
  • PowerShell: echo $lastexitcode