Pytest captures stdout/stderr and sets stdin to os.devnull. When a test involves the use of Python subprocess, the “capfd” Pytest fixture is required to capture the subprocess output. “capsys” fixture only captures native Python prints, without capturing subprocess output. This is also true when testing a Python script intended to be used from the command line.

import subprocess
import sys

def test_script(capfd):
    """test that a particular message is printed to terminal"""
    ret = subprocess.run([sys.executable, "-m", "mymod"], text=True)
    assert ret.returncode == 0

    cap = capfd.readouterr()
    assert cap.out.strip() == "success"
    assert cap.err.strip() == ""

Python aiohttp is a go-to library for asynchronous networking operations using “asyncio”. One of the things we use aiohttp for is to replace synchronous “requests” with asynchronous aiohttp operations, to run many concurrent downloads or uploads for example. However, this is a long-standing bug that causes spurious warnings on exiting the asyncio event loop (e.g. at the end of a script or function) like:

“aiohttp” ResourceWarning unclosed socket.socket

A workaround for this per the bug report above is to add a 250 millisecond delay just before closing the event loop, perhaps by:

await asyncio.sleep(0.25)

Asyncio 4.0.0 is the current milestone for this to be fixed.

Matplotlib Release Notes help you judge if it’s worth upgrading Matplotlib. The release candidates might not have release notes posted. Also check the Matplotlib website, putting in the new version number in the URL like: https://matplotlib.org/3.4.0/users/whats_new.html

Upgrade Matplotlib in seconds by:

conda update matplotlib

or using pip:

pip install --upgrade matplotlib

Needless use of administrator privileges during install of a library or program is generally undesirable. On Unix-like systems including Linux, BSD and MacOS, admin privileges are known as superuser, typically invoked for a single command by sudo. Casual use of “sudo” can goof up the operating system itself, create cybersecurity risks, and mess up the desired program installation. The install might do undesirable things like put itself under /root or make itself readable only by root etc.

In general we write procedures without invoking sudo, to avoid careless sudo invocation. That is we assume the user knows when it’s appropriate to invoke sudo. sudo is commonly used with apt install or yum install and other commands installing from repositories.

Help avoid issues by installing without sudo under “~/.local/” as in these examples. Many Linux distros including Ubuntu are preconfigured to use a hierarchy like ~/.local/bin, ~/.local/lib and so on.

For CMake, set the default install location in “CMakeLists.txt”. Users can set their own install location by cmake -DCMAKE_INSTALL_PREFIX=.

# --- default install directory under ~/.local
# users can specify like "cmake -B build -DCMAKE_INSTALL_PREFIX=~/mydir"
if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
  get_filename_component(HOME ~ ABSOLUTE)

  # will not take effect without FORCE
  set(CMAKE_INSTALL_PREFIX "${HOME}/.local" CACHE PATH "Install top-level directory" FORCE)
endif()

Autotools uses the --prefix= option to specify the top-level install location, like:

./configure --prefix=~/.local

When using system Python it is often preferred to use the PEP 370 --user option like:

pip install --user matplotlib

which installs Python modules under ~/.local. Pip ≥ 20 makes pip default to --user when necessary There are some edge cases such as GTK3 where using system Python libraries can be useful, such as python-gi with Beamer.

A big mess can be made when installing with sudo pip – don’t do that in general. Also, when installing Python environments (e.g. Miniconda), which is typically recommended instead of using system Python, don’t use sudo either.

CMake can place a .gitignore file in the build directory to allow making build directories with arbitrary names not cause clutter. Many projects have a project-wide .gitignore containing:

build*/

but some users like to use bin/ etc. instead.

Make this more flexible by putting a .gitignore in the build directory, regardless of name by putting in CMakeLists.txt:

# --- auto-ignore build directory
if(NOT EXISTS ${PROJECT_BINARY_DIR}/.gitignore)
  file(WRITE ${PROJECT_BINARY_DIR}/.gitignore "*")
endif()

The Mac Mini M1 is several hundred dollars cheaper than the M1 Macbooks. This and the $599 sale price seem to make the Mac Mini M1 a good value for developers. Homebrew has arm64 binaries. Most programs not specifically for arm64 work through Rosetta with nearly full performance.

Running benchmarks that take about ten minutes, the fan did not come on and the case was still cool to the touch. With 8GB of RAM, some large project require limiting build parallelism to avoid running out of RAM on build.

The M1 Mac mini has a modestly adequate internal speaker suitable for quiet offices. It was disappointing that with MacOS 11.2, the volume was not controllable in MacOS itself with HDMI audio on the same monitor that volume control worked with Raspberry Pi–to be clear, this is an OS setting, not adjusting the monitor itself. It’s also disappointing that a microphone isn’t built in–this would have been useful for Siri at least. I suppose Apple may have felt people would try to use an internal microphone for conferencing, and then get disappointed if they weren’t close enough to it for good sound.

As compared to a $50/month cloud Mac service, the Mac Mini M1 pays for itself in a year, while adding value as a media center and convenience of having a local physical Mac. A a developer, I didn’t feel the M1 laptops were a good value for my use cases, but others will of course enjoy them.

Compilers typically have an option to specify static linking of the compiler libraries. This can help mitigate “missing libgfortran” errors, particularly on Windows. This problem arises because the developer may switch compiler environments, or the end user may not have compiler libraries with a compatible ABI on PATH (Windows), LD_LIBRARY_PATH (Linux), LIBRARY_PATH (MacOS).

Build environment switching is especially common on Windows. Relatively few Windows users have compilers installed. Missing DLLs have long been a hallmark of distributing Windows programs in various programming languages. A typical error code is -1073741515 corresponding to hex error code 135.

GCC / Gfortran can statically link the compiler libraries e.g. libgfortran into the final executable. This increases the size of the executable, but the extra space may be negligible compared to the nuisances avoided.

gfortran -static -static-libgfortran myprog.f90 mylib.f90 -o myprog.exe

We often implement GCC static executable linking in CMake like:

Other compilers have similar static compiler library link options:

clang -static

flang -static-flang-libs

ifort -static-intel

We don’t universally apply static linking because some libraries may only be available as dynamic. MacOS static linked executable may only work on your specific computer; MacOS prefers dynamic linking. This technique does not guarantee portable executables, which are a more general problem. Using a slim libc like musl can be part of the solution for portable static executables.

The venerable and ubiquitous Zlib compression library fell into non-development several years ago. CMake build deprecation warnings have been emitting that will soon become errors. Not to fear, a new next generation Zlib has been under development, including by original Zlib developers. Keep an eye on zlib-ng as it prepares to take over for the legacy zlib.

CMake can generate pkg-config .pc files for packages. The .pc file serves as a Rosetta stone for many build systems. While we normally use the .cmake files for our packages, we also include the .pc files for non-CMake users. Meson has a built-in function for generating pkg-config files, but CMake does not yet.

A good basic reference for pkg-config .pc syntax is helpful. We use a my_package.pc.in template with contents generated by CMake configure_file() and associated variables.

CMake “find_{file,library,package,path,program}” have a NO_DEFAULT_PATH parameter that disables ALL search paths, even the CMake >= 3.12 <NAME>_ROOT priority. The need for this and the workaround is best shown by example with a non-system compiler such as Intel oneAPI. The reason that we use NO_DEFAULT_PATH for non-system compilers is because CMake will still try to use the system libraries that may not be ABI compatible with the other compiler. NO_DEFAULT_PATH disables the CMake default PATH_SUFFIXES, so those need to be specified as well.

To make the command line and environment variable hints work again, do like:

find_library(ZLIB_LIBRARY
  NAMES z zlib
  NO_DEFAULT_PATH
  HINTS ${ZLIB_ROOT} ENV ZLIB_ROOT
  PATH_SUFFIXES lib)

find_path(ZLIB_INCLUDE_DIR
  NAMES zlib.h
  NO_DEFAULT_PATH
  HINTS ${ZLIB_ROOT} ENV ZLIB_ROOT
  PATH_SUFFIXES include)