Scientific Computing

Git submodules can switch remotes. This is useful when making a pull request for a project that relies on another pull request submodule. Verify the change in the top project’s “.gitmodules” file.

Example: suppose the directory “subdir” is a Git submodule. In this command, do not put “./subdir” or “subdir/”, just “subdir” by itself. Suppose you also wish to change the branch to “feat1” in the submodule.

git submodule set-url -- subdir https://github.invalid/username/new

git submodule sync

git -C subdir pull

git -C subdir switch feat1

It’s straightforward to detect if running inside WSL or if WSL is available.

These functions use only Python standard library modules. Other languages are demonstrated as well for checking WSL.

For certain use cases, it’s feasible to run a Bash script from within Windows using Windows Subsystem for Linux (WSL). Another way to run Bash scripts from within Windows itself without WSL is the Bash shell installed with Git on Windows.

Start the Bash script you want to run from Linux or Windows with the shebang (first line of Bash script file):

#!/bin/bash; C:/Program\ Files/Git/git-bash.exe

This tells the shell (Linux or Windows) to first try /bin/bash which is a Unix-like path, and then try the Git Bash shell on Windows. If Python is on Windows Path, one can use Bash scripts that invoke Python scripts.

CMake outputs default compiler flags based on platform and project configuration, which can be overridden. This example shows how to override the default compiler flags by putting the user flags later in the command line

CMake allows hinting of package / library location through CMake variable ${PACKAGE_NAME}_ROOT or environment variable ${PACKAGE_NAME}_ROOT. For example, to hint OpenMP location, set environment variable OPENMP_ROOT to the location of OpenMP as in this standalone OpenMP CMake example.

Note that macOS needs a hint to the location of OpenMP in the example.

CMake FetchContent and ExternalProject bring remote projects into the top-level project to avoid making a monorepo or vendoring code into the top-level project. With FetchContent, the source code is retrieved at CMake configure time, allowing one to ignore the subproject build system and / or use only specific source files.

An example of this is using nRF5 SDK, which is a large project, but one may only wish to use a single source file and header as in this example:

In many data analyses we may generate a large number of plots saved to disk. For convenience of sharing these plots, we have created a Python script that collects all images in a directory into a single HTML document that can be exported to PDF via the web browser “save as PDF” function.

When reading HDF5 files, the HDF5 file version bounds chosen by the writing program must be no newer than the reading program’s HDF5 library version. Usually this is not a problem unless using a very old program to read HDF5 files. When necessary to write older version HDF5 files, select the HDF5 file version from the writing program API. At the time of writing, there is no way to introspect the HDF5 library version required to read a specific HDF5 file. The only way to know is to try to read data from the HDF5 file and see if it works.

Python h5py: use libver= when necessary:

with h5py.File(libver='earliest') as f:
   ....

Matlab low-level HDF5 interface H5P.set_libver_bounds() set HDF5 file write library version bounds.

C / C++ / Fortran HDF5 library can set the HDF5 library version bounds by H5Fset_libver_bounds() and H5Pset_libver_bounds().

Related: Check HDF5 files for corruption

When troubleshooting linking a library or object file, it’s useful to search the symbols defined in the file. The nm program lists the symbols in library and object files. Search for a symbol (e.g. function) by piping nm output to grep. For this example the ncurses library is used.

For an object or static library:

nm /usr/lib/libncurses.a | grep myfunc

For a shared library, the particular version suffix might be needed:

nm -D /usr/lib/libncurses.so.6 | grep myfunc

When the line starts with “T”, the symbol is defined in the library. When the line starts with “U”, the symbol is undefined in the library. Note that header defines, such as used in PDCurses are not listed by nm. That is, PDCurses curses.h is not detected by nm.

#define getch() wgetch(stdscr)

For Fortran, since the “.h” files are not used, the developer needs to provide an interface mapping the functions in Fortran. Note that the developer must use symbol detection code in the build system (e.g. CMake) to ensure the symbol is only defined once. This is important for a library like Curses that has multiple independent implementation. Specifically, the Ncurses library defines “getch” in the .c source, but PDCurses defines “getch” in the .h header.

The Blocktran project shows how to handle this situation with getch() macro.

pkgconf is a modern drop-in replacement for pkg-config. Build systems like CMake, Meson, and Autotools all support pkgconf to help them find dependencies if the package has pkgconf files.

Installing pkgconf:

• Linux: apt install pkgconf
• macOS: brew install pkgconf