Run Bash scripts from Windows

For certain use cases, it’s feasible to run a Bash script from within Windows using Windows Subsystem for Linux (WSL). Another way to run Bash scripts from within Windows itself without WSL is the Bash shell installed with Git on Windows.

Start the Bash script you want to run from Linux or Windows with the shebang (first line of Bash script file):

#!/bin/bash; C:/Program\ Files/Git/git-bash.exe

This tells the shell (Linux or Windows) to first try /bin/bash which is a Unix-like path, and then try the Git Bash shell on Windows. If Python is on Windows Path, one can use Bash scripts that invoke Python scripts.

CMake override default compiler flags

CMake outputs default compiler flags based on platform and project configuration, which can be overridden. This example shows how to override the default compiler flags by putting the user flags later in the command line

CMake FetchContent ignore build system

CMake FetchContent and ExternalProject bring remote projects into the top-level project to avoid making a monorepo or vendoring code into the top-level project. With FetchContent, the source code is retrieved at CMake configure time, allowing one to ignore the subproject build system and / or use only specific source files.

An example of this is using nRF5 SDK, which is a large project, but one may only wish to use a single source file and header as in this example:

Python collect images to HTML document

In many data analyses we may generate a large number of plots saved to disk. For convenience of sharing these plots, we have created a Python script that collects all images in a directory into a single HTML document that can be exported to PDF via the web browser “save as PDF” function.

Matlab file checksum

Computing file hash checksum allows some verification of file integrity, assuming the file is not maliciously altered to have the same checksum. While MD5 is a fast method to compute checksums, it is not cryptographically secure. That is, someone could create a malicious file with the same MD5 checksum as the desired file. A more secure checksum is SHA-256, which is slower to compute, but is considered adequate at the time of this writing.

We have created a Matlab function file_checksum that uses Java MessageDigest to compute the checksum of a file that is read in chunks to avoid overflowing the Java VM memory.

HDF5 file version

When reading HDF5 files, the HDF5 file version bounds chosen by the writing program must be no newer than the reading program’s HDF5 library version. Usually this is not a problem unless using a very old program to read HDF5 files. When necessary to write older version HDF5 files, select the HDF5 file version from the writing program API.

Python h5py: use libver= when necessary:

with h5py.File(libver='earliest') as f:

Matlab low-level HDF5 interface H5P.set_libver_bounds() set HDF5 file write library version bounds.

C / C++ / Fortran HDF5 library can set the HDF5 library version bounds by H5Fset_libver_bounds().

Related: Check HDF5 files for corruption

List symbols in library / object file

When troubleshooting linking a library or object file, it’s useful to search the symbols defined in the file. The nm program lists the symbols in library and object files. Search for a symbol (e.g. function) by piping nm output to grep. For this example the ncurses library is used.

For an object or static library:

nm /usr/lib/libncurses.a | grep myfunc

For a shared library, the particular version suffix might be needed:

nm -D /usr/lib/ | grep myfunc

When the line starts with “T”, the symbol is defined in the library. When the line starts with “U”, the symbol is undefined in the library. Note that header defines, such as used in PDCurses are not listed by nm. That is, PDCurses curses.h is not detected by nm.

#define getch() wgetch(stdscr)

For Fortran, since the “.h” files are not used, the developer needs to provide an interface mapping the functions in Fortran. Note that the developer must use symbol detection code in the build system (e.g. CMake) to ensure the symbol is only defined once. This is important for a library like Curses that has multiple independent implementation. Specifically, the Ncurses library defines “getch” in the .c source, but PDCurses defines “getch” in the .h header.

The Blocktran project shows how to handle this situation with getch() macro.

pkgconf is generally preferred over pkg-config

pkgconf is a modern drop-in replacement for pkg-config. Build systems like CMake, Meson, and Autotools all support pkgconf to help them find dependencies if the package has pkgconf files.

Installing pkgconf:

  • Linux: apt install pkgconf
  • macOS: brew install pkgconf