Scientific Computing

Ninja normally deletes each response .rsp file if the target build is successful. When debugging a program, we may need to see all the commands run before failure by keeping the .rsp files.

Ninja keeps the .rsp files after compilation by using option:

ninja -d keeprsp

Ninja is a build backend used by Meson build system and CMake as a fast, modern replacement for GNU Make. .rsp files are simply a plain text file containing the command fragment to be run on the command line.

For example, a build line:

cc -Iinc hello.c -lm

is implemented with an .rsp file by the build system as:

cc hello.rsp

where file foo.rsp contains

-Iinc hello.c -lm

There is currently not a Meson or CMake option to keep the .rsp files. Instead, manually invoke Ninja as above when Meson or CMake has configured the build. That is, instead of:

cmake --build build --target hello -v

or

meson compile -C build

do:

ninja -C build -d keeprsp hello

Sometimes a pull request/merge request needs changes before it can be merged. On GitHub, a typical workflow is like:

git switch -c feature main
git pull https://github.com/friend/repo.git main
# other person's repo branch PR/MR is from

Then one edits the files, and commits the changes. However, if the Git pull command fails like:

fatal: Not possible to fast-forward, aborting.

From the new feature branch:

git pull https://github.com/friend/repo.git main
git reset FETCH_HEAD --hard

Then one edits the files, and commits the changes.

Finally, the generic workflow to merge the PR/MR is:

git switch main
git merge --no-ff feature
git push

If the CI-detected change is git commit --amend or squash and force push over previous commits, the CI run may fail with a message like:

fatal: reference is not a tree: <commit id>
The command "git checkout -qf <commit id>" failed and exited with 128 during .

Different CI systems have distinct default Git depth. Checking out all commits takes too much time, while too shallow can falsely fail on force push.

We would suggest not specifying the Git clone depth, thereby using the CI system default, unless the repo is very large and Git clone is taking too long. If manually specifying Git depth, the depth must be larger than the number of Git commits your team would ever squash and force push (overwriting prior commits).

For example, in GitHub Actions:

- uses: actions/checkout
  with:
    fetch-depth: 5

Shell scripts can stop upon exception rather than handling each one manually. This can significanlty reduce logical clutter in a script. Stop a shell script on error for Unix / Linux / macOS shell by setting near the top of the .sh script file:

set -o errexit

This is a human-readable equivalent to set -e This works for commonly used Unix shells including Bash and Zsh.

Another useful option is to stop the script if any script variables are defined:

set -o nounset

Stop executing a Powershell script upon exception by adding near the top of the .ps1 Powershell script:

$ErrorActionPreference = 'Stop'

Downloading binary Homebrew bottles without installing Homebrew can be useful to check the location of the bottle contents. This is useful when developing Meson native-files or CMake Find*.cmake modules.

Homebrew distributes bottles from GitHub Packages. For example, HDF5 binary “bottle” may be inspected by:

tar --list -f <filename>

No Homebrew install is necessary for inspection. Using the libraries and binaries is best done by installing Homebrew.

Graphics and sound generally work via X Windows or Wayland in Windows Subsystem for Linux WSL. Networked SDRs (connected to computer via Ethernet) and simulations can work with GNU Radio and GNU Radio Companion on WSL. USB (non-networked) SDRs may or may not work–need an SDR hardware-specific USB to network driver from Windows and additional configuration at best.

Install and run GNU Radio Companion:

apt install gnuradio

gnuradio-companion

Troubleshooting

“ValueError: Namespace Gtk not available” can be fixed by:

apt install python3-gi gobject-introspection gir1.2-gtk-3.0

Despite apt install python3-pyqt5 this may help:

python3 -m pip install pyqt5

GNU radio ImportError fixes

If GNU Octave plot fonts are too small or the lines are too thin in GNU Octave plotting, typically the first of these methods will be adequate, the others are for reference.

Octave has multiple graphics “toolkits” or “backends”, and prefers QT. If you don’t want the graphical IDE, start Octave with octave -no-gui instead of octave-cli. Using octave-cli disables the QT backend.

Default plot settings are for qt backend. Avoid setting font sizes in the program itself. Tweaking plots for a aparticular computer display may reduce plot fidelity on other computers with different displays.

The “correct” way to scale plot fonts is thus to change your system defaults. Add this to ~/.octaverc instead of ~/Documents/MATLAB/startup.m so that you don’t disturb Matlab’s plotting defaults.

set(0, "defaulttextfontsize", 24)  % title
set(0, "defaultaxesfontsize", 16)  % axes labels

set(0, "defaulttextfontname", "Courier")
set(0, "defaultaxesfontname", "Courier")

set(0, "defaultlinelinewidth", 2)

adjust 16 to produce the most appealing text labels in:

• axes tick labels
• legend key
• title text

defaultline is the root category for lines, so defaultlinelinewidth is not a typo.

The alternative methods below are not normally needed, but are for reference. PPI adjustments: find your PPI by Internet search or spec sheet. Octave’s PPI estimate is:

get(0, 'screenpixelsperinch')

If Octave’s PPI estimate is too small, this is probably why your plot text is too small–Octave thinks your resolution is much less than it really is.

If still a font size problem, try changing system DPI scaling. On Ubuntu, try Gnome Tweak Tool → Fonts → Scaling Factor Octave GUI settings:

• → General → Interface → Icon Size: large
• → Editor Styles → Octave: default
• → Terminal → Font Size

You can also try changing the graphics toolkit. Usually Qt is the best and most modern.

Octave graphics toolkits available:

available_graphics_toolkits()

Active graphics toolkit:

graphics_toolkit()

Select graphics toolkit to see if font sizes are better. Be sure to open a new figure when trying different graphics toolkits.

Reference: GNU Octave default settings docs

Related: GNU Octave set defaults

Screen is a terminal multiplexer program. Screen allows programs to continue running after a remote user disconnects. If a remote connection is lost unintentionally, screen may not allow reconnection by default by the usual

screen -list

screen -r <id>

normally allows reconnecting to a remote session after logging off. When a connection is lost before disconnecting from screen, you may need the “-x” option:

screen -x <id>

A downside of screen is the difficulty scrolling back in history.

Screen is a terminal multiplexer, and some prefer tmux over screen. Another option is using nohup.

Multiple subprojects can be invoked from a top-level superprojects with build systems such as CMake or Meson:

• CMake FetchContent
• CMake ExternalProject
• Meson subproject

CMake FetchContent and ExternalProject can download subprojects, or the subproject can be included in the top-level project via Git submodule or monorepo approach. This subproject hierarchy works like any other CMake project from the command line or IDEs like Visual Studio.

Meson subproject and CMake ExternalProject keep project namespaces separate. Meson subproject and CMake FetchContent download and configure all projects at configure time. CMake FetchContent comingles the CMake project namespaces. FetchContent can be easier to use than ExternalProject if you control both software projects’ CMake scripts. If you don’t control the “child” project, it may be better to use ExternalProject instead of FetchContent.

For these examples, suppose we have a top-level project “parent” and a “child” project containing a library that is desired in parent. Suppose the child project can be built standalone (by itself) but also may be used directly from other CMake projects.

FetchContent populates content from the other project at configure time. FetchContent populates the “child” project with default values from the “parent” project. Varibles set in the “child” project generally do not affect the “parent” project unless specifically used from the “parent” project. CMAKE_ARGS and CMAKE_CACHE_ARGS have no effect with FetchContent_Declare. To set a value in the child project from the parent project, set variables in the parent project.

From “parent” project CMakeLists.txt:

cmake_minimum_required(VERSION 3.14)
project(parent Fortran)

include(FetchContent)
FetchContent_Declare(child
  URL https://github.invalid/username/archive.tar.bz2
)
# it's much better to use a specific Git revision or Git tag for reproducibility

FetchContent_MakeAvailable(child)

# your program
add_executable(myprog main.f90)
target_link_libraries(myprog mylib)  # mylib is from "child"

FetchContent_MakeAvailable

make “child” code configure, populating variables and targets as if it were part of “parent” CMake project.

suppose “child” project CMakeLists.txt contains:

project(child LANGUAGES Fortran)

include(GNUInstallDirs)

add_library(mylib mylib.f90)

target_include_libraries(mylib INTERFACE ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_INCLUDEDIR})

set_property(TARGET mylib PROPERTY Fortran_MODULE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_INCLUDEDIR})

The child project CMAKE_BINARY_DIR and CMAKE_SOURCE_DIR will be those of parent project. That is, if the parent project is in ~/foo and the build directory is ~/foo/build, then the child project in ~/childcode called by FetchContent will also have CMAKE_SOURCE_DIR of ~/foo and CMAKE_BINARY_DIR of ~/foo/build. So be careful in the child project when using such variables that may be defined by parent projects. This is why projects that aren’t specifically designed to work together may be better joined by ExternalProject. A typical technique within the child project that can operate standalone is to refer to CMAKE_CURRENT_SOURCE_DIR instead of CMAKE_SOURCE_DIR as the latter will break when used from FetchContent.

IMPORTANT: When using if() clauses to determine execution of FetchContent, ensure that the FetchContent stanzas are executed each time CMake is run. Otherwise, the FetchContent targets may fail to be available or may have missing target properties on CMake rebuild.

ExternalProject populates content from the other project at build time. This means the other project’s libraries are not visible until the parent project is built. Since ExternalProject does not combine the project namespaces, ExternalProject may be necessary if you don’t control the other projects.

ExternalProject may not activate without the add_dependencies() statement. Upon cmake --build of the parent project, ExternalProject downloads, configures and builds.

From “parent” project CMakeLists.txt:

project(parent LANGUAGES Fortran)

include(GNUInstallDirs)
include(ExternalProject)

set(mylist "a;b;c")
# passing a list to external project is best done via CMAKE_CACHE_ARGS
# CMAKE_ARGS doesn't work correctly for lists

set_property(DIRECTORY PROPERTY EP_UPDATE_DISCONNECTED true)
# don't repeatedly build ExternalProjects.
# dir prop scope: CMake_current_source_dir and subdirectories

ExternalProject_Add(CHILD
GIT_REPOSITORY https://github.com/scivision/cmake-externalproject
GIT_TAG main
CMAKE_ARGS --install-prefix=${CMAKE_INSTALL_PREFIX}
CMAKE_CACHE_ARGS -Dmyvar:STRING=${mylist}   # need variable type e.g. STRING for this
CONFIGURE_HANDLED_BY_BUILD ON
BUILD_BYPRODUCTS ${CMAKE_INSTALL_FULL_LIBDIR}/${CMAKE_STATIC_LIBRARY_PREFIX}timestwo${CMAKE_STATIC_LIBRARY_SUFFIX}
)

add_library(timestwo STATIC IMPORTED GLOBAL)
set_property(TARGET timestwo PROPERTY IMPORTED_LOCATION ${CMAKE_INSTALL_FULL_LIBDIR}/${CMAKE_STATIC_LIBRARY_PREFIX}timestwo${CMAKE_STATIC_LIBRARY_SUFFIX})
set_property(TARGET timestwo PROPERTY INTERFACE_INCLUDE_DIRECTORIES ${CMAKE_INSTALL_FULL_INCLUDEDIR})

add_executable(test_timestwo test_timestwo.f90)  # your program
add_dependencies(test_timestwo CHILD)  # externalproject won't download without this
target_link_libraries(test_timestwo PRIVATE timestwo)

add_dependencies()

make ExternalProject always update and build first

CONFIGURE_HANDLED_BY_BUILD ON

tells CMake not to reconfigure each build, unless the build system requests configure

BUILD_BYPRODUCTS

necessary for Ninja to avoid “ninja: error: “lib” needed by “target”, missing and no known rule to make it”. Note how we can’t use BINARY_DIR since it’s populated by ExternalProject_Get_Property()

The imported library ext is used in the “parent” project just like any other library.

“child” project CMakeLists.txt includes:

project(child Fortran)

include(GNUInstallDirs)

add_library(timestwo STATIC timestwo.f90)
set_property(TARGET timestwo PROPERTY Fortran_MODULE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_INCLUDEDIR})

Configure “child” Fortran_MODULE_DIRECTORY so that it’s not necessary for “parent” to introspect “child” directory structure.

We have created live ExternalProject examples:

• https://github.com/scivision/fortran2018-examples submodules directory
• https://github.com/scivision/sparse-fortran/ mumps directory

CMake can detect if a project is “top level” that is, NOT via FetchContent using PROJECT_IS_TOP_LEVEL.

target_link_directories() is generally NOT preferred because library name collisions can occur, particularly with system libraries.

Reference: CMake staff comparison of multiple project with CMake

WSJT-X puts QSO ADIF files in its log directory. Once a Trusted QSL (TSQL) profile is created and authenticated by ARRL LoTW, and the TQSL application is installed, our LoTW upload using TQSL Python script can be used to upload the QSO ADIF files to LoTW via TQSL in a second or two.