Scientific Computing

Measuring controller / joystick input lag to a computer is not as trivial as one might think. We encourage at least skimming the references below, especially the Lagmeter readme to get background. In short, take a video camera or smartphone capable of 60 frames/second or faster and use an LED wired to the joystick input and video it and the computer display while pressing a button. Video display lag (put the monitor in “game mode” if available) and USB polling cadence are two of several significant factors in input lag. In difficult absolute measurement scenarios such as this, where so many confounding factors are present, a more useful metric may be relative delay measurements, for example wired vs. wireless controllers.

References:

• Display Lag video game database
• Lagmeter using Arduino with detailed discussion of lag factors
• Activision discussion of numerous latency, lag, stability tradeoffs in modern pipelined parallel game design

curl is a universal command line tool for network data transfer. Build curl from source on Windows including native TLS support with Visual Studio and CMake. No external libraries are required for SSL using curl native Windows Schannel. The build is done “static” to avoid the hassle of DLL path.

cmake -DCURL_USE_SCHANNEL=on -DBUILD_SHARED_LIBS=off -DCMAKE_C_COMPILER=cl -Bbuild

cmake --build build --config Release

Test the standalone “curl.exe” like:

build/src/Release/curl.exe -L https://hsts.badssl.com/

We helped discover a curl bug when using native Schannel TLS 1.3 support on Windows for curl 7.85 and 7.86. The symptoms include that downloads may fail on Windows 11 with TLS 1.3 enabled on HTTPS or other TLS 1.3 server downloads. The bug is revealed when building curl on Windows using Schannel. We first observed this in CMake 3.25.0-rc3,rc4 that use vendored curl 7.86.

rclone seamlessly connects to remote cloud storage for cloud storage services including Google Shared Drive, Dropbox, Microsoft OneDrive and many more providers. The rclone CLI is easier than rsync. rclone works with encrypted filesystems.

Setup Rclone:

• Linux: Download and extract–for PC, use amd64. Add to “~/.bash_aliases”:

  alias rclone='~/rclone/rclone'

• macOS / Homebrew: brew install rclone

• Windows: Download, extract, install

Add various remote file shares (Google Drive, Dropbox, etc.) by:

rclone config

In these examples, remote is the particular drive name chosen during rclone config.

Configuration is stored in ~/.config/rclone/rclone.conf.

List remote directories:

rclone lsd remote:

List remote directories under path/to

rclone lsd remote:path/to

List remote files under path/to

rclone ls remote:path/to

Count number of files in path/to

rclone ls remote:path/to | wc -l

copy in rclone by default does not clobber existing remote files, if they haven’t been changed.

Recursively copy local computer ~/path to remote

rclone copy ~/path remote:path -v

Copy only this directory contents ~/path to remote

rclone copy --max-depth 0  ~/path remote:path -v

Using rclone sync requires caution, because sync DELETES files on remote that are not also on local PC!

Sync local computer ~/path to the remote

rclone sync ~/path remote:path -v

If too much swap space is filled, the system could become unusable until force-rebooted due to very slow hard drive (or SSD) I/O relative to RAM. It can be better to have the application terminated automatically due to running out of RAM instead of having the system become inaccessible until force-rebooted. There are disadvantages for no swap file.

If not using hibernation, 100 MB swap size is a general starting point for laptop, desktop and embedded systems. Consider the specific situation and test before deployment to avoid unexpected impacts. Setting swapfile size is only for systems using a swapfile. Systems using a swap partition require additional knowledge.

For systems using Dphys-swapfile such as Raspberry Pi, check swap status:

systemctl status dphys-swapfile

Persistently disable swap by:

systemctl stop dphys-swapfile
systemctl disable dphys-swapfile

The tilde “~” character is used by most terminal shells as shorthand for the user home directory. The home directory is typically a safer place to write files than the system root directory, so ~ give a convenient way to refer to an absolute path generic across systems. Many code languages require parsing the tilde, including Python, C++, Fortran and Matlab. GNU Octave understands that ~ tilde is the user’s home directory on any operating system, even Windows. Matlab does not consistently understand ~ as the user home directory, particularly on Windows.

The stdlib for Matlab has a function stdlib.expanduser() that expands the tilde character ~ to the user home directory on all operating systems, including Windows.

stdlib.expanduser('~/Downloads/foo')

Python paths starting with ~ need pathlib.Path(’~/mypath’).expanduser() method to expand the ~ into the user home directory.

import pathlib
pathlib.Path('~/Downloads/foo').expanduser()

Related: CMake expanduser

Matlab CI services include Azure Pipelines. Matlab Azure Pipelines CI automatically tests Matlab code on each “git push” as with other coding languages. The azure-pipelines.yml file in the Git repo controls CI behavior.

GitHub Actions Matlab CI is often easier to use than Azure in general and runs atop Azure.

This example Matlab azure-pipelines.yml will test a Matlab project on Azure Pipelines CI.

Matlab system() discards the shell environment variables. Set environment variables in name-value pairs like:

if ispc
  system("echo %test_var%", test_var="hello")
else
  system("echo $test_var", test_var="hello")
end

Multiple environment variables can be set in a single command by adding more name-value pairs.

To set specific environment variables desired to set in the system command, do like this example for “CC”:

system("make", CC=getenv("CC"))

To set multiple environment variables:

env = namedargs2cell(struct(CC="gcc", CXX="g++"));

system("make", env{:});

For more advanced Python-like subprocess control from Matlab, use matlab-stdlib subprocess_run() that allows setting environment variables, cwd, stdin pipe, timeout, and more.

Environment variables in CMake set outside of CMake before a CMake process is started have global scope in CMake. Setting environment variables in a CMake script don’t have visibility outside the current CMake process. That includes CMake ExternalProject, which will NOT see environment variables set(ENV{name}) in the parent CMake project. Also, build and test processes will NOT see environment variables set in the CMake script.

To make environment variables take effect in a CMake script or CMake configure, set the environment variable before running CMake, or on a Unix-like OS on the command line. For example, to tell CMake the HDF5 library location, set environment variable HDF5_ROOT before the first CMake configure command:

# Unix-like
HDF5_ROOT=~/mylib cmake -Bbuild

# Windows PowerShell
$env:HDF5_ROOT="~/mylib"
cmake -B build

Of course, one could also set the CMake variable at configure like:

cmake -Bbuild -DHDF5_ROOT=~/mylib

To make environment variables present during a CMake build, for example for an ExternalProject, do as above for the configure.

To control environment variables present for CTest tests, including for individual tests, set test properties.

For modern Fortran development, “.f90” is the recommended filename suffix. For legacy Fortran 77 code, “.f” is the recommended filename suffix.

To preprocess Fortran code files (e.g. with #ifdef statements), capitalize the filename suffix e.g. “.F90” Failing to capitalize the suffix of code that requires preprocessing can break build systems such as CMake and Meson. The introspection of Fortran code by CMake or Meson may make an incorrect build graph if the preprocessing is not done. This can show up as random (or deterministic) build failures, whether in serial or parallel build.

We use the first “free-form” syntax Fortran 90 standard “.f90” suffix to indicate the current Fortran standard. This helps avoid too many ambiguous file suffixes.

C and C++ likewise do not indicate their language standard year in the source code file suffix.