Scientific Computing

Detect executable file arch

For the simplest execution and best performance, an executable file binary architecture should generally correspond to the CPU architecture of the operating system it runs on. The late 1990s and early 2000s saw a proliferation of CPU architectures including SPARC, PowerPC, DEC Alpha, MIPS, Itanium, and x86. From the late 2000s through about 2020, the dominant CPU architecture for personal computers was x86_64. The 2020s saw the rise of ARM64 across operating systems including Windows on ARM, Apple Silicon, and Linux.

Given the development time and maintenance burdern of supporting multiple CPU architectures, there are in general situations where a user needs to run an executable file on a different CPU architecture than the executable file was built for. It’s a nice practice for program developers to print their native CPU architecture in their program’s about dialog to help users be aware of the CPU architecture of their executable files such that they can seek a native executable if available. For example, a 32-bit x86 executable might run on a 64-bit x86_64 operating system (via API thunking and environment emulation like WoW64), but a 64-bit x86_64 executable generally cannot natively run on a 32-bit x86 operating system.

With some performance overhead and compatibility limitations, executable JIT translation like macOS Rosetta 2, Windows Prism emulation, or Linux FEX emulation can allow executables of one architecture to run on a different CPU architecture. The operating system level translation or emulation development is typically a large investment in software development and optimization, and is not universally available for all architecture combinations. Apple Rosetta 2 translates x86-64 instructions to ARM64 on Apple Silicon, allowing x86-64 executables to run on ARM64 macOS. On Windows, Prism emulation generally allows x86-64 executables to run on ARM64 Windows. On Linux, FEX emulation allows x86-64 executables to run on ARM64 Linux with real-time API call forwarding.

The LIEF library can be used to detect the CPU architecture of Windows executable files from Python.

Meld Git difftool / mergetool

Git users often use Meld to graphically resolve 2-way differences and 3-way merges.

Configure Git to use Meld:

git config --global diff.tool meld
git config --global merge.tool meld

On Windows, Meld can be installed by:

winget install -e --id Meld.Meld

Troubleshooting

If Meld was installed manually and Git can’t find Meld, configure Git to use Meld by setting the path to Meld.exe like:

git config --global mergetool.meld.path "$Env:ProgramFiles\\meld\\Meld.exe"

On Windows if also using MSYS2, don’t add Meld.exe to environment variable Path as it has libstdc++.dll that conflicts with MSYS2 G++. The symptom is the G++-built executable will segfault silently.


Alternatives:

Python for Windows on ARM

Anaconda Python is working toward Windows on ARM support. For now, Anaconda / Miniconda Python work for Windows on ARM via the built-in Prism emulation. To use native ARM64 Python, which could be useful for benchmarking or maximum (best) computing performance, use plain CPython install for ARM64 such as:

winget install Python.Python.3.14

Upon installing and starting, one sees the ARM64 designation in the Python dialogs.

GDL GNU Data Language build

GDL (GNU Data Language) is a free/libre open-source program that runs a good percentage of IDL code. GDL is actively developed and easily installed by:

  • Linux: apt install gnudatalanguage
  • macOS: use weekly gdl-macOS-arm64-standard.dmg. We do this instead of Homebrew because the homebrew/science tap for gnudatalanguage is currently unmaintained.
  • Windows: get the latest release

Building GDL source uses the GDL build script “scripts/build_gdl.sh” to get the prerequisites. If Anaconda Python is present, conda deactivate first to avoid library problems when building GDL.

git clone https://github.com/gnudatalanguage/gdl

cd gdl/

cmake -B build --install-prefix=$HOME/gdl

cmake --build build --parallel

(optional) Check the install. You will see several plots appearing and disappearing automatically during this test, which takes a few minutes.

cmake --test-dir build -V

Install (do not use sudo):

cmake --install build

Do not build on an ExFAT / FAT32 drive, as the build will fail since symbolic links are not allowed on ExFAT / FAT32. If cmake reports libeigen being too old, install LibEigen3 or:

cmake -Bbuild -DEIGEN3=OFF

To use the Linux distro’s older version of GDL, just use /usr/local/bin/gdl or similar, or rename ~/.local/bin/gdl to ~/.local/bin/gdl0.98 or similar.

Troubleshooting build:

  • Runtime search path conflicts: temporarily comment out those paths in ~/.profile (typically from Anaconda Python, libreadline, libhistory, libz, libjpeg.so)
  • Problems with LZMA, try disabling HDF5: cmake -DHDF5=OFF

Clear Pacman database lock

If upon attempting Pacman operations a failure occurs like:

failed to synchronize all databases (unable to lock database)

This may occur if the system was interrupted during a Pacman operation, leaving a lock file that prevents further package management operations. The lock file is located by:

$(pacman-conf DBPath)/db.lck

which is typically “/var/lib/pacman/db.lck”. Check no other Pacman process is running:

ps -ef | grep pacman

Then the Pacman lock file can be removed:

rm $(pacman-conf DBPath)/db.lck

GitHub outage workaround with SSH instead of HTTPS

Anecdotally we have observed that during GitHub outages, Git over SSH operations may have a better chance of succeeding than Git over HTTPS operations. This includes cloning repositories.

Rather than reconfiguring .gitconfig to use SSH, simply clone using the SSH URL instead of the HTTPS URL.

For example, instead of:

git clone https://github.com/user/repo.git

Assuming Git over SSH is setup on the computer:

git clone git@github.com:user/repo.git

CMake VS 2026 on GitHub Actions

The CMake variable CMAKE_GENERATOR can be used with GitHub Actions to specify the Visual Studio 18 2026 generator. Currently, the runner image GitHub Actions runner image windows-2025-vs2026 is used until the “windows-latest” runner image incorporates VS 2026.

jobs:

  msvc:
    runs-on: windows-2025-vs2026

    steps:
    - uses: actions/checkout

    - name: Print CMake version
      run: cmake --version

    - name: Configure CMake
      run: cmake -B build -G "Visual Studio 18 2026"

    # and so on

Recursively clean CMake build directories

CMake build directories might contain 100s of megabytes of files for large projects. Over time, a developer computer might contain forgotten build directories that waste disk space. With Python, a script using send2trash allows safe removal of CMake build directories by first moving them to the OS trash/recycle bin. In distinction from shutil.rmtree, this send2trash approach allows recovery of files if the deletion was accidental. The heuristic used to detect a CMake build directory was inspired by ctest_empty_binary_directory.

mpi_f08 Fortran on Windows

use mpi_f08 is recommended for Fortran across computing platforms, including Windows.

For native x86 (Intel / AMD CPU) binaries, currently only free Intel oneAPI has mpi_f08 for Fortran. As time progresses and ARM64 CPUs are increasingly widespread, including for Windows PCs, and the complexity / disk space requirements of setting up Visual Studio for Intel oneAPI on Windows, it may be better (easier, faster, performance) to use WSL for Windows MPI. WSL can use OpenMPI or MPICH to access mpi_f08. For Windows ARM CPU users, WSL is the only straightforward option for mpi_f08 in Fortran.

GitHub Actions with Windows Subsystem for Linux

The setup-wsl GitHub Action configures WSLv2 environment in Windows GitHub Actions runners. This allows testing certain quirks and corner cases one might encounter when running software on Windows Subsystem for Linux. For scientific computing Windows users, WSL is often the best way to run computational software on Windows, including software using performance code for GPU and MPI.