Scientific Computing

Python asyncio OSError 24 too many open files

In general operation systems set a limit to the number of open files per process. A “file” might be a regular file, a socket, a pipe, etc. When a Python program exceeds this limit, it raises exception:

OSError: [Errno 24] Too many open files

Such issues might tend to arise with highly concurrent applications as enabled by asyncio or higher-level libraries. Resolving such asyncio concurrency issues generally involves asyncio primitives like asyncio.Semaphore. Implementing asyncio libraries correctly is so non-trivial that using higher-level libraries that implement such concurrency control correctly can be a better idea. It’s important to go beyond toy examples and consider real-world concurrent Python usage patterns.

CMake Position Independent Code

Certain projects with legacy build systems like Make or Autotools or in general may specify to build with flags like “-fPIC” for position independent code (PIC) on Linux systems. Consider not forcing these flags in CMake projects if there isn’t a specific known need, to let users and consuming projects decide whether they need PIC or not. When PIC is needed, do like:

if(...)
  include(CheckPIESupported)

  check_pie_supported()

  set(CMAKE_POSITION_INDEPENDENT_CODE true)
endif()

The check_pie_supported module checks whether the compiler supports PIE and should be run before setting target property POSITION_INDEPENDENT_CODE.

When PIC is enabled with certain platforms like Intel oneAPI on Linux, linker errors may result like

relocation R_X86_64_32 against `.rodata.str1.1’ can not be used when making a PIE object

where the solution is to disable PIC – or simply do not enable it by default in the CMake project as we suggested above.

CMake Two-Step Build Process

Since CMake 3.13 was released in 2018, a more elegant two-step CMake build process is available with the simple syntax

cmake -B ./build

cmake --build ./build

The CMake configure command with the “-B” option generates the build system files in the specified build directory. It’s best to build out-of-source in general, so we specify a separate “./build” directory here.

The cmake –build build command executes the build tool configured for the project, e.g. Make, Ninja, MSBuild, etc.

The “./” can be omitted, we only include it here for clarity. We keep seeing even the largest OEMs using the old-fashioned three-step process. Check your projects’ example and docs to update them to the less-confusing and compact two-step CMake build process.

CMake compiler feature detect

In CMake projects (or other build systems) it’s more robust to detect compiler features rather than make brittle, difficult to maintain nested if / elseif logic trees based on compiler vendor and version. There are edge cases where if() statements to work around known compiler bugs are necessary, but where we can, we use compiler feature checks instead. Each compiler vendor such as AppleClang, GNU GCC, LLVM Clang, typically publishes tables of compiler features vs. language standard support in their documentation. These tables might not cover every aspect of feature support, or the project invocation of that support may trigger bugs in specific compiler versions. Then, either hard-coded build system logic or preferably configure-time feature detection is needed.

An example using check_source_compiles in a CMake project requires C11 variable length arrays. The CMakeLists.txt would look like:

include(CheckSourceCompiles)

set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
# so that check_source_compiles sets the correct language standard

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)
# save link time, only compile is needed

check_source_compiles(C "int main(void){
for (int i = 1; i < 5; i++){
  int a[i];
}
return 0;
}"
c11vla)

if(NOT c11vla)
 # fatal error or disable feature using C11 VLA
endif()

add_executable(c11demo demo.c)

A similar meson.build configure-time compiler feature check example:

project('c11vla_demo', 'c',
  default_options : ['c_std=c11'])

c11_vla = meson.get_compiler('c').compiles(required: true, name: 'c11_vla',
  'int main(void){ for (int i = 1; i < 5; i++){ int a[i]; } return 0; }'
)

VHF marine FM channel spacing

12.5 kHz channel spacing is permitted in certain cases by the FCC in VHF marine radio. Despite experiments demonstrating the practicality of using 12.5 kHz channel spacing on VHF marine radio, currently available radios and practice appear to remain at +/- 5 kHz deviation and 25 kHz channel spacing. Unlike the vocal controversy around 1969 over narrowbanding VHF marine radio from +/- 15 kHz deviation to +/- 5 kHz deviation, there appears to be little controversy over channel spacing.

CMake VS 2026 with GitHub Actions

The CMake variable CMAKE_GENERATOR can be used with GitHub Actions to specify the Visual Studio 18 2026 generator. At the time of writing this article, the Windows GitHub Actions runner image held back to CMake 3.31, so check in case CMake ≥ 4.2 is available in the Windows GA runner image already.

jobs:

  msvc:
    runs-on: windows-latest
    timeout-minutes: 15

    steps:
    - name: Install CMake 4.2
      run: winget install -e --id Kitware.CMake --disable-interactivity --accept-source-agreements --accept-package-agreements
    # FIXME: until GitHub Actions have at least CMake 4.2, necessary for VS 2026

    - name: Print CMake version
      run: cmake --version

    - name: Configure
      run: cmake -B build -G "Visual Studio 18 2026"

    # and so on

Fortran Array Temporary debugging

Fortran array temporaries can be implicitly created when non-contiguous array slices are passed to procedures. This copy of data can impact performance. Older compilers like GCC 8.x (including 8.5), which may be the only installed choice on some HPC systems, have occasional bugs related to array temporaries that can lead to incorrect results or crashes.

It is possible that disabling optimization can change the bug behavior, but this can make the program run vastly slower and it’s just for debugging. Use git bisect or similar techniques to find the commit that introduced the bug. If the bug is introduced by a new / changed function call using an actual array argument that indexes into a non-contiguous array, that is a hint that the problem may be related to array temporaries. Enable compiler flags to help identify array temporaries in GFortran:

Suppose the following code snippet:

x = myfun(A(:, 4:16:3))

Here, A(:, 4:16:3) creates a non-contiguous array slice, leading to the creation of an array temporary when passed to myfun. Manually allocate an contiguous auxiliary array variable and copy the data into it before passing it to the function:

real, allocatable :: temp(:,:)

allocate(temp(size(A, 1), 5))
! Adjust size as needed

temp = A(:, 4:16:3)
! Copy data into contiguous array

x = myfun(temp)
! Pass the contiguous array

deallocate(temp)
! Clean up

Obviously this process is tedious, so we only employ it when needed to workaround a compiler bug.

How to use LaTeX directives

Many programming languages (including LaTeX) have directives or pragma–formatted comments that tells the compiler to do something specific.

LaTeX pragma: de facto directives are compatible with popular LaTeX editors including TeXstudio.

Specify LaTeX compiler and bibliography toolat the top of the main/root LaTeX document like:

% !TeX program = xelatex
% !BIB program = biber

LaTeX projects often use Biber for bibliography management. The above directives tell TeXstudio to use XeLaTeX compiler and Biber bibliography tool when compiling the document.

Trouleshooting missing XeTeX compiler

If XeTeX directive:

% !TeX program used: xelatex

yields:

Error: One command expansion invalid.
  Parent Command: compile
  Primary Command: compile

Fix by installing XeLaTeX compiler

  • Linux: apt install texlive-xetex
  • macOS: brew install mactex
  • Windows: get XeLaTeX via MikTeX

Restart TeXstudio to use XeLaTeX.

List MAC of Bluetooth devices on Windows

Find the MAC address of a Bluetooth device in Windows from Windows Device Manager under Bluetooth, (select device in list), Properties, Details, Bluetooth Device Address.

List all Bluetooth devices and their MAC addresses from PowerShell:

$devices = Get-PnpDevice | Where-Object { $_.Class -eq "Bluetooth" -and $_.FriendlyName }

$results = foreach ($device in $devices) {
    $id = $device.InstanceId
    $name = $device.FriendlyName

    $addressProp = Get-PnpDeviceProperty -InstanceId $id -ErrorAction SilentlyContinue |
                   Where-Object { $_.KeyName -like "*Address*" -and $_.Data -match '^\w{12}$' } |
                   Select-Object -First 1

    if ($addressProp) {
        $raw = $addressProp.Data.ToUpper()
        $bytes = for ($i = 0; $i -lt 12; $i += 2) {
            [convert]::ToByte($raw.Substring($i, 2), 16)
        }
        [array]::Reverse($bytes)
        $mac = ($bytes | ForEach-Object { $_.ToString('X2') }) -join ':'
        [PSCustomObject]@{ Device = $name; 'MAC Address' = $mac }
    }
}

$results | Format-Table -AutoSize

Git Bisect project with Git Submodules

The standard git bisect command help to quickly identify the specific commit that introduced a bug in a Git repository. In projects where branches with lots of commits were previously merged, there may be known-bad commits with build failure unrelated to the current bisect search. It’s possible to skip such commits during the bisect process, or avoid specific commits altogether.

For projects using Git Submodules, an additional command

git submodule update --init --recursive

at each step of the bisect process is required to update the submodules to the correct commit. One can make a simple shell script to use with git bisect run to automate this process, or simply run the command manually after each bisect step.

git bisect start
git bisect <good|bad> <commit>
git submodule update --init --recursive
git bisect run <test command>
git submodule update --init --recursive
# ... repeat as necessary ...
git bisect reset