Scientific Computing

Certain platforms default or have defaulted to use a 32-bit time_t, which will overflow in year 2038. To check if the platform uses a 64-bit time_t, use code like this Gist.

Matlab previously made prerelease users sign a non-disclosure agreement (NDA), which hindered users from giving feedback about significant changes or bugs. Developers of Matlab packages were likewise inhibited from updating their packages to work with new syntax or features. There was typically one prerelease and the official release would sometimes have large jumps between the prerelease and added, removed, or changed features. Each official release would have a few updates to fix bugs and add minor features. Each official release was a gamble as to its quality and support by third-party packages.

Thankfully, Mathworks loosened the prerelease restrictions to allow public discussion and implementation of prerelease features. Matlab prereleases are updated several times before the official release to help ensure features and fixes work across the diverse computing platforms Matlab supports. Kudos to Mathworks for having a more open prerelease process.

Git 2.49 added the option to build with Zlib-ng, which can give a reported 25% compression speedup, which can be quite significant for large repositories. Rust also began to be integrated in Git code along with further “libification” of Git code for reusability and quality.

Homebrew can install Fortran compilers including GCC and LLVM Flang.

Gfortran

Gfortran comes with GCC Homebrew package:

brew install gcc

As a complete C / C++ / Fortran compiler package, Gfortran doesn’t require additional flags or environment variables.

To use GCC compilers, source a script like:

p=$(brew --prefix gcc)/bin
v=14

export CC=$p/gcc-$v CXX=$p/g++-$v FC=$p/gfortran-$v

where v=14 is the major version number of the GCC compiler installed. It may be necessary to set SDKROOT if the compiler fails to find the C++ standard library headers like cstring, cstddef, etc.

LLVM Flang

LLVM Flang is a separate package from the LLVM C/C++ compilers:

brew install flang

To use Flang compiler (works with Clang, AppleClang, GCC C-C++ compilers), source a script like:

export FC=$(brew --prefix flang)/bin/flang

To use LLVM Clang with Flang, source a script like:

p=$(brew --prefix llvm)/bin
export CC=$p/clang CXX=$p/clang++

export FC=$(brew --prefix flang)/bin/flang

Troubleshooting

When a new compiler version or macOS version or Xcode SDK is released, it may be necessary to adjust the environment variables or flags temporarily until Homebrew updates the package.

Some examples to try if needed:

• LIBRARY_PATH for libSystem.tbd and libc++.tbd

  export LIBRARY_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib

• LDFLAGS to the C++ standard library

  export LDFLAGS=-lc++

SDKROOT may also be needed.

Since 2015, NVIDIA has supported development of LLVM-based Flang Fortran compiler. The legacy non-LLVM Flang was based on the PGI compiler, with proprietary optimizations removed. However, the C-based decade-plus of cruft led Flang developers and management to a ground-up rewrite in modern C++. LLVM flang targets Fortran 2018 syntax and is implemented in modern C++ internally. Flang became part of LLVM 11.

Building the LLVM Flang Fortran compiler from source is a resource-intensive process made simple by this script.

When doing large builds on a computer that’s also used for interactive tasks (say, a developer laptop), the system may feel sluggish to the user while building. Likewise, running tests can take a lot of CPU time and make the system less responsive. To mitigate the high CPU usage, the run priority of the build system or test runner can be lowered. This does not help if the system is running out of RAM or has low disk IOPS, but it can help if the system is CPU-bound. Running at a lower priority can be more efficient than simply hard-limiting the number of CPU cores used by the build system or test runner.

These examples apply to virtually any build system such as CMake or Meson.

On Unix-like systems (Linux, macOS, BSD, …) use nice to control process priority.

nice -n 19 cmake --build build

nice -n 19 meson compile -C build

nice -n 19 ctest --test-dir build

nice -n 19 meson test -C build

nice -n

run task(s) with priority, where bigger positive numbers are lower priority, and more negative numbers are higher priority.

On Windows the start command can control process priority.

start /low cmake --build build

start /low meson compile -C build

start /low ctest --test-dir build

start /low meson test -C build

start /low

runs a program and its child processes at low priority.

Matlab buildtool uses buildfile.m build plans that can each call other buildfile.m to stitch together Matlab subprojects from a top-level project. An example syntax for a top-level buildfile.m that invokes say Git submodule Matlab project is:

function plan = buildfile
plan = buildplan(localfunctions);
plan.DefaultTasks = "setup";
assert(~isMATLABReleaseOlderThan("R2024b"), "Subprojects with buildtool requires Matlab R2024b or newer")

end


function setupTask(context)

sub1 = fullfile(context.Plan.RootFolder, "MySubproject/buildfile.m")

% Matlab gitrepo() doesn't yet have the ability to run this command after cloning

if ~isfile(sub1)
  ok = system("git -C " + context.Plan.RootFolder + " submodule update --init --recursive");
  assert(ok == 0, "Failed to update MySubproject Git submodule");
end

buildtool("-buildFile", sub1, "setup")
% assumes that sub1 also has a task named setup that should be run

end

The red battery icon with a lightning bolt on a MacBook screen indicates that the battery is low and the Mac is not receiving enough power to boot. This can happen with a malfunctioning battery, an inadequate USB-C charger, or a damaged USB-C cable. If only the lightning bolt is under the red battery icon, this means the MacBook is receiving power via the USB-C port but there isn’t enough power to boot the MacBook. If there is a plug and lightning bolt under the red battery icon, this means the MacBook is not receiving power via the USB-C port.

This issue can arise when the MacBook battery has run out (0% charge) and the USB-C charger is not providing enough power to boot the MacBook. A solution is to find a USB-C (USB-PD) charger with at least 90 watts of power output to charge the MacBook battery to say 5% to 10% so that the MacBook can boot. One can then switch back to the lower wattage adapter that may take several hours to charge the MacBook battery. When traveling, especially with a high powered MacBook Pro, carry a USB-C adapter with at least the wattage of the original power adapter to avoid this issue. A temptation is to use say an iPhone or iPad lower wattage 15 W or 30 W charger with a MacBook Pro that may need 60 W or 90 W to boot from an empty battery.

Using the no boot checklist from Apple is also useful to troubleshoot the issue. It is uncommon to need to reset the SMC.

FFmpeg can optimally re-encode video from numerous formats for YouTube (or other service) upload. YouTube suggested settings for SDR video are implemented below. There are additional settings for HDR Video.

ffmpeg -colorspace bt709 -i in.avi -b:v 8M -bufsize 16M -c:v libx264 -preset slow -c:a aac -b:a 384k -pix_fmt yuv420p -movflags +faststart out.mp4

-colorspace bt709

BT.709 color space for SDR video. HDR should not use this flag.

-i in.avi

input file

-b:v 8M -bufsize 16M

8 Mbps video bitrate with 16 Mbps buffer size. This assumes 1080p input video–adjust this to the actual resolution of the input video. Use the bitrate table to choose appropriately.

-c:v libx264

H.264 video codec

-preset slow

better encoding quality at expense of more processing time

-c:a aac

AAC-LC audio codec

-b:a 384k

384 kbps audio bitrate (stereo) – this is not the sample rate which should be 48 kHz

-pix_fmt yuv420p

YUV 4:2:0 pixel format

-movflags +faststart

moov atom at the beginning of the file

out.mp4

MP4 container as suggested by YouTube

Additional flags that can be set, but require knowing the video parameters like frame rate:

-flags +cgop -g 30

GOP of 30 frames – should be 1/2 frame rate so here it was a 60 fps video

Video only, without audio

ffmpeg -i in.avi -c:v libx264 -preset slow -crf 18 -pix_fmt yuv420p -movflags +faststart out.mp4

Reference: FFmpeg for YouTube

When CMake fails on the configure or generate steps in a CI workflow, having CMakeConfigureLog.yaml uploaded a as a file can help debug the issue. Add this step to the GitHub Actions workflow YAML file:

  - name: Configure CMake
    run: cmake -B build

# other steps ...

  - name: upload CMakeConfigureLog.yaml
    if: failure() && hashFiles('build/CMakeFiles/CMakeConfigureLog.yaml') != ''
    uses: actions/upload-artifact@v4
    with:
      name: ${{ runner.os }}-${{ env.CC }}-CMakeConfigureLog.yaml
      path: build/CMakeFiles/CMakeConfigureLog.yaml
      retention-days: 5

The “retention-days” parameter is optional. Ensure the “name” parameter is unique to avoid conflicts with other jobs in the workflow. Here we assume that the OS and C compiler are unique between jobs.