Scientific Computing

Compare files between Git branches with a GUI like VS Code or Meld, showing all files different between the current Git branch to another Git branch “develop”:

git difftool develop

Compare a single file to another Git branch “develop”:

git difftool develop -- src/myfile.c

For projects consisting of more than a few files, or of files in different code languages, it can sometimes be beneficial to only run certain workflows depending on which files changed. For example, consider a Fortran-based simulation where the CI takes minutes or hours to run, with associated Python plotting code. If only the plotting code is changed, it might not be necessary to CI the entire simulation, but instead test just the Python code. This can be arranged via separate .yml files under the repo’s .github/workflows/ directory.

Example: only run Python analysis script tests (say, under “scripts/”) when analysis scripts are changed. If Fortran code or Python interface scripts are changed, run other CI.

File .github/workflows/ci_scripts.yml

name: ci_scripts

on:
  push:
    paths:
      - "scripts/**.py"
      - .github/workflows/ci.yml
  pull_request:
    paths:
      - "scripts/**.py"
      - .github/workflows/ci.yml

jobs:

  linux:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout
    - uses: actions/setup-python
      with:
        python-version: '3.x'

    - run: pip install .
    - run: pip install flake8 mypy pytest

    - run: flake8
    - run: mypy

    - run: pytest

File: .github/workflows/ci.yml

name: ci

on:
  push:
    paths:
      - "**.f90"
      - "**.cmake"
      - "**/CMakeLists.txt"
      - ".github/workflows/ci.yml"
  pull_request:
    paths:
      - "**.f90"
      - "**.cmake"
      - "**/CMakeLists.txt"
      - ".github/workflows/ci.yml"

jobs:

  linux:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
    - uses: actions/checkout
    - uses: actions/setup-python
      with:
        python-version: '3.x'

    - name: Install packages
      run: |
        sudo apt update
        sudo apt install --no-install-recommends gfortran libopenmpi-dev openmpi-bin

    - run: cmake -B build
    - run: cmake --build build
    - run: ctest --test-dir build -V

Matlab mustBeA is used in function argument validation to check that a variable is in one of multiple classes, instead of the single class coercion used by Matlab arguments block by default.

Simply create a file in the project directory “mustBeA.m”, or under private/mustBeA.m

function mustBeA(x, classes)
% for Matlab < R2020b
arguments
  x
  classes (1,:) string
end

mustBeMember(class(x), classes)

end

Matlab mustBeFile and mustBeFolder can fail to validate when the filename starts with a tilde “~”. Tilde is used by most terminal shells to indicate the user home directory. However, Matlab does not currently recognize tilde.

Workaround this issue with expanduser.m in the mustBe* call.

Example: myfun.m:

function myfun(A, B)
arguments
  A {mustBeA(A, ["string", "char"])}
  B (1,1) {mustBeNumeric}
end

...

end

A summary of public Windows Subsystem for Linux changes for the latest Windows Fast Ring build and recent builds is provided. A master listing of current build vs. Ring is also available.

Additional notes:

• Windows Subsystem for Linux release notes
• Windows Subsystem for Linux: About with link to issue trackers, presentations, etc.
• Microsoft Commandline Blog

Common web page / blog post applications where concealing text until the viewer clicks include:

• table of contents
• spoilers
• long details not all readers are interested in at that point in the article

This is implemented via the HTML5 details tag. This technique works for many Markdown-based sites such as GitHub (GitHub Issues click-to-reveal long debug text) and GitLab.

<details>
  <summary>click for spoilers</summary>

  <ul>
    <li>The season finale for Season 1 showed ...</li>
    <li>And season 2 opener revealed ...</li>
  </ul>
</details>

Example

IANA RFC8615 defines a “.well-known/” folder for web servers to put metadata fundamental to a web server’s organization, policies or contact information. Several popular files are seen under public website https://example.invalid/.well-known/ directory.

keybase.txt

verify ownership of domain for Keybase.io

security.txt

security incident contact info

dnt-policy.txt

Do Not Track Policy statement

humans.txt

author information

The 2020 movie “The Midnight Sky” is based on the 2016 Lily Brooks-Dalton book Good Morning, Midnight. I customarily approach book-based films by reading the book first. This review is only about the book.

I found the character-oriented narrative quite compelling, with multiple characters having development and reflection. The only slight downside to the book is as typical in most fiction (including post-apocalyptic and science fiction), the radio communications and vehicle aspects of the book could have been described more accurately by having a technical reviewer. I think the author did a better than average approach to the radio comms, minus the glitches noted below. I compliment the author’s approach–these slight issues are endemic in fiction books. Contradictions I notice involved transportation and radios comms.

A gasoline engine that has a pull rope start is subsequently described as having a dead battery and therefore unable to start. In fact, it’s possible in some cases to improvise a pull start on small electric start engines. Since another similar vehicle is described as having electric start, this may have been an editing mistake (perhaps the final intent was to have both vehicles be electric start).

The author’s ham radio callsign is used by a character–cute! The author could have more specifically addressed LEO satellite comms window for simplex VHF comms. The characters could simply say VHF / UHF, as it’s uncommon to spell out the abbreviated terms. There are frequent references to RF sine waves, when actually random noise was seemingly implied.

Modern scientific practices are concerned with obtaining results in a socially conscious and responsible manner, minimizing external impacts. Recent papers including Strubell, Ganesh, McCallum 2019 “Energy and Policy Considerations for Deep Learning in NLP arXiv note the energy consumption impacts of machine learning and AI. A tech ethics leader pushing for responsible AI was abruptly fired. Critical challenges in efficient food and packaging supply chains include plastic recycling, plastic ocean patches. and human factors such as child slavery in chocolate production. The American Geophysics Union (AGU) reorganized sections including Science and Society and GeoHealth oriented around the intersection science has with human well-being and government policies.

Companies increasingly use systems approaches to product quality and minimizing external costs. Goodnow Farms chocolate production of top-quality confectioneries considers the whole cacao lifecycle. Each chocolate type has an associated story and photo book showing the local farmer’s process.

Meson projects using Meson subprojects have a project directory structure like:

meson.build
main.c
subprojects/
  lapack/
  lapack.wrap

where “subprojects/lapack/” is automatically created as a Meson subproject from the metadata in “subprojects/lapack.wrap”. It’s desirable to ignore the subprojects code directories in Git.

For Meson subprojects, using the negate .gitignore syntax Git will ignore subdirectories but track the subprojects/*.wrap files, by including in the project top-level .gitignore:

/subprojects/*

!/subprojects/*.wrap

This technique can be applied to similar non-Meson scenarios.

NOTE: The .gitignore syntax has a precise meaning. For example, /subprojects/* means to ignore the directory contents of top-level directory “subprojects”, but not the directory itself. This is important for the next stanza !/subprojects/*.wrap, which means to not ignore files with .wrap suffix in top-level directory “subprojects”.

Thanks to @lunasorcery for pointing out the correct .gitignore syntax.

Empty files can be used as a mean to pass information between programs in a persistent manner, or a form of configuration. Empty files can be used on web servers to declutter 404 logs of endless bot scans.

For the examples below, we assume relative filename “path/to/empty”, where directories “path/to” already exist. We assume that a file does NOT exist at the filename, and it may be overwritten if it exists.

NOTE: For simplicity, some of these examples overwrite existing files, but some do not overwrite. Extra code is required to give a consistent behavior. All examples at least create an empty file if one doesn’t exist.

Windows users often use PowerShell. Create empty file with PowerShell (on any operating system):

New-Item path/to/empty

For Linux, macOS and other Unix-like systems, most shells create empty file like:

touch path/to/empty

Python creates an empty file by:

pathlib.Path('path/to/empty').touch()

Fortran creates an empty file by:

open(newunit=u, file='path/to/empty')
close(u)

Create an empty file in C++:

#include <fstream>

std::ofstream output("path/to/empty");