When using Intel oneAPI compilers on Windows, you may get a message like:

INTERNAL ERROR: pgopti_Create_Full_Path: buffer too small

This is probably caused by a filepath that’s more than 139 characters. You may not see the paths printed as being that long, because CMake with Ninja does preprocessing on each Fortran file that lengthens the filename.

Fix

The fix is to make the project paths shorter. Users may have to build code from a directory nearer the top of the filesystem hierarchy.

Future-dated blog posts make daily or weekly blogging much more feasible, by allowing one to spontaneously create multiple posts made public over time. Another application is when reviewing embargoed topics, to be ready to deploy on release day. Use caution that the features being reviewed are actually in the release–I’ve seen software drop beta and even release candidate features in the public release.

Hugo

Hugo is perhaps the fastest static website generator, which we prefer over Jekyll. Hugo can handle future-dated blog posts by simply regenerating the web page after the future blog post time has arrived. This can be done with a dummy Git commit and push, or triggering the build via the Netlify API control panel.

The config.yaml option for making future posts appear (which some might use for “pinning” a blog post) is:

buildFuture: true

Omitting that config.yaml option (default: buildFuture: false) allows future blog posts to appear on the desired date upon redeploy after the future post timestamp. To preview future posts on the local computer:

hugo server -F

-F == --buildFuture

set post date

The date / time of a Hugo blog post is typically set with the date: front matter variable. Note the publishDate: and expiryDate: variables for advanced usage. The lastmod: variable for last modified date HTML header can be implemented automatically in config.yaml:

enableGitInfo: true

The last modified date is for updates to a blog post without changing the original posted date of the article. The Git commit update time will thereby be in the article metadata, indicating the “freshness” of the article.

list future and draft posts

Hugo posts can be set as “draft”, normally not rendered for deployment by front matter variable draft: true. Future and draft posts can be listed from the top level Hugo project directory by:

hugo list future

hugo list drafts

When building autotools project via CMake ExternalProject, it is useful to first check in CMake that autotools is available to avoid emitting errors at build time. The goal is to emit errors about missing packages during build system configuration instead of during the build. We do this by including the file FindAutotools.cmake

Building an autotools project as a CMake ExternalProject saves the time of converting that other project to CMake. This technique makes it easy to automatically build that other project when it’s not easily installable otherwise, or you wish to build it optimized. This technique has project-specific aspects, but here is a high-level overview. One of the key issues this technique overcomes is the ExternalProject rebuilding each time the main CMake project is rebuilt.

This example is simplified from the p4est-cmake project.

Main CMakeLists.txt

if(NOT p4est_external)
  find_package(p4est)
endif()

if(p4est_external OR NOT p4est_FOUND)
  include(cmake/p4est.cmake)
endif()

cmake/p4est.cmake

set(p4est_external true CACHE BOOL "build p4est library")

include(ExternalProject)

set(p4est_LIBRARIES)
# list each of the library binaries used by the project
foreach(_l p4est sc)
  list(APPEND p4est_LIBRARIES ${PROJECT_BINARY_DIR}/lib/lib${_l}${CMAKE_STATIC_LIBRARY_SUFFIX})
endforeach()

if(EXISTS ${PROJECT_BINARY_DIR}/lib/libp4est${CMAKE_STATIC_LIBRARY_SUFFIX})
  # this is what we use to tell that the ExternalProject was already built,
  # and to then hide the ExternalProject_Add
  set(p4est_FOUND true)
endif()

if(NOT p4est_FOUND)

  ExternalProject_Add(p4est_proj
  GIT_REPOSITORY https://github.com/cburstedde/p4est.git
  GIT_TAG prev3-develop
  PREFIX ${PROJECT_BINARY_DIR}
  CONFIGURE_COMMAND ${PROJECT_BINARY_DIR}/src/p4est_proj/configure
  BUILD_COMMAND make -j
  INSTALL_COMMAND make -j install
  TEST_COMMAND ""
  BUILD_BYPRODUCTS ${PROJECT_BINARY_DIR}/src/p4est_proj-build/src/.libs/libp4est.${CMAKE_STATIC_LIBRARY_SUFFIX}
  )

  ExternalProject_Get_Property(p4est_proj SOURCE_DIR)

  ExternalProject_Add_Step(p4est_proj
    bootstrap
    COMMAND ./bootstrap
    DEPENDEES download
    DEPENDERS configure
    WORKING_DIRECTORY ${SOURCE_DIR})

  file(MAKE_DIRECTORY ${PROJECT_BINARY_DIR}/include)  # avoid race condition
endif()

add_library(p4est::p4est INTERFACE IMPORTED GLOBAL)
target_include_directories(p4est::p4est INTERFACE ${PROJECT_BINARY_DIR}/include)
target_link_libraries(p4est::p4est INTERFACE ${p4est_LIBRARIES})
# set_target_properties didn't work, but target_link_libraries did work

if(NOT p4est_FOUND)
  # this is necessary to trigger the ExternalProject
  add_dependencies(p4est::p4est p4est_proj)
endif()

Switching to CMake from autotools or plain Makefiles gives generally better support of many operating systems, in particular Windows. Another benefit is generally faster speed of configuration and build. In 2020, MonetDB switched from autotools to CMake. They observed MonetDB with CMake builds in 40% of the original time. Note also the improvement due to Ninja instead of GNU Make.

Techniques

I have converted a number of projects from autotools to CMake. Generally, I use an automated autotools to CMake script and then manually compare or redo by inspection of the autotools scripts.

In general, livestream data bandwidth depends on the video content (lots of motion vs. talking head) and wireless conditions.

Twitter 1280 x 720 streaming is available via Twitter Live Producer API. The recommended data bandwidth is 2500 kbps ~ 20 MByte / minute.

YouTube Live has livestreaming modes from 240p to 4K @ 60 fps. Consider 480p to start to keep the connection from dropping out, with data bandwidth of 500 - 2,000 kbps ~ 16 Mbyte/minute.

These live streaming methods are easily accessible via PyLivestream that lightly wraps the FFmpeg command line. It generates a command line that you can use directly, or copy / paste for future reference.

Hugo shortcodes are a powerful, easy way to template code used in blogs and websites. Ana Ulin describes a one-line Hugo shortcode to inline HTML in Hugo:

Under the top-level Hugo website Git directory, add file “layouts/shortcodes/rawhtml.html” containing:

{{.Inner}}

Then in the Markdown file for the particular blog post, do like (removing the space between the left brace and the left caret):

{{ < rawhtml >}}
<p>arbitrary HTML here</p>
{{ < /rawhtml >}}

example

Comparing the current Git branch to another Git branch “develop” with a GUI is done like:

git difftool develop

Comparing a single file to another Git branch “develop” is done like:

git difftool develop -- src/myfile.c

Git difftool setup

Meld is a popular tool for comparing files and directories that works on Linux, Mac and Windows. Setup Meld with Git like:

git config --global diff.tool meld
git config --global merge.tool meld

I have seen some risky SSH/Remote Desktop home setup of embedded systems such as the Raspberry Pi or other devices used in remote work or hobbies. The tips below do not imply a “secure” SSH home server setup, but may be better than not taking any security measures at all. While it’s obvious that startup companies running SSH/Remote Desktop servers at home or work should be very concerned about infiltration and IP theft, the average home user with a Raspberry Pi SSH server can be hacked too. The Raspberry Pi can become a launching pad for vulnerabilities on the laptops, tablets, phones, IP cameras, etc. in the home.

VPN

If you have a standalone hardware firewall such as pfSense / opnSense that has a built-in VPN server, consider using that as a first line of defense for devices behind the firewall.

Port forwarding

Typical home internet “modems” for cable, DSL, fiber, etc. have the ability to port forward a public-facing port to a specific LAN IP and port. This doesn’t add any security, but is a way to provide a known “place” to connect to from the outside world. Even without static home IPs, many ISPs have the same client modem public IP addresses for months. On the LAN, be sure to assign a fixed IP for the device, as DHCP servers will often assign a new LAN IP to the device.

SSH hardening

SSH server hardening is an extensive topic. At the bare minimum:

• do not allow password-based login
• allow only public key authentication
• make distinct remote login ID that is non-sudo enabled
• strongly suggest using IP range allow-lists

For projects consisting of more than a few files, or of files in different code languages, it can sometimes be beneficial to only run certain workflows depending on which files changed. For example, consider a Fortran-based simulation where the CI takes minutes or hours to run, with associated Python plotting code. If only the plotting code is changed, it might not be necessary to CI the entire simulation, but instead test just the Python code. This can be arranged via separate .yml files under the repo’s .github/workflows/ directory.

Example

As per above, only run Python analysis script tests (say, under “scripts/") when analysis scripts are changed. If Fortran code or Python interface scripts are changed, run other CI.

File .github/workflows/ci_scripts.yml

name: ci_scripts

on:
  push:
    paths:
      - "scripts/**.py"
      - .github/workflows/ci.yml
  pull_request:
    paths:
      - "scripts/**.py"
      - .github/workflows/ci.yml

jobs:

  linux:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2
      with:
        python-version: '3.x'

    - run: pip install .
    - run: pip install flake8 mypy pytest

    - run: flake8
    - run: mypy

    - run: pytest

File: .github/workflows/ci.yml

name: ci

on:
  push:
    paths:
      - "**.f90"
      - "**.cmake"
      - "**/CMakeLists.txt"
      - ".github/workflows/ci.yml"
  pull_request:
    paths:
      - "**.f90"
      - "**.cmake"
      - "**/CMakeLists.txt"
      - ".github/workflows/ci.yml"

jobs:

  linux:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2
      with:
        python-version: '3.x'

    - name: Install packages
      run: |
        sudo apt update -yq
        sudo apt install -yq --no-install-recommends gfortran libopenmpi-dev openmpi-bin        

    - run: cmake -B build
    - run: cmake --build build
    - run: ctest -V