Scientific Computing

CNAME flattening was popularized in part by Cloudflare. We have used CNAME flattening successfully for years with multiple web-hosting providers. DNS A records are a pre-WWW artifact necessary to resolve the apex domain example.invalid from www.example.invalid or radar.example.invalid.

Implement CNAME flattening

To enable CNAME flattening on DNS records, first screenshot or otherwise backup DNS settings. Don’t do this experiment during busy times, better to use on a little-used or test website first to ensure it works correctly with the proposed setup.

1. Determine the web address the web hosting provider puts the website at. E.g. for GitHub Pages it would be username.github.io or at Netlify username.netlify.app
2. Remove DNS A record for example.invalid that points to a specific IP address
3. Add a CNAME record pointing example.invalid to the server address from step #1

For GitHub username joe with GitHub Pages site at joe.github.io, with desired web address example.invalid: make a CNAME DNS record with example.invalid as an alias to joe.github.io.

A or AAAA DNS records are unneeded with CNAME.

Notes

• Cloudflare CNAME flattening article
• You may see some old 2014 blog posts initially complaining about CNAME flattening, but these initial hiccups have been resolved long ago.

Fortran submodule is supported by all popular Fortran compilers. While designed as a way to better manage public/private exposure of variables in large Fortran modules, submodules can also be used to seamlessly switch in/out advanced functionality.

For example, the GEMINI 3-D ionospheric model was created with raw binary file I/O. Since we had already written an object-oriented HDF5 interface, we integrated HDF5 file I/O into GEMINI. To help ensure a smooth transition with seamless fallback to raw binary if HDF5 wasn’t available, we used Fortran submodule with CMake. The user would call file_read and file_write subroutines with the same name, regardless of whether HDF5 was enabled. CMake would switch in submodule files depending on whether HDF5 was working or not.

Requirements

Fortran submodule requires adequate support from the Fortran compiler, and from the build system. CMake and Meson fully support Fortran submodule across compilers.

Compilers supporting Fortran submodule include:

• Gfortran ≥ 6
• Intel oneAPI
• Cray
• IBM XL / OpenXL
• Flang
• NAG
• Nvidia HPC SDK

CMake

Rather than maintain a compiler feature table, in general we create simple test programs and verify that they compile–all automatically handled within CMake.

Insert into CMakeLists.txt

include(CheckSourceCompiles)

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)
# save link time, only compile is needed

check_source_compiles(Fortran
"module b
interface
module subroutine d
end subroutine d
end interface
end

submodule (b) c
contains
module procedure d
end
end

program a
end"
f08submod)

if(NOT f08submod)
 return()  # or make FATAL_ERROR here
endif()

Selectively enable program functionality using submodule in CMakeLists.txt. This example is for HDF5:

add_library(io io.f90)

if(USE_HDF5)
  target_sources(io PRIVATE hdf5.f90)
else()
  target_sources(io PRIVATE raw.f90)
endif()

Complete examples of submodule are provided.

While both GitHub Pages and GitLab Pages are adequate for most personal, group and project pages, when website size and / or traffic have grown beyond what is feasible for these solutions, a more comprehensive hosting provider like Netlify may be considered. Netlify provides its own CDN, so those that had been using Cloudflare for DNS and CDN can configure Cloudflare to provide only DNS, if they so choose. Netlify is free for single users, allowing a private GitLab, GitHub or Bitbucket repo (or other suitable source) to deploy to a public custom domain HTTPS website. SSL certificates can be user-provided or can be created through Netlify for your custom domain.

Why transfer site to Netlify

Netlify provides a comparison of GitHub Pages and Netlify. GitLab Pages allows user choice of static site generator (Hugo, Jekyll, etc.). GitHub Pages can using GitHub Actions for Hugo. GitLab Pages private repos have a monthly runtime quota. Netlify has a monthly traffic quota on the free tier, and monthly build quota. For sites that are becoming very popular, GitHub Pages will simply want you to move elsewhere, while Netlify will have a paid plan to offer. This process may be too burdensome for those with limited IT or bandwidth resources, or simply the lack to time to learn how to do this.

Netlify uses webhooks to detect a git push to the website GitLab repo, and then builds the site. Netlify has a CDN and DDoS protection built-in. Even if the other features aren’t needed, a key feature is the ability to have the website code in a private repo with unlimited public website deployments and traffic.

Build minute limits (such as on GitLab and Netlify) can legimately be worked around by building the site locally on your laptop and pushing the publish-ready HTML.

Transfer site to Netlify

Note: This process may take down your site for a day or two if things go wrong. Even under normal conditions, all site visitors may need to allow an HTTPS exception due to SSL certificate error since Netlify requires all DNS servers to update before generating the domain certificate.

1. if not already on GitLab, copy your website repo to GitLab (any name repo is fine).
2. disable Auto DevOps and ensure no file named .gitlab-ci.yml exists.
3. Login to Netlify using Gitlab, which will ask for your website repo.
4. pick a custom Netlify subdomain like mycompany.netlify.app. Ensure this site is totally working before proceeding.
5. Set Cloudflare or whatever your DNS provider is to point CNAME or A to mycompany.netlify.app (THIS IS THE PART THAT CAN TAKE YOUR MAIN WEBSITE DOWN!)
6. Under Netlify Domain Management → HTTPS → Verify DNS config, ensure the verification completes. Until the DNS change propagates worldwide, your main HTTPS domain visitors are getting SSL verification errors. They can use https://mycompany.invalid instead of https://mycompany.invalid temporarily. Do this at a low traffic time range! If using Cloudflare CDN, the old records may point to DigitalOcean while the new records point to *.netlify.app

If the Git commits have already been pushed to remote, this process will require other users of the repo to reset, rebase or reclone, as for any Git operation that edits history. If the Git commits have not already been pushed, then this process will not require extra steps from other repo users.

Show Git commit AuthorDate and CommitDate by

git show <commit_hash> --pretty=fuller

for the most recent commit, simply:

git show --pretty=fuller

edit last commit only

To reset the author of the last commit to the current Git username and email, as well as setting AuthorDate and CommitDate to the current time:

git commit --amend --reset-author --no-edit

--reset-author

reset date/time/author to current

--no-edit

skip opening text editor

edit previous commits, including already pushed

Use git rebase -i as usual and for the commits to reset author / date, change the operation to e to edit each by:

git commit --amend --reset-author --no-edit

Specific date setting

Specific commit times can be set with a combination of the “–date” option and environment variable GIT_COMMITTER_DATE.

Reference

GitHub commit troubleshooting

This procedure is for PyPI Warehouse.

Assuming a Python package named “myprogram”, have a directory structure like

pyproject.toml
myprogram/
   __init__.py

A minimal pyproject.toml is all that’s required for a PyPI Python package. The package “version” number must be distinct for each release, or PyPI will prevent upload.

Ensure prereqs are installed:

pip install --upgrade twine build

One-time setup: sign up for a PyPI account. Create ~/.pypirc with the PyPI token.

In the project directory, create the package:

python -m build

Upload the package to PyPI:

python -m twine upload dist/*

Now the package is live to the world on PyPI, installable via pip install.

Intel oneAPI on Windows provides an easy way to use Fortran MPI on Windows. The Intel oneAPI compile and link commands on Windows are distinct from those on Linux, perhaps reflecting the internal use of Visual Studio on Windows. The HDF5 1.10.6 release changed the naming convention for the HDF5 Fortran library files on all operating systems.

• old: hdf5hl_fortran.
• new: hdf5_hl_fortran.

If experiencing trouble finding HDF5 with CMake, try FindHDF5.cmake. FindHDF5 specifically works with Intel compilers and HDF5 across operating systems including Windows.

To build HDF5 library, including on Windows, use CMake to build HDF5.

If you use include/static you will get errors like

error LNK2019: unresolved external symbol H5GLOBAL_mp_H5F_ACC_TRUNC_F referenced in function MAIN__

The free open source SSH app ConnectBot allows connecting to SSH servers with port forwarding using public key authentication, including ED25519.

Sometimes it’s necessary to share SSH keypairs on multiple clients. Perhaps the server owner isn’t willing to bother with more than one SSH client key. ConnectBot has an open issue since it cannot import OpenSSH keys created on a PC.

Generally users should create unique SSH public/private keypairs for each device. Sharing keys between devices means if a device is compromised, deleting its key from ~/.ssh/authorized_keys on the SSH server disables all other devices sharing that key.

Workaround by creating an SSH keypair in ConnectBot. Copy the ConnectBot-created public/private keypair to the PC ~/.ssh directory.

The stem (filename without extension) of the public and private keys must match. The public key should have a .pub suffix, while the private key has no suffix.

Thereby the same SSH keypair is used on the phone with ConnectBot and the PC with OpenSSH client.

To integrate changes from branch “feature1” without cherry picking can be done file by file or line by line.

Compare current unstaged changes file-by-file, with the ability to line-by-line implement changes:

git difftool feature1

Whole branch compare does NOT allow line-by-line edits, so perhaps open a second terminal and “git checkout” files from the feature branch as desired.

git difftool -d feature1

Meld and Visual Studio Code are each good GUIs for Git difftool and mergetool.

Carriers in numerous countries worldwide have shutdown 2G networks to free spectrum for 4G and 5G services. Embedded modems such as automotive (OnStar) and alarm systems may again be impacted by these shutdowns. In developing regions 2G networks may linger for several more years, due to cost-effective legacy devices.

Those designing IoT and other embedded devices with cellular modems should consider LPWA 4G LTE, particularly Cat M1 and NB1 to help ensure global functionality. Each geopolitical region has unique LTE bands, but often OEM modules with the same pinout have region-specific models. OEM LTE modules will incorporate at least some bands for each region, so that global LTE roaming for even inexpensive LTE modems will become increasingly common.

2G shutdown dates from Digi

Notes

• 2G / 3G shutdown table
• 2G shutdown chart from Digi

The USA 1G AMPS and 2G D-AMPS networks were shutdown in 2008. In June 2013, Sprint shutdown the Nextel iDEN network for LTE band 26.

A local Git repo can become corrupted in rare circumstances, perhaps doing a git commit just as a computer crashes or loses power. A common symptom of a corrupted local Git repo is any Git command except for perhaps git diff giving error:

fatal: your current branch appears to be broken

The changes are likely still present, as seen via git diff. Previous commits that were not pushed to remote are likely present as well in the form of the modified files, but the historical local Git commit deltas may not be recoverable. This recovery will in effect “squash” the local commits that weren’t previously pushed to remote.

NOTE: work done on other branches that weren’t pushed to remote may not be retrievable.

Recovery

1. Copy the directory tree of the affected local Git repo, preferably on another hard drive or in the cloud.
2. git clone a fresh copy of the remote Git repo to a new directory
3. Compare folders to incrementally copy into the new directory the changes from the old corrupted directory. If there are a large number of changes, consider making the changes via multiple Git commits.

Notes

• git recovery reference