Python
pathlib.Path.iterdir
only iterates over a single directory level.
It does not recurse directories.
A common task is to iterate over each subdirectory of a top-level directory non-recursively.
Given directory structure:
a/b
z
y/1/2
The Python
iterdir example
would return (in unspecified order):
a
z
y
Notice in the C++ and Python examples, iterators are used that emit one element at a time rather than building up an entire list at once.
Since directory operations are often unordered, there is no advantage to retrieving every directory name in a greedy operation rather than the lazy generators shown, particularly if networked file systems are being used.
The
C11 standard
defines
optional bounds-checking functions
with an “_s” suffix in their names in
Annex K.
There are
numerous reasons
why these functions aren’t implemented in popular compilers / stdlib except MSVC.
The most salient points are in the
field experience note
that observes that static analysis, dynamic analysis, address sanitization, etc. provide benefits that are largely beyond what the secure functions could provide, without the end user runtime penalty.
For totally new projects, one could consider coding languages that have inherently more secure memory access such as Rust.
Or for a less dramatic change, using C++ for string-heavy portions of the project where the
string class
can be easier to use than
C char.
Major Git projects commonly have a workflow where other contributors
fork
a primary Git repository, make changes, and then contribute back to the primary project.
This article describe the maintainer workflow.
The contributor workflow is in a separate article.
This workflow is also suitable for projects using Git submodules, where you may want a submodule to temporarily use a branch from another repository.
Maintainers of a primary Git repo can make a local copy of the forked Git branch from the contributor’s Git repo to ensure the changes work as desired.
Two ways for the maintainer to make this local copy are described in this article.
This workflow is suited to accommodate regular contributors to a project, for example colleagues or employees.
In this example, we assume the primary project branch to merge new contributions into is “main” and that the remote contributor branch is “add-feat1”.
This workflow is suitable for occasional contributors.
It avoids cluttering the local repo with many upstream repos metadata in .git/config.
On the local copy of the primary project, create a temporary branch in which to put the contributor’s remote branch.
Here we assume the remote branch is “add-feat1”:
CMake uses vendored libraries including curl and nghttp2 to provide internet connectivity in CMake.
As web technologies are created and obsolesced given the ubiquitous nature of HTTPS today, over various library versions, CMake version, and system configurations some failures may occur.
To help catch these failures in new releases of CMake, as well as check connectivity on a particular platform and CMake version, try script
check_https.cmake.
Public Git repos, GitHub Gists, and GitLab Snippets can use HTTPS for “git fetch”, “git pull”, and other Git download operations.
Using HTTPS to download and verifying the author PGP signed Git commits can help assert that the content is from the intended authors.
Git download operations over HTTPS are perhaps twice as fast as Git over SSH and use less CPU.
By default,
Git verifies SSL
certs.
Typical Git hosting providers such as GitHub require SSH for enhanced security.
Since “git push” operations typically take longer than “git pull”, particularly where
pre-commit hooks
and
PGP commit signing
are used, SSH speed penalty on “git push” is often acceptable.
For developers there are speed benefits from a hybrid Git configuration where Git downloads use HTTPS and Git uploads use SSH.
Git has intrinsic functionality for this setup in a global configuration.
The one-time setup below uses “https://” for the remote repo URL instead of “ssh://”.
Replace “your_username” with your GitHub username for SSH GitHub Gists Git push.
Major Git projects commonly have a contributor workflow where other contributors
fork
a primary Git repository, make changes, and then contribute back to the primary project.
This article describe the contributor workflow.
The maintainer workflow is in a separate article.
The contributor forks the primary project Git repo.
On the local copy of the fork, create a feature branch:
git switch -c add-feature1
Once the new work is complete, make the branch consistent with the primary repo by pulling in
the latest changes from the primary repo:
git switch main
# whatever the primary or development branch of the primary repo isgit remote add upstream https://github.invalid/primary/repo_url
git fetch upstream
git rebase upstream/main
Update the local branch to remote main
git switch add-feature1
git rebase main
Test the final updated code.
Create the
Pull Request
/
Merge Request
for the maintainer to evaluate the changes and possibly put them into the primary project.
OpenMPI 4.x uses temporary files for UNIX sockets, which are limited to 100 character path length.
This can become an issue on macOS, which by default sets POSIX environment variable TMPDIR to a pseudorandom path under “/var” about 50 character long.
We have experienced this as a race condition that becomes more likely as the number of MPI workers is eight or more.
A
workaround
is to set an alias for mpiexec or within CMake test properties etc. such that mpiexec gets a sufficiently short working path so that UNIX sockets don’t fail due to excessive path length.
Midnight Commander
is an
open source
Terminal-based GUI for browsing filesystems.
It’s useful to have a non-graphical method to manage and view files, particularly on remote systems over SSH.
Installing Midnight Commander is done by:
Fortran 2018 contiguous arrays are discussed in pp.139-145 of “Modern Fortran Explained: Incorporating Fortran 2018”.
In general, operating on contiguous arrays is faster than non-contiguous arrays.
Contiguous variables happen by default, so unless using pointers or array striding, the variable is likely to be contiguous for simple Fortran code.
Check contiguous status of a variable with is_contiguous intrinsic function.
A non-contiguous array actual argument into a contiguous subroutine dummy argument is made contiguous by copy-in, copy-out.
This copy-in copy-out as needed is part of the
Fortran 2008
and
Fortran 2018
standard.
GCC ≥ 9,
Intel oneAPI,
IBM Open XL Fortran,
etc. work to Fortran 2008+ standard for contiguous dummy argument copy-in, copy-out for non-contiguous actual argument.
CMake can use clang-tidy as a build option for a CMake project by adding a few parameters to CMakeLists.txt.
It’s also possible to use clang-tidy without modifying CMakeLists.txt.
Generate the file compile_commands.json used by clang-tidy.
You don’t have to build the project as we are interested in checking and possibly fixing the C++ source code.
To install clang-tidy:
macOS: brew install llvm, then edit ~/.zshrc, adding the line alias clang-tidy=$(brew --prefix llvm)/bin/clang-tidy or similar
Linux: apt install clang-tidy or similar
cmake -Bbuild -DCMAKE_EXPORT_COMPILE_COMMANDS=on
Run clang-tidy. Specify options or use a file .clang-tidy, which is usually in the top directory of the project.