Scientific Computing

Numpy ABIs change between releases, which can lead to errors upon

import numpy

like:

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

try fixing by upgrading Numpy.

conda install numpy

# or

pip install --upgrade numpy

For scientists, engineers and other performance-sensitive coders modern Fortran offers immediate advantages in developer time savings. The clarity, conciseness and power of modern Fortran are widely available in contemporary compilers. This brief post was motivated by viewpoints encountered including:

• those whose boss insisted on Fortran 77–they didn’t know anything newer than Fortran 90 existed.
• those who thought essentially no compilers supported newer than Fortran 95 standard (in calendar year 2018).

To be effective with programmer time, one generally shouldn’t needlessly upgrade all Fortran 77 code to modern Fortran, since Fortran has always maintained good backward compatibility. However, new and upgraded Fortran code should almost never be written in Fortran 77 unless specific job conditions dictate. Of course, Fortran 66 / Fortran IV is little supported and will need to be upgraded to Fortran 77 syntax, which is very similar except for file I/O.

New Fortran code should at least use Fortran 2003, which is universally supported by current compilers. In HPC environments, Gfortran and Intel oneAPI are widely supported, so we use modern Fortran features in virtually every program.

Modern Fortran support

The compilers easily available supporting modern Fortran include:

• Gfortran
• Intel oneAPI
• NAG
• LLVM Flang
• Nvidia HPC

Polyhedral benchmarks compilers supporting modern Fortran features.

What did the major Fortran versions add

• Fortran 95 brought strong N-dimensional array operations. It is a key step toward modern Fortran, enabling arbitrary size (elemental) intrinsic and non-intrinsic procedures. With Fortran 95, one no longer had to to explicitly loop over almost every array operation.
• Fortran 2003 brought polymorphism and true object-oriented procedures, critical parts of modern generic programming.
• Fortran 2008 strengthened polymorphism, and baked coarray (distributed parallelism) directly into Fortran, transparently using underlying libraries such as OpenMPI. Improved Fortran software architecture comes through submodule enabled by Fortran 2008.
• Fortran 2018 strengthened coarray support, and did further important language cleanup such as enabling error stop within pure procedures

Related: Gfortran feature set vs. OS and version

Summary:

• stop (1956 Fortran I): return integer on stderr (recommendation)
• stop (Fortran 77): return integer or constant character – no integer return with character
• error stop (Fortran 2008): constant character with error code
• error stop (Fortran 2018): variable character with error code, also allowed inside pure procedure.

Fortran 2018 finally brought the needed behavior for convenient error messages and continuous integration.

CMake and Meson handle automatic detection of compiler supported features like error stop.

Fortran 2008 error stop with constant code:

• Gfortran ≥ 5
• NAG ≥ 6.0
• Intel oneAPI
• Nvidia HPC SDK

Fortran 2018 error stop with variable code

• Gfortran ≥ 7
• NAG ≥ 6.2
• Intel oneAPI
• Nvidia HPC SDK

Fortran 2018 error stop,QUIET=.true./.false.

• NAG ≥ 6.2

This feature was promoted by Steve Lionel, but has not yet been widely adopted. From the Fortran 2018 standard, the quiet= parameter not only suppresses any console output but also may suppress the error code? I would prefer to have the error return code, without the console text.

Since Fortran I in 1956, the stop statement has generally displayed a return code to indicate an error if an integer value was provided. Over time, stop statement behavior has changed to allow more refined signaling on program stop.

Since Fortran I in 1956, stop without return code to stop execution normally has been supported, along with stop with integer return code to indicate abnormal termination.

stop 1

The Fortran 2008 and 2018 standards recommend that the error code be returned on iso_fortran_env: error_unit, which was first defined in Fortran 2003 standard. Earlier standards don’t say where the error code should go. Through Fortran 2018, stop with integer code is still normal program termination.

Since Fortran 77, stop may instead return a constant scalar character like “goodbye”. This generally sets stderr return code to 0, that is, no error is indicated.

For continuous integration, having a reliable way to indicate error behavior is crucial. For HPC, indicating abnormal conditions to the shell is also vital to avoid taking resources on runs that suffered a computational error.

Fortran 2008 brought the long overdue error stop statement. stop still indicates normal program termination, and can for example stop individual images in a parallel executing program. Say an individual cell in a 3-D simulation did not find a stable solution. Depending on the simulation, that can be OK, perhaps set that value to NaN and stop with an error code on stderr, while letting the other images continue running.

However, in other types of simulations, an early failure to converge to a solution in a cell may invalidate the entire simulation taking a month of CPU time. Instead of writing cumbersome external checking code, the programmer can instead use error stop to reliably terminate all images when a critical failure is detected. Fortran 2008 error stop with constant string or integer code: both return non-zero exit status on stderr.

use, intrinsic:: iso_fortran_env, only: stderr=>error_unit

write(stderr,*) 'the failure was in ' // failedmod

error stop

Fortran 2018 added error stop with variable scalar string or variable integer code. A vital addition of Fortran 2018 is that error stop can be used within pure procedures, a very commonly needed use case. Fortran 2018 error stop variable character string allows for cleaner syntax, for example:

error stop 'the failure was in ' // failedmod

Python package installs can use MANIFEST.in to install arbitrary files to the package install directory. MANIFEST.in might look like:

include src/mypkg/intf.f90

This places the files for pip install . under:

<python-install>/site-packages/mypkg.egg/mypkg/intf.f90

The Google Chrome and Microsoft Edge web browsers can be forced into “dark mode”. Good websites follow the OS system theme, but some websites need this forced dark mode.

This works on mobile and desktop. In the browser address bar, type:

chrome://flags/#enable-force-dark

There are several settings–usually “Enabled” works while leaving images alone. Most websites work correctly.

Windows enables WPA3 at the operating system level. The computer may also need device-specific firmware / software updates for WPA3. Some older hardware doesn’t connect to WPA3, even in transition mode.

Check enabled WiFi features:

netsh wlan show drivers

look for “802.11ax” under:

Radio types supported : 802.11b 802.11a 802.11g 802.11n 802.11ac 802.11ax

also look under “Authentication and cipher supported in infrastructure mode”.

–

Determine if the network is WPA3 from a Windows device that works on the network by:

netsh wlan show networks

Look for the SSID of the desired network. It should include a field “Authentication”.

Saving PDF of plots from Matlab is an effective way to use Matlab plots in LaTeX or for general sharing of high quality plots. Matlab exportgraphics, solved many of the previously long-standing issues with getting production-ready plots from Matlab. We also recommend tiledlayout for creating subplots. Use “exportgraphics” instead of the “print” and “saveas” functions, which are generally not recommended for saving plots.

Example:

fg = figure;
ax = axes('parent', fg);
plot(ax, 1:10)

exportgraphics(fg, 'line.pdf', 'Resolution', 300)

Matlab functions can detect if they are being executed in a Matlab Parallel Computing Toolbox parallel worker using is_parallel_worker.m. This can be useful to workaround parallel plotting bugs or for whatever reason distinct parallel logic might be desired in a function.

GitHub, GitLab, Bitbucket and similar services allow free, fast static websites under usage limits. Netlify is recommended for use with any static site generator (SSG). GitHub Pages can use GitHub Actions for Hugo. GitLab Pages may be used with any SSG.

GitHub Pages is noticeably easier to use than GitLab or Bitbucket Pages. GitLab runners are slow and build quota can run out before month’s end. Most should start with GitHub Pages for websites of any size. While we generally recommned Hugo over Jekyll, here’s how to setup a Jekyll website.

The Minimal Mistakes Jekyll template is one of numerous quick-loading Jekyll templates. Forget about AMP, get lightning-fast mobile browsing Google PageSpeed scores with Jekyll and Minimal Mistakes. This procedure is based on Linux (including Windows Subsystem for Linux).

Install prereqs:

apt install ruby-dev libssl-dev

gem update --system

Configure Ruby Gem install without sudo and install Gem bundler (without sudo):

gem install jekyll bundler

Download and extract latest Minimal Mistakes release. Install needed Gems:

mv minimal-mistakes username.github.io
cd username.github.io

bundle install

where username is your GitLab or GitHub username. On GitHub/GitLab, create a new blank repository username.github.io (for GitLab, username.gitlab.io).

Edit _config.yml, change the following lines to fit your needs: title, name, description, url, repository

Connect your new website to GitLab/GitHub (swapping gitlab for github as appropriate)

git init
git add .
git commit -am init
git remote add origin https://github.invalid/username/username.github.io
git push

Future edits will follow the usual

git commit -am foo
git push

The test page should be live at username.github.io. See Github Pages docs for custom domains and advanced configs.

Tips

Folders under _posts with filenames starting with date appear on the site. Subfolders under _posts are transparently processed. This is useful to organize posts by year for example, without affecting URL formatting.

Example filename: _posts/2018/2018-09-23-joes-big-vacation.md appears to the public with URL: https://username.github.io/joes-big-vacation

Enable search icon in _config.yml with key:

search: true

This enables site-wide Lunr instant search as the user types. The search icon is at the upper right corner of the toolbar on top of every page/post. It’s much better/faster than Google-based search of your site! This instant as-you-type search scales well for sites with thousands of pages.

Edit static navigation buttons in _data/navigation.yml. To improve default formatting, copy/paste into _config.yml these lines (anywhere in file):

defaults:
  -
    scope:
      path: ""

    values:
      layout: "single"
      toc: true
      author_profile: false
      read_time: false
      comments: true
      share: true
      related: true

include: ["_pages"]

To configure the banner, add to index.html header (between three dashes) the lines:

header:
    overlay_color: "#000"
    overlay_filter: "0.5"
    overlay_image: /images/header.jpg
excerpt: "text overlaid on banner image"

Remove the author image by:

author_profile: false

Remove category from permalinks: in case you later decide a page category should change, without screwing up your search engine results, in _config.yml:

permalink: /:title/

Control the number of posts per archive page in _config.yml:

paginate: 10 # amount of posts to show

Reference: Jekyll install reference

• GitHub Pages is substantially easier to setup and use, and is capable of medium websites getting several million hits / year
• GitLab Pages has more features and flexibility for advanced users

GitLab Pages quick setup

Create a new GitLab Project named username.gitlab.io (put the GitLab username in for “username”). If one already has a GitHub Pages website: Import from GitHub OR create/copy in your existing static website (if you had a GitHub Pages website, copy it here). If the latter, clone to the PC.

On the GitLab project page e.g. https://gitlab.com/username/username.gitlab.io click Set up CI Create the .gitlab-ci.yml under the apply a GitLab CI YAML template. If coming from GitHub Pages use GitHub Actions for Hugo.

The site is now building as seen with the Pipelines tab of the website project. It takes about 3-4 minutes to install the gems for a Jekyll site, then 2-3 more minutes to complete the build depending on the size of the website. The public URL should be like username.gitlab.io.

If a custom domain was purchased, tie to GitLab Pages by: Project Settings → Pages add TWO new domains

example.invalid
www.example.invalid

Transfer DNS can take the website down so do this at low traffic times. Once ready, setup/transfer DNS to GitLab. Suppose the domain is example.invalid, then set DNS records to

example.invalid CNAME username.gitlab.io
www             CNAME username.gitlab.io

assuming the DNS provider supports CNAME flattening.

SSL Config

GitLab Pages used with for example Cloudflare works well to provide HTTPS with your custom domain name as per this procedure. With that procedure enable SSL “Full (Strict)” security.

Free GitLab accounts have a monthly quota for build “pipeline” time. For a small to moderate size static website it should be enough. Save quota by canceling pipelines / runs for unneeded builds.

For frequently updated, medium sized websites (hundreds or thousands of pages) consider Netlify with GitLab Pages or GitHub Pages.

Both GitLab and GitHub allow the source files (e.g. Markdown) to be private for a public website. Consider a private website repo, otherwise Google may present search results from Markdown code before the actual webpage.

Notes

• For larger or active websites use Netlify, or build on laptop or cloud service like Wercker with any static generator such as Hugo and push HTML to GitHub Pages
• Useful Jekyll plugins that GitHub doesn’t allow include jekyll-archives (page per category/tag)
• GitLab Pages from scratch

Netlify works well with GitHub or GitLab, adding speed and reliability among other benefits