Scientific Computing

Matlab users can share code projects as toolboxes and/or packages. Matlab packages work for Matlab ≥ R2008a as well as GNU Octave. Matlab toolboxes work for Matlab ≥ R2016a and not GNU Octave. The packages format brings benefits to toolboxes as well.

Matlab namespaces: a key issue with Matlab vs. Python arise from that Matlab users often add many paths for their project. If any function names clash, there can be unexpected behavior as it’s not immediately clear which function is being used without further investigation of path ordering. As in Python and other languages, there is considerable benefit for using a package format where the function names are specified in their namespace.

addpath example: Matlab package format. Suppose project directory structure:

myproj
  utils
    mem1.m
  conversion
    deg1.m
  sys
    disk1.m

To use these functions, the end users do:

addpath(genpath('myproj'))

This is where the namespace can have clashes, and with large projects it’s not clear where a function is without further introspection.

package example: make this project a Matlab / Octave package by changing the subdirectories containing .m files to start with a “+” plus symbol:

myproj
  +utils
    mem1.m
  +conversion
    deg1.m
  +sys
    disk1.m

The end users simply:

addpath('myproj')

access specific functions like:

myproj.utils.mem1(arg1)

Then multiple subdirectories can have the same function name without clashing in the Matlab namespace. Suppose the function “mem1” is used frequently in another function. To avoid typing the fully resolved function name each time, use the import statement:

function myfunc()

import myproj.utils.mem1

mem1(arg1)

mem1(arg2)

Private functions: Matlab packages can have private functions that are only accessible from functions in that level of the namespace. Continuing the example from above, if we added function:

myproj
  +utils
    private
      mysecret.m

then only functions under +utils/ can see and use mysecret.m function. mysecret() is used directly, without import since it’s only visible to functions at that directory level.

Matlab .mltbx toolboxes became available in R2016a. The Matlab-proprietary toolbox format also allows end users to create their own packages containing code, examples and even graphical Apps. In effect .mltbx provides metadata and adds the package to the bottom of Matlab path upon installation. The installation directory is under (system specific)/MathWorks/MATLAB Add-Ons/Toolboxes/packageName. Whether or not the project uses .mltbx, the namespace of the project is kept cleaner by using a Matlab package layout.

Python raises ModuleNotFoundError when a Python module is not able to be found. This exception catches a much narrow range of faults than the parent exception ImportError.

Although we often like to make exception handling more specific, we have found as a practical matter that using ImportError in a try: except: exception handler is almost always the most appropriate choice. This is particularly true for modules like h5py that require compiled-language interfaces. For example, h5py relies on the HDF5 library. We have found a small percentage of systems with conflicting HDF5 library versions on the system path, which causes h5py to raise ImportError. In these cases, we usually wish to detect that the imported module is ready to work, not just whether it is found or not.

Example

For Python imports loading compiled-language modules, the following is generally recommended:

try:
    import h5py
except ImportError as e:
    h5py = None

def myfun():
    if h5py is None:
        raise ImportError(f"myfun() requires h5py, which failed to import with error {e}")
    # rest of myfun()

Many coding languages have objects that are useful as sentinels to indicate missing data or unused parameters. For example, if a boolean parameter’s state has not been checked, it’s a bit disingenuous to say available = false. Sentinel values give a state that can communicate that something is unknown or unset.

Some example sentinels:

• Julia: nothing
• Python: None

In Matlab an empty array can be used as a sentinel.

Create empty arrays for Matlab data types by appending .empty to the data type name. Examples:

datetime.empty
string.empty
struct.empty

The commonly used isempty() works for any Matlab type:

function out = myfun(cfg)
arguments
  cfg struct = struct.empty
end

if isempty(cfg)
  cfg.lims = [0,100];
end

Windows and macOS have few maintained versions due to their commercial nature. In contrast, FOSS operating systems like BSD and Linux have hundreds of maintained distros. Only a few Linux distros dominate such as Debian, Ubuntu and Red Hat. Less common Linux distros are commonly based on popular distros.

For certain system management tasks and install scripts it’s useful to programatically identify which major Linux distro family the current OS belongs to. A standard method to detect Linux operating system version is via plaintext /etc/os-release. Prior to this de facto standard, older Linux distros used other files.

The algorithm we use to identify older and newer Linux distros is to first check the file “/etc/os-release” versus known distros. If “/etc/os-release” is not present, the algorithm looks for OS-specific files:

To detect if running under MSYS2 Python, check the parameters described in this GitHub Issue.

We have created a Python program that retrieves and parses FCC ULS data. We provide example scripts for amateur radio and GMRS.

lynx text web browser can dump all URLs in a single web page like:

lynx -listonly -nonumbers -dump https://www.scivision.dev

This does not check the URLs. The -traversal option is HTTP-only, it does not work for HTTPS at least through Lynx 2.9.0.

For checking links, we recommend Python link checker.

These programs allow hosting or attending a live Web group conference. We considered factors including: large attendee count, live feedback, several speakers/presenters who can share their screen, and allowing others to draw on that screen or edit the document. There are many more options such as Talky, etc. These web conferencing methods work for Linux / macOS / Windows / Android / iOS.

High quality livestreaming HD broadcasts via an API requires just a bit of reasonable technical knowledge to accomplish. Google Meet only requires a web browser and plugin (or mobile app)–it’s simpler to get started and is a good business-grade choice.

According to Omnicore, 95% of global Web users visit YouTube. YouTube Live is free for live events with live user feedback and multiple speakers. YouTube Live is easy to stream using FFmpeg or OBS Studio. YouTube Live streaming is also possible directly from the web browser.

The Facebook Live Graph API makes it possible to live stream broadcast from a laptop. FFmpeg can stream to Facebook Live from a laptop.

Zoom currently allows limited free use. Paid plans have unlimited time with a large participant count. The Zoom client is available for many platforms, or can be used directly from the web browser app. The Zoom Linux client also works well.

GoToMeeting allows small free meetings, with a large participant count on paid plans. Web browsser GoToMeeting works well in the Google Chrome browser across operating systems.

WebEx web browser platform-independent conferencing works better than the installed WebEx app. WebEx limits participant count by pricing tier.

Google Meet has browser-based screen sharing / video group calls. Phone dialout / dialin is also available.

Discord can be used from an web browser, and allows thousands of simultaneous many-to-many voice and text chat users. Several speakers can share video/screen and audio as well. Discord is free to use with very low latency HD voice. Discord uses discontinous voice via voice activation or push-to-talk. Many-to-many conversations allows interrupting without pause/break.

Skype has plugin-free access via the web browser with a limited number of conference participants. Skype for Business has higher user count limits, but is not free.

Numpy ABIs change between releases, which can lead to errors upon

import numpy

like:

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

try fixing by upgrading Numpy.

conda install numpy

# or

pip install --upgrade numpy

For scientists, engineers and other performance-sensitive coders modern Fortran offers immediate advantages in developer time savings. The clarity, conciseness and power of modern Fortran are widely available in contemporary compilers. This brief post was motivated by viewpoints encountered including:

• those whose boss insisted on Fortran 77–they didn’t know anything newer than Fortran 90 existed.
• those who thought essentially no compilers supported newer than Fortran 95 standard (in calendar year 2018).

To be effective with programmer time, one generally shouldn’t needlessly upgrade all Fortran 77 code to modern Fortran, since Fortran has always maintained good backward compatibility. However, new and upgraded Fortran code should almost never be written in Fortran 77 unless specific job conditions dictate. Of course, Fortran 66 / Fortran IV is little supported and will need to be upgraded to Fortran 77 syntax, which is very similar except for file I/O.

New Fortran code should at least use Fortran 2003, which is universally supported by current compilers. In HPC environments, Gfortran and Intel oneAPI are widely supported, so we use modern Fortran features in virtually every program.

Modern Fortran support

The compilers easily available supporting modern Fortran include:

• Gfortran
• Intel oneAPI
• NAG
• LLVM Flang
• Nvidia HPC

Polyhedral benchmarks compilers supporting modern Fortran features.

What did the major Fortran versions add

• Fortran 95 brought strong N-dimensional array operations. It is a key step toward modern Fortran, enabling arbitrary size (elemental) intrinsic and non-intrinsic procedures. With Fortran 95, one no longer had to to explicitly loop over almost every array operation.
• Fortran 2003 brought polymorphism and true object-oriented procedures, critical parts of modern generic programming.
• Fortran 2008 strengthened polymorphism, and baked coarray (distributed parallelism) directly into Fortran, transparently using underlying libraries such as OpenMPI. Improved Fortran software architecture comes through submodule enabled by Fortran 2008.
• Fortran 2018 strengthened coarray support, and did further important language cleanup such as enabling error stop within pure procedures

Related: Gfortran feature set vs. OS and version