Scientific Computing

The Matlab -batch command line option makes running Matlab code from the command line more reliable. However, Matlab is not really designed to be used extensively by end users this way. Matlab does not have a command-line parser like many other languages (scripted and compiled) do.

Python has the argparse module that we generally find sufficient. In Fortran, using the get_command_argument() with select case can make a straightforward command line parser without resorting to external packages.

Perhaps a reason for Matlab not having these features is the startup time of Matlab. While Python can complete a small program in 100 ms from the shell, Matlab takes 10-30 seconds or so depending on the number of toolboxes and the speed of the computer. GNU Octave starts up nearly as quickly as Python when Octave toolboxes are not auto-loaded. It would be nice if Matlab someday will make an actual command-line interface with fast startup time.

Many CI services use Ubuntu Linux images. The default Linux / Ubuntu version is typically an older version. It can be convenient to switch CI jobs to a newer Linux version to get more recent libraries. Here are examples of switching Linux version for a few popular CI services.

GitHub Actions default Ubuntu version is controlled with .github/workflows/*.yml file(s).

jobs:

  linux:
    runs-on: ubuntu-<version_number>

Code profiling and micro-benchmarks can be useful to discover hot spots in code that might be targets for improving efficiency and overall execution time. Matlab has long had an effective profiler built in from the factory.

Commercial and free software often prioritized quality and correctness over speed as part of a good quality software product. There are cases where the Matlab factory functions are simply not optimized due to development time constraints. Yair Altman has expertise with the undocumented scripts inside Matlab that many Matlab users rely on. He has written a three-part series on profiling and optimizing Matlab factory code, which I find to be worthwhile reading even for general Matlab code optimization.

Speeding up builtin Matlab functions:

• Part 1
• Part 2
• Part 3

Homebrew is a popular framework for quickly installing development tools on macOS, including Gfortran. macOS cloud services such as MacInCloud may say they provide Homebrew, but you may not be able to install packages without a sudo/admin account. To install Homebrew without sudo/admin in the user home directory for cloud or physical hardware, follow these steps:

mkdir ~/homebrew && curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C ~/homebrew

If one truly doesn’t have sudo / admin access as typical with a managed (less-expensive) cloud macOS plan, and if Xcode is not the appropriate version, GCC may compile from source, which can take tens of minutes. This may occur when doing

brew install gcc

Reference: Homebrew without sudo script

Running compiled M-scripts using Matlab Compiler deploytool may result in warning messages about tbb.dll. Updating the Threaded Building Blocks library may help. Also consider updating the Matlab version if applicable.

Download and install latest Intel oneTBB. If using a command prompt, make oneTBB available by command tbbvars. Recompile with Matlab deploytool.

MyPy is a Python type annotation checker. MyPy recursively checks all files in a Python project by typing:

mypy ~/myproject

We typically use the PyPI MyPy instead of conda to have the most recent MyPy version:

pip install mypy

It’s often useful to have a per-project MyPy configuration file to avoid excessive command line options. Configure MyPy defaults in pyproject.toml:

[tool.mypy]
files = ["src", "Examples"]
ignore_missing_imports = true
strict_optional = false
show_column_numbers = true

Where “files” is set appropriately for the project. Making a per-project files is strongly recommended to ensure files aren’t missed in the type check. One can make a system-wide ~/.mypy.ini, that is overridden by the per-project pyproject.toml.

Enhanced mypy usage

We often use executables from Python with data transfer via:

• stdin/stdout (small transfers, less than a megabyte)
• temporary files (arbitrarily large data)

This provides a language-agnostic interface that we can use from other scripted languages like Matlab or Julia, future-proofing efforts at the price of some runtime efficiency due to the out-of-core data transfer.

Here is a snipping we use to compile a single C code executable from Python (from GeoRINEX program):

"""
save this in say src/mypkg/build.py
then from the code that needs the output executable, say "myprog.bin":

from .build import build
exe = "myprog.bin"
...
if not exe.is_file():
    build("src/myprog.c")
# code that passes data via stdin/stdout and/or files using subprocess.run()

"""
import subprocess
import shutil
from pathlib import Path


def build(src: Path, cc: str = None) -> int:
    """

    Parameters
    ----------

    src: pathlib.Path
        path to single C source file.
    cc: str, optional
        desired compiler path or name

    Returns
    -------

    ret: int
        return code from compiler (0 is success)
    """
    if cc:
        return do_compile(cc, src)

    compilers = ["cc", "gcc", "clang", "icx", "clang-cl"]
    ret = 1
    for cc in compilers:
        if shutil.which(cc):
            ret = do_compile(cc, src)
            if ret == 0:
                break

    return ret


def do_compile(cc: str, src: Path) -> int:
    if not src.is_file():
        raise FileNotFoundError(src)

    if cc.endswith("cl"):  # msvc-like
        cmd = [cc, str(src), f"/Fe:{src.parent}"]
    else:
        cmd = [cc, str(src), "-O2", f"-o{src.with_suffix('.bin')}"]

    ret = subprocess.run(cmd).returncode

    return ret

LTE smartwatches may get up to 90% of the communications range of a smartphone. Most providers have turned off (or are turning off) 2G and 3G so coverage may be dynamic. Generally Bluetooth headsets can be used with LTE smartwatches, which helps call quality for any phone device.

Mobile devices including smartwatches may switch frequency bands when going from idle to phone call or data usage:

E

2G EDGE, the oldest digital network mode still in use, very slow.

H

3G HSPA/HSPA+, good enough for basic web browsing and email.

4G

really good 3G. Carriers may call their upgraded 3G networks 4G.

LTE

actually using 4G LTE.

5G

not necessarily faster than LTE when in NSA (non-standalone mode), but can be much faster in SA (standalone mode).

The signal bars may jump up/down a few notches when going from idle to active due to the phone band switching e.g. 700 MHz vs. 1900 MHz. Apps like Network Cell Info can help reveal these behaviors.

For continuous integration, it’s important to test the traditional package install

pip install .

along with the more commonly used in situ pip development mode

pip install -e .

Otherwise, the Python package install may depend on files not included in the MANIFEST.in file and fail for most end users who don’t use “pip install -e” option.

A particular failure this will catch on Windows CI is graft path/to/ where the trailing / will fail on Windows only.

Python psutil allows accessing numerous aspects of system parameters, including CPU count. We recommend using a recent version of PSutil to cover more computing platforms.

Ncpu = psutil.cpu_count(logical=False)

usually gives the physical CPU count.

PSutil uses Python script and compiled C code to determine CPU count–it’s not just a simple Python script.

Related: Matlab CPU count