Scientific Computing

DOSBox-X USB to Serial RS-232 Port Setup

DOSBox-X uses configuration file dosbox-x.conf for numerous parameters, including for serial port access via USB to serial adapters.

First, identify the device ID for USB to serial adapter:

  • Linux: ls /dev/ttyUSB* or ls /dev/ttyACM*
  • macOS: ls /dev/cu.* or ls /dev/tty.*

On Linux, allow non-sudo serial port access by adding the username to “plugdev” and “dialout” groups, then logout and login.

adduser $(whoami) plugdev dialout

Locate and edit dosbox-x.conf. Under [serial] modify the serial1=dummy according to the device found above. For example on Linux, if the device is at “/dev/ttyUSB0”:

serial1=directserial realport:ttyUSB0

DOSBox-X would present the serial device on COM1 as seen from within DOSBox-X.

Troubleshooting

On Linux, if trouble accessing a device, ensure the username is in “plugdev” group as noted above, and perhaps try (assuming device is /dev/ttyUSB0):

chmod 666 /dev/ttyUSB0

Matlab Startup Accelerator for Windows

Matlab Startup Accelerator for Windows is intended to speed up Matlab startup. However, by default Matlab disables the Startup Accelerator on many systems. To check if Matlab Startup Accelerator is enabled on a Windows system, check Windows Task Scheduler by the snapin:

taskschd.msc

Look under Task Scheduler Library for a task named Matlab R20XXx Startup Accelerator (where 20XXx is the Matlab release). Consider disabling it to avoid needless resource usage if it is not already disabled. On modern computers with SSD drives, Matlab startup is already fast.

Python os.statx for Linux

Python 3.15 added os.statx() function that provides access to the high performance statx() system call available on Linux kernels 4.11 and newer with glibc 2.28 and newer. The Ffilesystem library uses statx() by default if available on Linux. Having os.statx() available in Python allows for easier prototyping of features that might use statx(). See Ffilesystem statx() code for usage examples with graceful fallback to stat().

Minicom serial comms with file transfer

The Minicom serial communication program allows connecting to devices over the serial port and transferring files using XMODEM, YMODEM, or ZMODEM protocols–PuTTY can’t currently do file transfer over serial links.

Install Minicom like:

  • macOS: brew install minicom lrzsz (lrzsz is for file transfer)
  • Linux: apt install minicom lrzsz (or similar for relevant package manager)
  • Windows: use WSL (Windows Subsystem for Linux)

On macOS in particular for file transfers, first be sure that Minicom can find the sz, rz commands by looking under Esc-O for “File transfer protocols”. It may be necessary to point to lsz and lrz installed by Homebrew.

As noted in the documentation for Minicom, it is possible to save profiles. To just use a device directory, set command line options like:

minicom -D /dev/tty.usbserial-0001 -b 115200

List the USB-serial adapter ports:

  • Linux: ls /dev/ttyUSB*
  • macOS: ls /dev/tty.usbserial*

To send a file from Minicom once the device is ready, press Esc-Z and then S for send, select the appropriate protocol.

Python asyncio.run boilerplate

Concurrency is built into Python via asyncio. AsyncIO generators are implemented with yield much like synchronous generators. async for also simplifies expression of asynchronous for loops.

As in Julia, the expression of asynchronous structures in Python does not implement concurrent execution. Concurrent execution in Python is governed by collections of tasks or futures such as asyncio.gather and initiated by a runner such as asyncio.run.

Debugging asyncio code is facilitated by introspection built into Python.

asyncio.subprocess

AsyncIO subprocess may need specific asyncio loop configuration. The options needed are not the same for every project, depending on the asynchronous functions used.

Example date_coro.py uses AsyncIO subprocess.

asyncio.open_connection

For networking apps asyncio.open_connection allows massive amounts of connection, as shown in findssh.

Import Python user modules in Matlab

Matlab can call user Python modules via the Matlab external language interface. For concurrent Python modules using asyncio, you may need to create a shim function to allow Matlab to call the Python module. Anaconda Python works fine from Matlab.

Configure Matlab for Python: Matlab is designed to work with specific Python versions for each Matlab version. Matlab will not specifically tell you when you’re using an incompatible Python version, but you may get unstable operation or errors. pyenv Python environment manager configures the Python interface.

The Python executable is found from Terminal / Command Prompt:

python -c "import sys; print(sys.executable)"

For Windows, it may be like C:/Miniconda3/python.exe and for macOS / Linux it may be like ~/Miniconda3/python.

This Matlab command is persistent–Matlab remembers this Python choice even after restarting Matlab.

pyenv(Version='C:/Miniconda3/python.exe')

Verify Matlab → Python config by typing pyenv from within Matlab.

Troubleshooting

If a non-supported Python version is used, upon the pyenv() command Matlab may show a warning like:

Warning: Python version 3.xx is not supported. See this topic.

In that case, some Python operations may work, but others may not. An example test that may fail is:

py.tuple([1,1]);

PythonError ImportError: PyCapsule_Import could not import module "libmwbuffer"'

Numpy

If only Python standard library modules work, and even commonly used modules like Numpy will not work, errors may occur from Matlab like:

py.numpy.arange(1)
Unable to resolve the name py.numpy.arange

Try diagnosing the issue from Matlab like:

pyenv

help('py.numpy')

Verify the desired Python install is being used, and that there isn’t an error as below. If such an error occurs, check the PATH as seen by Matlab by:

getenv('PATH')

If the Python directories are missing, this may be why user modules are not importable in Python. These paths can be added to Matlab. On Windows, the necessary path is like ~/Miniconda3/Library/bin, the directory with lots of *.{so,dylib,dll} files and the fix is like:

setenv('PATH', fullfile(getenv("USERPROFILE"), "Miniconda3/Library/bin") + pathsep, getenv('PATH'))

The install path on non-Windows OS is like ~/miniconda3/lib.

This command can be issued after a Python command was attempted, it’s not necessary to restart Matlab. Note that this needs to be in PATH, since PYTHONPATH does not help in this case. This change does not persist across Matlab sessions. If it works for you, put it in the ~/Documents/MATLAB/startup.m file to have it reload each time Matlab is started.

This Numpy problem is fixed by the procedure above:

problem in numpy - ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
  1. Check that you expected to use Python3.7 from "C:/Miniconda3/python.exe",
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy version "1.17.4" you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.

Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.

Original error was: DLL load failed: The specified module could not be found.
  • Python module import is implicit in the Matlab → Python module function call. There is no need to import numpy etc. from Matlab
  • Python executable choice persists across Matlab sessions–Matlab “remembers” even after you restart Matlab or reboot the computer.
  • editing imported Python module code requires restarting Matlab to take effect.

WMIC.exe removed from Windows

Starting with Windows 25H2, WMIC.exe (Windows Management Instrumentation Command-line) has been removed from the operating system. Microsoft recommends using PowerShell cmdlets for managing and querying system information instead. There is a migration guide available to help users transition from WMIC to PowerShell cmdlets that invoke WMI.

There are a lot of .bat batch scripts and .ps1 PowerShell scripts that use WMIC.exe, but they can be updated to use PowerShell cmdlets instead.

CMake / Meson force shared library

Build systems like CMake and Meson have specific syntax and variables to manage shared library creation and linking. By default, CMake builds static libraries, but this can be changed by setting the BUILD_SHARED_LIBS variable to ON or by setting the SHARED type option of the add_library() command.

To use C, C++, Fortran, etc. library binaries from Python, shared libraries are required to load the compiled code at runtime in Python. Of course, the Python code must be configured to match the symbols in the shared library.


Meson builds shared libraries by default, but this can be changed by setting the default_library option to static in project() or by the shared_library() function.

Strip geotag location from photos

Strip most EXIF and IPTC headers from image:

exiftool -all= -photoshop:all= myphoto.jpg

A copy of the image _original is kept. The image is not recompressed so image quality is not affected.


Compare the metadata present on old and cleaned images using ImageMagick:

magick identify -verbose myphoto.jpg

magick identify -verbose myphoto.jpg_original

Not all headers can be removed, but it usually works.


Install Exiftool:

  • Linux: apt install libimage-exiftool-perl
  • macOS: brew install exiftool
  • Windows installer