Scientific Computing

Matlab curl instead of websave

Matlab websave or ftp might not work in cases where a plain “curl” or “wget” command works. A symptom of this issue is HTML is downloaded instead of the intended binary file. Websites such as Dropbox recognize the HTTP User Agent of curl and Wget and mutate the web server response to be automation-friendly. Since Matlab is much less commonly used than Python, curl, Wget, etc. this user agent-dependent behavior results. We recommend understanding why Matlab websave doesn’t work, or use the low-level Matlab HTTP Interface.


To use curl from Matlab, recognize this may require unique setup for each system, despite that curl is included (preinstalled) in modern operating systems including Windows.

The extra quotes around “url” allow for arbitrary characters to be used in the URL that can confuse shells like zsh. The “-L” option to curl allows redirects.

function curlsave(filename, url)

cmd = "curl -L -o " + filename + " '" + url + "'";

assert(system(cmd) == 0, "download failed: " + url)

end

Linux systems with multiple curl versions installed may need to set an environment variable to prioritize. Set the filename as appropriate for computer (ensure the file exists).

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libcurl.so.4 matlab

This can help errors like

curl: (48) An unknown option was passed in to libcurl

Even this doesn’t always work, so we recommend understanding why Matlab websave doesn’t work, or use the low-level Matlab HTTP Interface.

CMake generate Fortran from template

CMake configure_file can serve to generate source code files including Fortran. For templates beyond a few lines of code, it may be useful to read the template from a file. This requires setting CMake directory property CMAKE_CONFIGURE_DEPENDS to have CMake regenerate the file if the template file changes as in the example below.

file(READ my_template.in.f90 template_code)
configure_file(example.in.f90 example.f90 @ONLY)

set_property(DIRECTORY PROPERTY CMAKE_CONFIGURE_DEPENDS my_template.in.f90)

#example use
add_executable(main ${CMAKE_CURRENT_BINARY_DIR}/example.f90)

iftop config file

iftop is a handy utility on macOS, Linux and other Unices for a live Terminal graph of network data flow to particular addresses. On computers with many network interfaces, including virtual interfaces such as on macOS, it is handy to set a default interface in a config file. iftop uses the file ~/.iftoprc.

For example, on macOS you may be interested in interface “en1”. To help determine the desired interface, use ifconfig or ip a to find the interface with the public IP address. Then create ~/.iftoprc containing like:

interface: en1

where “en1” is your desired interface determined as per above.

Uniden PC 78 models

The Uniden PC 78 model family evolved from the Uniden PC 76 XL of the 1990s. The Uniden PC 78 XL added Dynamic Squelch Control DSC. In 1999 the Uniden PC 78 LTW brought full-time WX alert and backlit controls. In 2003 the Uniden PC 78 Elite added WX alert even if power knob off and backlit panel indicators.

Uniden DSC didn’t work so well because it did not have an adjustable threshold, and like virtually all noise-based auto-squelches can drop out undesirably on overmodulated signals. The PC 78 LTX came out in 2018, and dropped DSC. The Uniden PC 78 LTX also brought a new circuit board using surface-mount components instead of the through-hole construction common to older CB radio designs from the pre-2010 era. Today’s CB radio buyer should steer toward radios having digital noise reduction / DSP, which is a much more effective way to reduce noise and improve audio quality than the old noise-based auto-squelches.

Red Hat firewalld port add

RHEL uses firewalld to provide network firewall. firewalld has the concept of runtime vs. permanent rules, which help avoid getting the firewall into an unusable state. Permanent rules become live at next restart/reboot, while runtime rules disappear at restart/reboot.

Suppose one wishes to put the SSH server on a non-default port 12345 to mitigate auth log clutter. First configure the SSH server in /etc/ssh/sshd_config, then restart SSH and verify the SSH configuration is working by adding the port to firewalld (here, 12345):

firewall-cmd --add-port=12345/tcp

If this works, make the firewalld rule permanent:

firewall-cmd --permanent --add-port=12345/tcp

SELinux will also need an updated policy to allow the SSH port change, like:

semanage port -a -t ssh_port_t -p tcp 12345

CMake expanduser tilde ~

To have the most reliable path operations in CMake, it’s typically best to resolve paths to the full expanded path. Note: there are a few CMake functions that desire relative paths, but those are clearly spelled out in the docs.

expanduser.cmake for CMake expands the ~ tilde character to the user home directory on all operating systems, including Windows.


Related:

List all CMake tests with CTest

As a CMake project grows, increasing complexity can make it hard to discern what tests are to be run and their properties. Perhaps the project logic is unexpectedly omitting necessary tests. The CI system or human can verify the list of tests by:

ctest -N

For machine parsing and human-readable verbose details including fixtures and labels, output JSON:

ctest --show-only=json-v1

To ensure an accurate test list, the project must first be configured and built as usual:

cmake -B build

cmake --build build

ctest --test-dir build -N

CMake file separator

In many cases, using the Unix-type slash file separator / will work even on Windows. Trying to manually specify backslash Windows file separator \ can cause problems in CMake and many other languages. Thus in CMake and other languages like Python, we always use / as path separator, and only transform to backslash for the rare cases it’s needed.

Transform to Unix file separator:

cmake_path(CONVERT "path\in\file.txt" TO_CMAKE_PATH_LIST out)

That switches backslash \ file separators to Unix slash / file separators. This becomes relevant if manually adjusting Include paths by appending to lib_INCLUDE_DIRS or similar. If backslashes sneak through, unexpected build-time errors can result, and even configure-time errors with “check_source_compiles()” and similar. As the docs note, put quotes "${mypath}" around the variable expansion to ensure CMake doesn’t mangle the path.

Transform to native file separator is generally more rarely used. CMake can transform paths to native file separator, with the caveat that this can cause unpredictable Windows-specific backslash problems, as with any program.


Related: CMake expanduser ~

Upgrade Anaconda for latest Python

Note: it may be necessary to reinstall Miniconda from scratch if packages break during a Python version upgrade. Consider this before attempting an in-place Python upgrade. There is often a couple month delay between a major Python release and Anaconda defaulting to the new version.

Use the new Python version in a new conda environment by:

conda create -n py3x python=3.x

switch to this environment by

conda activate py3x

Legacy hard-coded GUIs using external libraries have considerable overhead to maintain, and suffer bit rot far faster than the underlying code. At the very least, be sure your code is usable from the command line and/or as a plain importable module without requiring a graphics library. In stripped-down cloud / embedded applications, unused graphical imports cause stability problems and considerable startup lag.

Force integer axis label ticks on Matplotlib

To make a Matlabplot figure axis have integer-only label ticks, use method like:

ax.yaxis.set_major_locator(MaxNLocator(integer=True))
# or
ax.xaxis.set_major_locator(MaxNLocator(integer=True))

A complete standalone example follows:

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.ticker import MaxNLocator

x = np.arange(0.1,10.5,0.1) # arbitrary data

fg = plt.figure(layout='constrained')
ax = fg.gca()
ax.plot(x)

ax.yaxis.set_major_locator(MaxNLocator(integer=True))

plt.show()

If too few ticks are displayed, as per the Matplotlib MaxNLocator, you must have “at least min_n_ticks integers…found within the view limits.” Set “MaxNLocator(nbins=5,integer=True)” or similar if the defaults aren’t forcing integer ticks.