Scientific Computing

A bug I fixed for Numpy 1.19.0 is Numpy internally did not correctly compare two-digit version numbers. To use GCC ≥ 10 with Numpy upgrade Numpy to at least version 1.19.0.

The PDF version output by LaTeX is set by \pdfminorversion and this also sets the newest PDF version that LaTeX can use via \includegraphics. If \pdfminorversion default or setting is too low, by default LaTeX only issues a warning like:

PDF inclusion: found PDF version <1.7>, but at most version <1.5> allowed

and the PDF-input graphic will be missing. The too-old default PDF version of LaTeX is increasingly becoming an issue as PDF creation tools are rightfully increasing their default PDF output version from the ca. 2003 PDF v1.5 specification.

To mitigate these issues, we do two things near the top of each project’s main LaTeX file:

1. create an error if the current PDF version is too old

   \pdfinclusionerrorlevel=1

2. set the default PDF version to ca. 2008 PDF v1.7

   \pdfminorversion=7

It’s OK for these PDF configuration lines to come after LaTeX directives.

3-D mesh wiregrid: minimal working example for the current version of Matplotlib. Pan, rotate, etc. in the figure window that opens.

from matplotlib.pyplot import figure,show
import numpy as np

x,y = np.meshgrid(np.linspace(0,2*np.pi), np.linspace(0,2*np.pi))

z = np.sin(x+0.5*y)
ax = figure().gca(projection='3d')
ax.plot_wireframe(x,y,z)
show()

Windows users may not have permission to create a symbolic link by default. Symptoms from Python may result:

from pathlib import Path

Path('x').symlink_to('y')

OSError: symbolic link privilege not held

Fix by enabling Windows symbolic links by enabling Windows developer mode.

Restart computer, and once again try Python script above to see that Python can create symbolic links.

Group Policy: press Windows r key (run dialog) and type

gpedit.msc

If gpedit.msc is not available, use Windows in Developer Mode as above.

Navigate to create user permission to create symbolic links:

Computer Configuration → Windows Settings → Security Settings → Local Policies → User Rights Assignment → Create symbolic links

type the user name and click “Check Names” then OK.

Reboot the computer (or logoff, logon) to make the permissions take effect.

Notes:

• Python os.symlink() on Windows
• symbolic links on Windows

When files need to be shared between two computers, whether at office, home or conference, major operating systems have peer-to-peer file sharing built in.

Windows

Windows Nearby sharing uses Bluetooth only, which means file transfer speeds are a tiny fraction (far slower) than even 3G tethered internet. Windows Nearby has the advantage like other operating systems of not requiring internet, but the real-world file transfer throughput is on the order of 1 Mbps, due to the limitations of Bluetooth itself and interference from other 2.4 GHz devices, even when the two laptops are less than 3 meters apart.

Windows Nearby file sharing is generally not practical for files over about 10 megabytes, due to the transfer rate of about 100..200 kilobytes / second in real-world use. That is, a 10 megabyte file takes about 2 minutes to transfer using Windows Nearby.

Apple

Apple AirDrop is much faster than Windows Nearby since AirDrop uses Bluetooth to negotiate a peer-to-peer WiFi connection that doesn’t rely on WiFi AP (router). Depending on the RF congestion and the nature of files transferred (few large files faster than many small files) the data throughput of AirDrop can be in the 100..500 Mbps range, 100 times faster than Windows Nearby sharing. It would be very beneficial if Windows Nearby would use adhoc WiFi like AirDrop.

Android

The deprecated Android Beam used NFC to negotiate a Bluetooth connection, which was very slow like Windows Nearby. Android Nearby is reported to a possible feature in Android 11, ChromeOS and Chrome browsers. We will hope that Android Nearby will use WiFi like AirDrop for useful speed.

Linkchecker-markdown is a very fast Python module that checks internal and external website links using the raw Markdown before rendering by the static site generator. Avoid broken internal links by using a Git pre-commit hook to check all internal links upon each “git commit”.

In the top directory of the Markdown website repo, do:

git config core.hooksPath .git/hooks

add the file .git/hooks/pre-commit (no .py extension)

Related: global Git pre-commit for checking Python PEP8, etc.

Newer version of compilers generally have more useful and detailed warning messages. As with any compiler, newer versions of Gfortran may require rebuilding other libraries linked with the Fortran compiler if the ABI presented by libgfortran changes. On Linux, one can switch Gfortran versions with update-alternatives. On Ubuntu, first install the distro-packaged compiler:

apt update
apt install gfortran

If you need a newer GCC / GFortran than your Ubuntu release provides, try:

add-apt-repository universe

The Ubuntu toolchain test PPA is often the simplest way to get a newer GCC / GFortran on Ubuntu: Ubuntu Toolchain Test PPA

Add the PPA by:

add-apt-repository ppa:ubuntu-toolchain-r/test

apt update

Install the desired compiler version, for example:

apt install gfortran-15

Then select the compiler version you want to use. Switch between compiler versions with update-alternatives.

Related:

• Windows: Install latest Gfortran
• macOS: get latest gfortran by brew install gcc

Continuous integration for program that plot or need a display can be tricky, since in many cases the CI doesn’t have an X11 display server. Workarounds include generating plots using X server virtual framebuffer (Xvfb) dummy X11 display server. This maintains code coverage and may allow dumping plots to disk for further checks

GitHub Actions: “.github/workflows/ci.yml”: assuming the project uses PyTest, the xvfb-action enables Xvfb for that command:

- name: Run headless test
  uses: GabrielBB/xvfb-action
  with:
    run: pytest

Related: Detect CI via environment variable](/ci-detect-environment-variable)

GitHub Actions can use Conda to more easily install certain complicated Python packages via the setup-conda action.

For example to install Mayavi:

- uses: s-weigand/setup-conda

- run: conda install mayavi

Upload raw .wav WSJT-X data to the HamSci Zenodo data archive to help future data analysis. The location of the WSJT-X raw data is found by the WSJT-X menu: File → Open Log Directory. The raw data save location is typically:

• Windows: $Env:LocalAppData/WSJT-X/save
• Linux: ~/.local/share/WSJT-X/save
• macOS: ~/Library/Application Support/WSJT-X/save

To save the raw data, from the WSJT-X menu: Save → Save All. One .wav file is saved per two minute cycle. This setting is persistent.

Archive raw WSPR data for easier upload to HamSci Zenodo archive:

Upload raw data to Zenodo by creating a Zenodo account to upload WSPR data to Zenodo. Upon clicking “Publish” the data is assigned a DOI and is citable.

Tips:

• Avoid using a virtual machine for WSJT-X due to issues with broken/choppy audio.
• WSJT-X collects about 1.7 GByte/day depending on how often you transmit (no recording occurs when you transmit).
• raw audio data file size is: 12000 samples/sec * 16 bits/sample / 8 bits/byte * 86400 sec/day * 0.8 RX duty cycle = 1.7 GByte / day. That’s 2.88 Mbytes per 2 minute WSPR RX cycle.
• Since this is 6 kHz of spectrum, you can widen your receiver filters (particularly if using an SDR or other advanced receiver) to also pass JT65, FT8, or other useful transmitters for even more potent results that fall within the 12 kS/s sampling bandwidth.

The raw data .wav files are uncompressed PCM audio. “tar” is used to make one archive file instead of thousands of sound files per day. The files are full of noise, which by definition is poorly compressible.

Related: Load raw WSPR data for analysis