Scientific Computing

Beggar Barons of Software

Zed Shaw coins the term Beggar Barons in his blog post on being a popular software author begged to make significant changes to software to accommodate the large company’s needs. The recent situation involving Python vis-à-vis the Apple App Store reminds us of this phenomenon via Zed’s article. Python with iOS or Android requires using libpython in a compiled application, rather than the more typical terminal use–although there are apps that bring that experience to mobile devices as well.

Datetime vectors in Matlab / Octave

Generating a range of datetime data is a common data analysis and simulation task across programming languages. Matlab and GNU Octave can also generate datetime vectors.

Matlab datetime deprecates datenum. Generate a sequence of datetime like:

t0 = datetime('2020-01-05 12:30:00');
t1 = datetime('2020-01-06 18:15:10');
dt = hours(5.5);

t = t0:dt:t1;

disp(t)

GNU Octave can use many datetime features via the tablicious package.

pkg install tablicious

Load in Octave prompt:

pkg load tablicious

Then use the same Matlab code above.

No strict aliasing C / C++

Optimizing compilers may enable strict aliasing. For a wide variety of existing projects, strict aliasing provides additional optimization. For some projects, such as MUMPS, memory leaks have been observed that are resolved by disabling strict aliasing using GCC flag “-fno-strict-aliasing”.

Compilers with ability to switch on / off strict aliasing include:

libuv recommends -fno-strict-aliasing due to type punning.

References:

Captive portal public Wi-Fi URLs

tl;dr: When connected to a captive portal, try visiting http://neverssl.com to trigger the portal.


Public WiFi often has captive portal login. Portals may coerce users to accept terms and conditions and absolution of liability before accessing the Internet. The portals may be worked around by DNS tunneling, MAC spoofing, etc. Many users just tolerate the portals.

Web browsers may try to trigger captive portals by checking servers, in case the OS hasn’t already triggered the captive portal. Sometimes captive portals aren’t triggered. HSTS blocks HTTP captive portal redirects. Try visiting a deliberately non-HTTPS portal-triggering site like http://neverssl.com

Devices on networks that block the platform’s automatic network checking may indicate like:

Connected, no Internet

If there’s not a captive sign-in webpage, the network connection may still work to non-Google sites.

Check connectivity manually using curl like:

curl -I -w "%{response_code}" http://neverssl.com
-I
show header only
-w "%{response_code}"
write the HTTP response code to console

Use cURL to check connectivity through captive portals

Return code 200:

  • Firefox
    curl -I -w "%{response_code}" http://detectportal.firefox.com/success.txt
  • Windows
    curl -I -w "%{response_code}" http://www.msftconnecttest.com/connecttest.txt
  • macOS
    curl -I -w "%{response_code}" http://captive.apple.com/hotspot-detect.html
  • Fedora
    curl -I -w "%{response_code}" http://fedoraproject.org/static/hotspot.txt

Return code 204:

  • Google
    curl -I -w "%{response_code}" http://connectivitycheck.gstatic.com/generate_204
  • Android
    curl -I -w "%{response_code}" http://connectivitycheck.android.com/generate_204
  • Ubuntu
    curl -I -w "%{response_code}" http://connectivity-check.ubuntu.com/generate_204

Python script to check connectivity through captive portals

Using Python requests, one can script testing of URLs with friendly feedback.

python ./captive_portal.py

Firefox: http://detectportal.firefox.com/success.txt returned 200
Windows: http://www.msftconnecttest.com/connecttest.txt returned 200
macOS: http://captive.apple.com/hotspot-detect.html returned 200
Fedora: http://fedoraproject.org/static/hotspot.txt returned 200
Google: http://connectivitycheck.gstatic.com/generate_204 returned 204
Android: http://connectivitycheck.android.com/generate_204 returned 204
Ubuntu: http://connectivity-check.ubuntu.com/generate_204 returned 204

Cobra 19 Ultra 6 CB radio

The Cobra 19 Ultra 6 appears similar to (but has different firmware from) radios like

The importance of the firmware difference is that the Cobra 19 Ultra 6 has a more limited feature set, and jumpering either of the white outlined jumper holes that the Anytone or Albrecht radios use for expansion next to the CPU has no effect on the Cobra 19 Ultra 6. Users should instead consider the Radioddity CS-47 over the Cobra 19 Ultra 6 for expandibility and key features including NRC noise reduction and CTCSS / DCS coded squelch that the Cobra 19 Ultra 6 lacks.

From practical in-vehicle use, the speaker is adequately loud. However, there doesn’t appear to be a noise blanker (NB), which means ignition noise may significantly degrade reception in some vehicles. There is an automatic noise limiter (ANL) that helps with some other types of noise, but the missing noise blanker and lack of digital noise reduction (DNR/NRC) means that the Cobra 19 Ultra 6 is not as good a value as more modern radios in the same price range. There is no “dual watch” channel capability, but this may not be a requirement for many users.

RF Gain is essential on any CB radio and the RF gain adjustment range of the Cobra 19 Ultra 6 is good. Typically RF gain between 9 to 30 (bigger numbers if interference increases) is useful to reduce background noise without reducing receive range too much.

The radio appears to use audio compandoring, which helps improve apparent audio SNR. The radio has a traditional signal squelch as well as “auto” noise squelch. The general problem with noise squelch across CB radios is that overmodulated AM signals can close the squelch regardless of how strong the signal is.

Setting the RF Gain and Squelch for any AM CB radio is best done by:

  1. open the squelch – static is heard.
  2. set the RF Gain so that faint static and/or skip signals are heard.
  3. set the signal squelch just until closed.

Channel Scan

On firmware version 1.1, entering scan mode is different than the user manual states. To enter channel scan, press and hold the microphone “up” or “down” button until the radio goes once through all 40 channels, then the radio beeps and release the button. The radio then scans all 40 channels repeatedly. Unfortunately, there does not appear to be a way to skip channels in scan mode. This means the radio will often stop on channels 6, 11, etc. with high power stations that may not be what the user wants to listen to.

Matlab read FITS image stacks

In Matlab, fitsread is used to read specific frame(s) from a FITS file. Read frames one at a time from a large multi-frame FITS file in MATLAB. This avoids overwhelming RAM or taking an excessive time to load just one or a few image frames from a FITS file.

Example: sequentially read and plot each frame of a 4096-frame FITS file, with each frame being 256 x 256 pixels.

for i = 1:4096
  currFrame = fitsread('myFile.fits', PixelRegion={[1 256],[1 256],i});
  imagesc(currFrame)
  pause(0.05)
end

GNU Octave FITS reading is done with the cfitsio package using a similar API.


Related: read FITS image stack in Python

Install AMD AOCC C C++ Fortran compiler

The no-cost AMD AOCC compiler is tuned for AMD CPUs, akin to how Intel oneAPI is tuned for Intel CPUs. For a RHEL-based Linux, install the downloaded “.rpm” file like:

dnf install ./aocc-compiler*.x86_64.rpm

Create a source script “~/aocc.sh” like:

root=/opt/AMD/aocc-compiler-5.0.0/

[[ ! -d ${root} ]] && echo "ERROR: ${root} not found" && exit 1

source ${root}/setenv_AOCC.sh

# setenv_AOCC doesn't set these
export CC=clang CXX=clang++ FC=flang

export CXXFLAGS=
export MPI_ROOT=

flang --version

To use the AOCC compiler:

source ~/aocc.sh

AOCL libraries

The AOCL libraries provide accelerated math functions for AMD CPUs for several popular libraries for use in C, C++, or Fortran. AOCL is compatible with AOCC and other compilers as listed in the AOCL downloads.

7-zip LZMA file compression

7-zip is a popular LZMA cross-platform compression algorithm. Install 7zip by package manager. For example, on Windows:

winget install --id=7zip.7zip

Examples:

  • Compress: 7z a big.7z big.dat
  • Extract: 7z e big.7z
-mmt=on
enable multithreading (far faster compression)
-m0=lzma2
enable LZMA2 compression (more advanced)
-mx=3
like “fast” compression.

Example 7-zip for very large files (100s of gigabytes or terabyte-size files)

7z a -t7z -mx=3 -mmt=on -m0=lzma2  mycomp.7z *.txt

Higher compression settings might not add a lot of compression but can take vastly longer. LZIP advantages vs. 7-zip.

Android 5G SA no data

5G wireless network providers across the United States and elsewhere are turning on 5G standalone mode (SA), that can enable significantly more data bandwidth and lower connection latency. Some phones, even recent phones, are not properly capable of supporting 5G SA even though they work fine in 5G NSA (non-standalone mode) that operates like 4G LTE. Recent Android releases and carrier updates have disabled 5G SA for certain phones that previously had 5G SA enabled due to data connectivity problems when the carrier enabled 5G SA. A symptom of lost data connectivity, possibly due to 5G SA incompatibility, is the phone signal showing 5G signal bars with an exclamation point.

What may possibly but not always help is to turn off 5G in the phone network settings by setting “Preferred network type” to “LTE” or similar instead of “5G”. This may force the phone to use 4G LTE instead of 5G, which may work better for data connectivity. However this does not always help. If a newer phone on the same network carrier works fine with 5G SA, then the phone with data connectivity problems may need a software update or carrier update to fix the problem, or simply a newer phone may be needed. The Network Cell Info app can show the network type and signal strength, and may help diagnose the problem.

CMAKE_TLS_VERIFY global

TLS verification default is ON since CMake 3.31. Users can override this default for all projects with environment variable CMAKE_TLS_VERIFY. or per-project with CMake variable CMAKE_TLS_VERIFY. The default TLS version may be set by CMAKE_TLS_VERSION. If the system TLS certificate location needs to be specified, this can be done by CMAKE_TLS_CAINFO.

Meson build system uses TLS verification by default, warning if verification fails. TLS verification is part of CMake’s internal nightly testing.

The example uses badssl.com, that purposefully has a variety of certificate problem URLs.


Reference: Issues that would have been caught with this default