Scientific Computing

Those running a public web server are frequently subject to bot scans. While using best practices is the proper approach, there may still be a lot of 404 log results from bots. Declutter the log by redirecting bot URLs to a single zero-byte empty page while still returning code 404.

Another choice is to return HTTP status codes. Netlify only allows a few _redirects HTTP status codes. We redirect bot URLs to /bot.html and return a 404 code using an empty file.

Empty files: since these files are legitimate, but may not be used by your site, we create empty files for these:

favicon.ico
ads.txt
app-ads.txt

The Netlify _redirects file should be in the site publish directory. For Hugo it may be “public/_redirects”. We prefer the _redirects file to netlify.toml, because the redirect syntax is much more concise in _redirects files. Use Netlify Playground to instantly test file syntax.

A _redirects file for a Hugo blog may need editing to be suitable for your site.

References:

• honeypot bot report
• brute force URL lists

The Pixel 4 and Pixel 5 have multi-frequency GPS L1 and L5 bands for substantially increased navigation accuracy, as well as increased usefulness in citizen science GNSS projects. Dual frequency GNSS receivers enable additional types of ionospheric measurements not readily available with single frequency L1 GNSS receivers.

The Google Pixel 4 has been noted as having a key flaw–battery life too often less than 12 hours. Even disabling features such as Project Soli / Motion Sense and having the latest Android 11 did not greatly increase battery life beyond 18 hours. Short battery life was historically also quite an issue across numerous iPhone models, but is no longer such an issue.

The Pixel 5 uses a high-midrange CPU instead of top-end CPU in Pixel 4–the Pixel 5 CPU seems nearly as speedy as the Pixel 4.

The display panel itself is used for the top speaker–this restricts fidelity of the speaker, making it greatly emphasize midrange frequencies. This is fine for voice, but if you wish to listen to stereo orchestra music, you’d benefit from headphones.

The Pixel 5 does not have the “squeeze for Google Assistant” or Motion Sense.

The baseline battery life in the Pixel 5 is about 24 hours, which can be pushed to 36+ hours with battery saver–much better than Pixel 4. All in all, even if a used Pixel 4 is substantially cheaper, go for a used (or new) Pixel 5 instead.

Reference: dual frequency GNSS phone model spreadsheet

We don’t use favicon.ico because the .ico format is uncompressed ~ 16 kB. Instead we suggest creating a compressed favicon that works in place of favicon.ico for all web browsers we tried. These files are typically 1..2 kB instead of 16 kB, which is significant for a text-based website.

favicon.png
favicon-16x16.png
favicon-32x32.png

Then we create an empty file favicon.ico that web browsers will ignore, and use the PNG instead.

Bots scan a set of URLs that can clutter the site 404 log. One can simply ignore these items, apply redirects, return HTTP codes, or create empty files. Creating a lot of empty files can be a bit awkward. Redirection is a server / service specific configuration that can possibly return HTTP error codes. We choose to redirect bots to an empty file.

Pytest allows clean handling of numerous continuous integration challenges. Many programs use advanced plotting that may not be desired to test on continuous integration. For example, one might opt not to install matplotlib as part of CI runs. We put plotting routines in a separate file to facilitate this, allowing the core module functions to have a bare minimum set of prereqs. This is useful for HPC, CI and cloud scenarios that either may not plot locally or that may use varied plotting systems depending on whether running on web or local PC.

pytest.importorskip() is a vital tool to handle to cleanly and clearly handle these scenarios.

Imagine a Python package file hierarchy like:

mymod/
  __init__.py
  plot.py
  adv.py
  tests/
    test_mod.py
    test_plots.py

Skipping function imports is demonstrated in this example. Suppose optional module mymod.adv() requires scikit-image, which itself has a lot of prereqs. One might opt to have CI not install scikit-image, and then use pytest.importorskip() to cleanly skip complicated, optional tests on CI, HPC etc.

import pytest
import mymod


def test_advanced():

    adv = pytest.importorskip('mymod.adv')

    ...

Skipping module imports is demonstrated in this example. Suppose “test_plots.py” has every function requiring Matplotlib. The whole file may be skipped instead of skipping each function like:

import pytest
try:
    import matplotlib
except ImportError:
    pytest.skip("Matplotlib not available", allow_module_level=True)


def ...

Conditional skipping can handle more complicated cases by making a decorator with pytest.skipif.

import pytest
import os

@pytest.mark.skipif(os.name=='nt', reason='only for Unix-like systems')
def test_unix():
    ...

If you have more than one function to skip with the same test skip condition, make your own decorator:

import pytest
import os

unixonly = pytest.mark.skipif(os.name=='nt', reason='only for Unix-like systems')


@unixonly
def test_unix_plot():
    ...


@unixonly
def test_unix_network():
    ...

Google via the OSSF introduced a Python API to compute Criticality Score of open-source projects. While there will continue to debate about how exactly to clip and weight parameters, being able to reduce metrics to a single number gives the community a quick first pass to judge which projects need help. Like h-index for academic bibliographies, Criticality tells only part of the picture. What about the score of fundamental libraries that have a large number of unhandled pull requests and infrequent updates.

The GitHub Primer design system underlying GitHub.com UI frontend was recently updated to enable dark mode theme, by @broccolini team.

Previously web browser accessibility settings could achieve a similar effect, but the dark theme is more consistent and effective.

Intel vPro AMT is not for directly internet-connected remote PCs. The operating system firewall does not protect vPro ports, which exist outside the operating system. Be sure the PC is behind an external firewall. Exercise great care if making these changes on an already remote PC, as a wrong checkbox hit can necessitate a physical field trip to the PC to correct. Since RealVNC Viewer Plus is discontinued, having an open-source connection via MeshCommander helps keep Intel AMT robust long-term.

Intel MeshCommander replaces Intel Open Manageability Toolkit.

1. On local laptopm Download and install the free, open-source Intel Mesh Commander.
2. Open the “Manageability Commander Tool” and click File → Add → Add Intel AMT computer and type the IP address and vPro username (typically “admin”) and password. Set the password in the remote PC BIOS if you haven’t already.
3. Click the little plus sign by Network and click on your AMT PC’s name–then in the Connection tab, click Connect. This will take about 5-10 seconds to connect–if Unsuccessful, the button will fall back to saying Connect. If successful, the button will change to say Disconnect.
4. Go to the Remote Control tab and wait about 10 seconds for the “remote desktop” items to change from “unknown” to the actual state. If Remote Desktop Settings is “Disabled”, click the little box to its right to open a new window.
5. Click OK and then go back to the Connection tab and click Disconnect. Don’t mess around with any of the other settings unless you know exactly what you’re doing and are willing to drive out to the remote PC to fix it if you mess something up!  Close the MeshCommander program.

Now you should be able to connect using a standard VNC program. Use “localhost” since we SSH into the remote PC first.

DO NOT expose VNC port 5900 to the internet or you are highly likely to get quickly hacked.

The factory Windows OpenSSH server is robust for secure SSH port forwarding. Consider SSH ED25519 public key authentication as it is vastly more secure than keyboard passwords.

Related: Intel vPro remote control ports

MSYS2 is useful for end user and developer laptops as well as CI. Using MSYS2 with Python on GitHub Actions CI requires adding MSYS2 to PATH just like the laptop. One factor is that MSYS2 is installed to a temporary directory.

Here is an example YaML workflow using MSYS2 with Python. Note we don’t use MSYS2 Python packages because they tend to be older versions than might be needed. The packages installed below aren’t specifically needed, it’s just an example of real-life use where I often use Ninja, Gfortran, HDF5, and CMake. This is a “build on run” package where the “pytest” command ultimately invokes CMake. We use GitHub Actions timeout parameters to avoid accidentally trying to build pip packages from source.

jobs:

  windows:
    runs-on: windows-latest
    steps:
    - uses: msys2/setup-msys2
      id: msys2
      with:
        update: true
        install: >-
          mingw-w64-ucrt-x86_64-ninja
          mingw-w64-ucrt-x86_64-gcc-fortran
          mingw-w64-ucrt-x86_64-hdf5

    - uses: actions/setup-python
      with:
        python-version: '3.x'

    - uses: actions/checkout

    - run: echo "${{ steps.msys2.outputs.msys2-location }}/ucrt64/bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append

    - run: python -m pip install .[tests]
      timeout-minutes: 2

    - run: pytest
      env:
        CMAKE_GENERATOR: Ninja
      timeout-minutes: 7

Some older Python projects may have typing as a package prerequisite. This can break Python itself from running. The “typing” PyPI module is not needed as all currently maintained Python versions have “typing” built in. Errors can result from having PyPI “typing” installed over factory Python “typing”.