Scientific Computing

Intel vPro AMT is not for directly internet-connected remote PCs. The operating system firewall does not protect vPro ports, which exist outside the operating system. Be sure the PC is behind an external firewall. Exercise great care if making these changes on an already remote PC, as a wrong checkbox hit can necessitate a physical field trip to the PC to correct. Since RealVNC Viewer Plus is discontinued, having an open-source connection via MeshCommander helps keep Intel AMT robust long-term.

Intel MeshCommander replaces Intel Open Manageability Toolkit.

1. On local laptopm Download and install the free, open-source Intel Mesh Commander.
2. Open the “Manageability Commander Tool” and click File → Add → Add Intel AMT computer and type the IP address and vPro username (typically “admin”) and password. Set the password in the remote PC BIOS if you haven’t already.
3. Click the little plus sign by Network and click on your AMT PC’s name–then in the Connection tab, click Connect. This will take about 5-10 seconds to connect–if Unsuccessful, the button will fall back to saying Connect. If successful, the button will change to say Disconnect.
4. Go to the Remote Control tab and wait about 10 seconds for the “remote desktop” items to change from “unknown” to the actual state. If Remote Desktop Settings is “Disabled”, click the little box to its right to open a new window.
5. Click OK and then go back to the Connection tab and click Disconnect. Don’t mess around with any of the other settings unless you know exactly what you’re doing and are willing to drive out to the remote PC to fix it if you mess something up!  Close the MeshCommander program.

Now you should be able to connect using a standard VNC program. Use “localhost” since we SSH into the remote PC first.

DO NOT expose VNC port 5900 to the internet or you are highly likely to get quickly hacked.

The factory Windows OpenSSH server is robust for secure SSH port forwarding. Consider SSH ED25519 public key authentication as it is vastly more secure than keyboard passwords.

Related: Intel vPro remote control ports

MSYS2 is useful for end user and developer laptops as well as CI. Using MSYS2 with Python on GitHub Actions CI requires adding MSYS2 to PATH just like the laptop. One factor is that MSYS2 is installed to a temporary directory.

Here is an example YaML workflow using MSYS2 with Python. Note we don’t use MSYS2 Python packages because they tend to be older versions than might be needed. The packages installed below aren’t specifically needed, it’s just an example of real-life use where I often use Ninja, Gfortran, HDF5, and CMake. This is a “build on run” package where the “pytest” command ultimately invokes CMake. We use GitHub Actions timeout parameters to avoid accidentally trying to build pip packages from source.

jobs:

  windows:
    runs-on: windows-latest
    steps:
    - uses: msys2/setup-msys2
      id: msys2
      with:
        update: true
        install: >-
          mingw-w64-ucrt-x86_64-ninja
          mingw-w64-ucrt-x86_64-gcc-fortran
          mingw-w64-ucrt-x86_64-hdf5

    - uses: actions/setup-python
      with:
        python-version: '3.x'

    - uses: actions/checkout

    - run: echo "${{ steps.msys2.outputs.msys2-location }}/ucrt64/bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append

    - run: python -m pip install .[tests]
      timeout-minutes: 2

    - run: pytest
      env:
        CMAKE_GENERATOR: Ninja
      timeout-minutes: 7

Some older Python projects may have typing as a package prerequisite. This can break Python itself from running. The “typing” PyPI module is not needed as all currently maintained Python versions have “typing” built in. Errors can result from having PyPI “typing” installed over factory Python “typing”.

Older hardware, particularly those with spinning hard drives, benefit from disabling unused services just as with any operating system. On older Macs, Siri can be so slow as to be not that usable anyway.

This change is persistent even after rebooting.

1. click Apple icon in upper left system menu bar and select System Preferences → Siri
2. clear the “Enable Ask Siri” checkbox. The Siri icon also disappears from the top right system menu bar.

Developer computer purchases today should have at least a 512 GB hard drive (solid state drive) and preferably 1 TB or larger. Less than 256 GB hard drive may not be usable for today’s developers. Particularly for macOS and Windows, where Xcode or Visual Studio respectively take about 50 GB of storage, and other developer libraries and toolchains can easily take another 50-100 GB, this leaves even a 256 GB system drive nearly unusably full. To help future proof and avoid installing and uninstalling and moving around files, just get a 1 TB or larger PC if a computer is intended for software development. Computer OEMs make larger storage computers non-linearly increase in price perhaps as a bit of a developer / creator tax. The workaround for this is to get a computer that has replaceable hard drive, but there’s time and risk involved in swapping hard drives as well, as well as typically larger laptop size.

Visual Studio Code is a free open source code editor with numerous extensions supporting most code languages, including for proprietary languages like Matlab. Fortran is well-supported by Visual Studio Code using the Modern Fortran extension, wrapping Gfortran as a linting tool.

The Grammarly API enables extensions like Visual Studio Code Grammarly Plugin. This plugin is useful for documenting code, Markdown Readmes, and Markdown-based blogs.

Simulations with spatial grids in 3D can be visualized via scatter plots. The grid may have irregular spacing such that each of the x, y, z dimensions is itself a 3-dimensional array. This can be visualized in Matlab or Python by reshaping the 3D arrays to a vector in the plotting command.

Scatter plots are one way to visualize 3D data in Matlab.

scatter3(x(:), y(:), z(:))

Python Matplotlib has several 3D visualization methods. Matplotlib scatter() also requires 1D vectors, which can be obtained at O(0) cost by the Numpy ndarray ravel method.

from matplotlib.pyplot import figure, show

ax = figure().gca()
ax.scatter(x.ravel(), y.ravel(), z.ravel())
show()

We use the free Poppler tools instead of using Acrobat, despite having a license. Extracting one or more pages from a PDF file can be done with paid Adobe Acrobat. The free Adobe Reader or FoxIt Reader cannot extract pages. Adobe Acrobat is a large and cumbersome program that installs startup daemons that are not trivially disabled.

To extract pages 2 to 3 from in.pdf using Poppler:

pdfseparate -f 2 -l 3 in.pdf out.pdf

Note: you must leave a space after -f and -l as shown.

Poppler is installed by:

• Linux: apt install poppler-utils
• macOS: brew install poppler
• Windows: use WSL poppler

An alternative to Poppler is GhostScript can also extract pages from PDF, but it’s a more complicated command:

gs -sDEVICE=pdfwrite -dFirstPage=2 -dLastPage=3 -dNOPAUSE -dSAFER -dBATCH -sOutputFile=out.pdf in.pdf

Related: extract raw full-quality images from PDF

GitHub CodeQL semantically analyzes Python code for security issues. Also, CVE Lists are checked vs. your GitHub repo’s dependency graph. CodeQL can install the Python package for more fidelity.

This approach finally fixes the concerns we had with the previous implementation that simply did CVE scans versus dependency graphs. The prior method of extracting dependencies did not work for modern Python packages. The new CodeQL method is much more robust and useful.