Fix Gfortran stack to static warning

GCC / Gfortran 10 brought new warnings for arrays too big for the current stack settings, that may cause unexpected behavior. The warnings are triggered like:

real :: big2(1000,1000)

Warning: Array ‘big2’ at (1) is larger than limit set by ‘-fmax-stack-var-size=’, moved from stack to static storage. This makes the procedure unsafe when called recursively, or concurrently from multiple threads. Consider using ‘-frecursive’, or increase the ‘-fmax-stack-var-size=’ limit, or change the code to use an ALLOCATABLE array. [-Wsurprising]

Fix

This is generally a true warning when one has assigned arrays as above too large for the stack. Simply making the procedure recursive may lead to segfaults.

Correct the example above like:

real, allocatable :: big2(:,:)

allocate(big2(1000,1000))

For multiple arrays of the same shape do like:

integer :: M=1000,N=2000,P=500

real, allocatable, dimension(:,:,:) :: w,x, y, z

allocate(w(M,N,P))
allocate(x,y,z, mold=x)

Caveats

As with the Intel Fortran heap-arrays command-line options, there could be a penalty in speed by having large arrays drive off the stack into heap memory.

Install latest GFortran 10 on Linux

Newer version of compilers generally have more useful and detailed warning messages. Recent GCC versions have been steadily improving Fortran 2018 support. As with any compiler, newer versions of Gfortran may require rebuilding other libraries linked with the Fortran compiler if the ABI presented by libgfortran changes. On Linux, one can switch Gfortran versions with update-alternatives. If experiencing errors getting any version of gfortran installed in Ubuntu, try:

add-apt-repository universe

Ubuntu PPA

The latest GCC / Gfortran for Ubuntu is available from the Ubuntu-test PPA. Add Ubuntu-test PPA by:

add-apt-repository ppa:ubuntu-toolchain-r/test

apt update

Install the most recent Gfortran (similarly for gcc-10, g++-10) by:

apt install gfortran-10

Switch between compiler versions with update-alternatives.

  • Windows: Install latest Gfortran
  • MacOS: get latest gfortran by brew install gcc

Setup Astrometry.net and usage tips

Astrometry.net is easy to use on Linux and Windows. Windows uses Windows Subsytem for Linux for Astrometry.net.

Install

After install, if you need the star index files, use: downloadIndex.py

Linux / Windows Subsystem for Linux

  1. Download/install the pre-compiled binary code

    apt install astrometry.net
    
  2. Install the star map data necessary to solve images.

    apt install astrometry-data-2mass-08-19 astrometry-data-tycho2-10-19
    

MacOS

Using Homebrew:

brew install astrometry-net

Usage

The major steps in achieving a useful WCS starfield polynomial fit are:

  1. source extraction (identifying star position in image frame)

  2. quad asterism hashing, including tolerance for star position wiggling due to noise.

  3. Match hash to star catalog, at least 3 choices are available:

  4. Bayesian decision process, find extremely high probability solution or reject.

We have written a separate article on Astrometry.new tips and techniques

Notes

[Optional] compile

This is normally not necessary, unless you want to customize/optimize.

Prereqs:

apt install libcairo2-dev libnetpbm10-dev netpbm libpng12-dev libjpeg-dev zlib1g-dev swig libcfitsio-dev

Install: If you have Anaconda/Miniconda Python as your default, Astrometry.net will use them (or whatever your default Python is). Back to at least Astrometry.net 0.70, Astrometry.net is Python 3 compatible.

wget http://astrometry.net/downloads/astrometry.net-latest.tar.gz

tar xf astrometry.net-*.gz

cd astrometry.net-*

make
make py
make extra

make install INSTALL_DIR=~/astrometry.net

Add to ~/.bashrc

export PATH="$PATH:$HOME/astrometry.net/bin"

do not use ~ or you’ll get error

cannot find executable astrometry-engine

Uncomment inparallel in ~/astrometry.net/etc/astrometry.cfg (or /etc/astrometry.cfg)

Copy the star index files with downloadIndex.py

python downloadIndex.py

If it can’t find the index file, be sure ~/astrometry.net/etc/astrometry.cfg contains:

add_path /home/username/astrometry/data

~ or $HOME will NOT work!

Reference paper

Program giving azimuth/elevation for each pixel of your sky image

Alternative: online astrometry.net image scaling

Update Logitech Unifying firmware

Eavesdropping / injection vulnerabilities allow unencrypted wireless mouse connection to be used as a keyboard by attackers to inject unwanted keystrokes, possibly taking over your PC. Force pairing allows unauthorized input to your PC. Logitech device firmware has distinct per-OS update procedures.

Windows

Unifying software is used to update firmware and pair receivers with mice and keyboards. In Logitech Unifying software click Advanced → Update Firmware

Linux

Since May 2017, the Linux “fwupd” utility has supported updating Logitech Unifying receivers. Ubuntu, Fedora and other modern Linux distros will raise a prompt to update Logitech receiver firmware, which is a seamless quick process.

Check firmware version and pair devices to the Logitech Unifying receiver with

apt install solaar

Update thanks to Richard Hughes, a senior developer at Red Hat.

Check firmware version

List all recognized devices, including firmware versions where applicable:

fwupdmgr get-devices

Notes

Xvfb makes fake X11 for CI

Continuous integration for program that plot or need a display can be tricky, since in many cases the CI doesn’t have an X11 display server. Workarounds include:

  • Pytest conditional tests that detect CI via environment variable, totally avoiding generating plots. This can reduce code coverage.
  • generate plots using xvfb dummy X11 display server. This maintains code coverage, and may allow dumping plots to disk for further checks

Xvfb

This method uses X server virtual framebuffer (Xvfb) on continuous integration services.

GitHub Actions

Add to the “.github/workflows/ci.yml”, and assuming the project uses PyTest, the xvfb-action enables Xvfb for that command:

- name: Run headless test
  uses: GabrielBB/xvfb-action@v1.2
  with:
    run: pytest

Travis-CI

Travis-CI supports Xvfb by adding to project “.travis.yml”:

services: xvfb

Detect CI inside Python

Pytest handles conditional tests well.

import os
import pytest

CI = os.environ.get('CI') in ('True', 'true')


@pytest.mark.skipif(CI, reason="no plots for CI")
def test_myfun():
    from matplotlib.pyplot import figure,show

    ...

CI Environment variables

These CI’s and more set the environment variable CI as a de facto standard for easy CI detection.

One-step build/install CMake

CMake ≥ 3.15 is strongly recommended in general for more robust and easy syntax.

Compile/Install CMake

This will get you the latest release of CMake. For Linux and Mac, admin/sudo is NOT required.

pip

There is an unoffical PyPi CMake package:

pip install cmake

CMake major versions

  • 3.18: CMake Profiler cmake -B build --profiling-output=perf.json --profiling-format=google-trace
  • 3.17: Ninja Multi-Config generator, --debug-find to see what find_package() is trying to do,
  • 3.16: Precompiled headers, unity builds, many advanced project features
  • 3.15: CMAKE_GENERATOR environment variable works like -G option, enhanced Python interpreter finding
  • 3.14: check_fortran_source_runs(), better FetchContent
  • 3.13: ctest --progress, better Matlab compiler support, lots of new linking options, fixes to Fortran submodule bugs, cmake -B build incantation, target_sources() with absolute path
  • 3.12: transitive library specification (out of same directory), full Fortran Submodule support
  • 3.11: specify targets initially w/o sources
  • 3.10: added Fortran Flang (LLVM) compiler, extensive MPI features added
  • 3.9: further C# and Cuda support originally added in CMake 3.8.
  • 3.8: Initial Cuda support
  • 3.7: comparing ≤ ≥ , initial Fortran submodule support
  • 3.6: better OpenBLAS support
  • 3.5: Enhanced FindBoost target with auto Boost prereqs
  • 3.4: Limit CPU usage when using ctest -j parallel tests
  • 3.3: List operations such as IN_LIST

Git pull don't merge or rebase by default

Git 2.27 has default git pull behavior that we feel is beneficial. The Git 2.27 default is to not merge or rebase by default, unless the user specifies a default behavior. With Git ≥ 2.0, explicitly specify “safe” default behavior for git pull as follows. This setting that linear Git history is maintained unless manually specifying git pull options. Git services such as GitHub allow enforcing linear history.

git config --global pull.ff only

That adds to “~/.gitconfig”

[pull]
  ff = only

If encountering a Git remote that cannot be fast-forwarded, the user can then either git rebase or git merge.

Notes

Git: rebase vs. merge

CMake CTest cost data

CMake’s CTest assigns a dynamic COST to each test that updates each time the test is run. Kitware considers the cost test data to be undocumented behavior, so it’s not part of the CMake COST docs.

The computed test cost data is stored under ${CMAKE_BINARY_DIR}/Testing/Temporary/CTestCostData.txt This file stores data for each test in a row:

  TestName NumberOfTestRuns Cost