Scientific Computing

Removing line numbers from text file

Assuming an OCR’d book or other text file that has line numbers embedded in file neat.py:

1 # cool program
2 import sys
3 def howneat():
4     sys.exit('Thanks for visiting')

Remove the line numbers by typing in Terminal:

perl -pe 's/^[ t]+d+//' neat.py  > neater.py

Regular expression:

^[ t]+d+
\^
beginning of the line
[ t]+
match space or tab (one or more)
d+
match one or more digits
-p
implicitly loop over (read each line of) the file neat.py.
-e
enable PCRE

And we replace with nothing. This leaves alone the indentation of the code (relevant for Python).

I could have used the -i flag to edit in place, but I instead redirected STDOUT to neater.py in case I made a mistake (inputting the wrong file, for example).

Graph of .tex document progress to website

This Python program extracts the LaTeX word count and equation count and several other counts such as floating element count from a LaTeX document set, using the Perl script texcount.

git clone https://github.com/scivision/texcounts.git
pip install -e .

Plot progress:

python texwcpost.py mydissertation.tex

Upload to the SFTP server:

python phdwc.py mydissertation.tex serverAddress username serverDirectory

LaTeX harpoon vector symbol

A favorite vector symbol is a harpoon.

  1. in document header

    \usepackage{harpoon}
    \newcommand*{\vect}[1]{\overrightharp{\ensuremath{#1}}}
  2. in document body where desired.

    $\vect{x}$

Notes

  • The harpoon package defaults to text mode, which will fail when math characters like ell are used in the math mode.
  • reference

Put external USB drives into standby

Manually put a USB HDD that doesn’t automatically spin-down after inactivity to sleep with sg_start

apt install sg3-utils

Determine which device ID by:

lsblk

Assuming the HDD to spin down was at /dev/sdc:

sg_start 0 --pc=3 /dev/sdc

The HDD wakes up automatically when writing/reading a file on it.

We spin down hard drives because HDDs put out substantial heat when spinning, but don’t spin up/down too frequently to avoid excessive wear.

Using Phoronix Test Suite

Phoronix Test Suite is a well regarded benchmarking suite that supports most operating systems. This gives the ability to accurately benchmark operating system performance errors, and has been used to find Linux kernel performance regressions.

  1. Download and install

  2. run tests

    phoronix-test-suite benchmark c-ray
    phoronix-test-suite benchmark hpcc
    phoronix-test-suite benchmark scimark2
    phoronix-test-suite benchmark xonotic
  3. List results

    phoronix-test-suite list-saved-results
  4. Convert results to PDF

    phoronix-test-suite result-file-to-pdf  myTest

Matlab - fixing exponential tick labels

Sometimes for large number > 10000, Matlab/Octave axes ticks will appear as exponentials. Fix this to the desired number of digits by:

set(gca,'YTickLabel',sprintf('%7d|',get(gca,'YTick')))

This example is for 7 digits.

HiST 2014 aurora season start

This was the first night of operations. No GPS 1 PPS timing on either station.

The data will be put in Zenodo, but is available via Google Drive, email me to access.