Scientific Computing

Both the shell and Python methods get public IPv4 and IPv6 addresses. This is good for verifying the computer IP is in the organization’s IP address range. These scripts use the “reflector” method, which can be more reliable on complex networks.

Shell: specify the network interface with curl --interface eth0 option.

Bash shell script “getIP.sh”

url=('https://ident.me' 'https://api.ipify.org')

for u in ${url[@]}; do
  curl -6 -s -m 2 $u && break
done

for u in ${url[@]}; do
  curl -4 -s -m 2 $u && break
done

Python script getIP.py

#!/usr/bin/env python
"""
gets interface IPv4 and IPv6 public addresses using libCURL
This uses the "reflector" method, which seems more reliable for finding public-facing IP addresses,
WITH THE CAVEAT that man-in-the-middle, etc. attacks can defeat the reflector method.
"""
from ipaddress import ip_address
import pycurl
from io import BytesIO

urls = ['https://ident.me', # ipv6 and ipv4
        'https://api.ipify.org'] # ipv4 only
length=45
# https://stackoverflow.com/questions/166132/maximum-length-of-the-textual-representation-of-an-ipv6-address

def getip(interface=None):
    for url in urls:
        addr = []
        for ipv in (pycurl.IPRESOLVE_V4,pycurl.IPRESOLVE_V6):
            buffer = BytesIO() # must clear like this
            C = pycurl.Curl()
            if interface:
                C.setopt(pycurl.INTERFACE,interface)
            C.setopt(C.URL, url)
            C.setopt(pycurl.IPRESOLVE, ipv)
            C.setopt(C.WRITEDATA, buffer)
            try:
                C.perform()
                result = buffer.getvalue()
                try: #validate response
                    addr.append(ip_address(result.decode('utf8')))
                except ValueError:
                    pass
            except pycurl.error:
                pass
            finally:
                C.close()

        if len(addr)>1: #IPv4 and IPv6 found
            break

    return addr

if __name__ == '__main__':
    import signal
    signal.signal(signal.SIGINT, signal.SIG_DFL)

    from argparse import ArgumentParser
    p = ArgumentParser()
    p.add_argument('-i','--iface',help='network interface to use')
    p = p.parse_args()

    addr = getip(p.iface)
    print(addr)

A data acquisition system typically runs multiple programs simultaneously:

1. High-priority data collection - a program driving high data bandwidth acquisition
2. Low-priority monitoring tools - check system status or process data
3. Low-priority services - web servers displaying processed results

Properly prioritizing these processes ensures the critical data collection isn’t interrupted by secondary tasks.

Suppose a Python script “preview.py” is monitoring a camera’s status and needs to run at low priority.

On Unix-like systems (Linux, macOS, BSD, …) use nice to control process priority.

nice -n 19 python preview.py

nice -n

run task(s) with priority, where bigger positive numbers are lower priority, and more negative numbers are higher priority.

On Windows the start command can control program run priority.

start /low python preview.py

start /low

runs a program and its child processes at low priority.

A symbolic link may become broken due to updating/compiling/installing software, showing up as red text in ls Terminal.

Suppose the broken link is named “foo.so”: Find where link pointed to:

readlink -v foo.so

Point link to new file (perhaps foo2.so)

ln -sfn foo2.so foo.so

Verify new link is blue color

ls foo.so

Easily record the sounds heard through system speakers on Linux with Pulseaudio and Audacity.

apt install audacity pavucontrol

To record “what you hear” in Audacity: “click to monitor” (right of microphone in main Audacity screen). Start Pulseaudio advanced configuration tool

pavucontrol

Under pavucontrol Recording tab input, select “Monitor of Built-in XXX” where XXX is the playback device you wish to monitor. This loopback audio from pavucontrol allows recording what you hear in Linux using Audacity and PulseAudio.

On Windows, Adobe Reader can export and view a list of comments through the Comment tab on the right side. On Linux, in Evince PDF viewer, the side pane has a drop-down annotations menu that lists comments. Press F9 key if you don’t see the left-hand panel.

One can extract all comments and annotations from a PDF file by

apt install texlive-latex-extra

pdfannotextractor --install

pdfannotextractor in.pdf

This yields a .pax file without plain text. However, it was not clear how to read this file.

It’s often useful to open the same PDF file multiple times, particularly when reviewing journal article drafts to view figures simultaneously with text describing the figure.

From Evince (popular Linux PDF viewer), Menu select “Open a Copy”. Open as many copies of the same Postscript or PDF document as desired.

This is for Ubuntu < 18.04. Ubuntu ≥ 18.04 no longer use Unity.

If the desktop seems to freeze for more than a few seconds, try

1. press together the keys: Ctrl Alt F1
2. login and type

unity & disown

This leaves desktop/apps all as they were while fixing the broken state.

Full desktop reset

Restart the whole Unity desktop (which is like unto logging out and logging in, discarding the current desktop):

service lightdm restart

Jekyll uses Markdown or HTML files to statically render complex websites from simple text files. While Jekyll syntax highlighting is possible, more often we might use more generic Markdown commands.

Enable syntax highlighting: Before every code block, simply include the language name, for example

```fortran

this even works for gdb.

The color code syntax highlighting on your webpage looks quite striking, and the number of languages covered is very extensive. Despite the different syntax highlighter modules used by different Jekyll web hosting services, almost every language works with the method above.

Github uses Linguist syntax highlighting for its own service (Issues, README.md, etc.) Github/Gitlab Pages can use Rouge syntax highlighting via Jekyll _config.yml – specify an allowed syntax highlighter.

Python collections.deque is a fast FIFO that does not require thread locking as the more sophisticated queue.

Use deque to read the last line of files as follows.

from pathlib import Path
from collections import deque

fn = Path('myfile.txt')

with fn.open('r') as f:
    last = deque(f, 1)[0]

last

last line of file. Throws IndexError if file is empty.

By changing 1 the last N lines of the file may be read and indexed.

deque examples

At first glance, the two watt transmit power limit across five VHF channels can make MURS seem appealing for long-range license-free radio systems. In fact, companies such as GoTenna initially chose MURS for their first generation product.

The clear choice for almost all applications is to use 900 MHz modems over MURS 150 MHz modems. Contact us for a more formal analysis for your application.

MURS data bandwidth:

At first glance, MURS seems to have a comparable data rate to the $39 one watt 868/900/920 MHz modems, and similar hardware pricing, with seemingly longer range due to 150 MHz vs. 900 MHz frequency. Interference limits data throughput.

MURS interference

MURS arose in part because of the decades-long abuse of these five VHF itinerant frequencies. Like the delicensing of 27 MHz CB Radio, the license-free MURS channels arose due to chronic unlicensed use. Warehouses, hotels, shopping malls, construction sites, etc. use MURS frequencies.

The critical point is that in general MURS modems are stuck on one frequency until reprogrammed. $14 walkie talkies meanwhile are clogging up the channels. Cable TV leakage is also a problem at VHF.

Many applications of wireless modems have a latency requirement–how long will an MURS channel be blocked by noise and other users? The answer comes in using 900 MHz spread-spectrum radio.

900 MHz advantages

900 MHz is also a heavily used frequency band. The key distinction is this use is spread over as much as 26 MHz (USA). The one watt transmit power allowed in the USA and numerous other countries, and tens of milliwatts in other countries allows multi-kilometer line-of-sight range, just like 150 MHz radios.

Some 900 MHz modems have a channel list where they hop upon interference, while the high power radios typically employ FHSS. Having a clear RF channel via FHSS is key to getting even low data bandwidth transmissions through reliably with low latency.