Scientific Computing

xarray NetCDF LRU cache replaced autoclose

Python code using xarray.open_dataset() or xarray.open_dataarray() or similar functions reading from NetCDF4 use an LRU cache that automatically closes unneeded files.

As of xarray 0.11.0, the obsolete autoclose=True option should no longer be used.

Problems fixed by LRU cache: the LRU cache used to open NetCDF4 files with xarray fixes these problems, and gives high performance:

  • random segmentation fault while reading NetCDF4 .nc files, where the same file is reopened in the program.
  • OSError from too many open files, where even increasing ulimit doesn’t help

Animate imagesc movies in Matlab

The pause() Matlab / Octave statement causes figures to refresh.

%% create data, 25 frames of 512x512 pixels
data = rand(512,512,25);

%% create blank image
h = imagesc(data(:,:,1));

%% for loop to play "movie"
for i = 1:size(data,3)
  set(h, cdata=data(:,:,i)) % update latest frame
  pause(0.01) % feel free to reduce, but keep greater than 0 to ensure redraw
end

Related: animate plots in Matplotlib

Fix VirtualBox kernel module not found

On upgrading major versions, say from VirtualBox 5.2 to VirtualBox 6.0, you may find you can’t start any of your VM images, getting an error including “kernel modules do not match”. This may mean you haven’t removed all of the old version VirtualBox components. Consider using synaptic or

apt list --installed | grep virtualbox

to see if any old VirtualBox components are installed that may be conflicting with the new VirtualBox version.

Fix

You don’t have to uninstall the new VirtualBox version.

  1. as noted above, remove any components from the old VirtualBox install
  2. reboot the computer to fully flush the old VirtualBox components
  3. install the Extension Pack for the new VirtualBox version.

Matplotlib in Windows Subsystem for Linux

To use Matplotlib on WSL, first setup Anaconda Python on WSL.

conda install matplotlib

As in general, to speed up plotting when display of plots is not needed, use .savefig() by beginning the Python program code:

import matplotlib
matplotlib.use('agg')

print(matplotlib.get_backend())

you should see agg printed. Figure can then be saved to disk without displaying them on screen.

Brother printers on Linux

Network and USB printing with Brother printers generally work great on Linux and Windows Subsystem for Linux. The Brother installer script automatically downloads and installs necessary programs. Opt out of the last program brscan-skey, as it’s unnecessary.

Download Brother printer driver.

apt install cups

gunzip linux-brprinter-installer-*.gz

bash linux-brprinter-installer*

“Will you specify the Device URI? [Y/n]”  y   (if networked) “Specify IP address”: “Enter IP address” the static IP printer address

  • BRscan: YES allows xsane for Brother scanning printers.
  • BRsaneconfig: YES xsane scanner over the network
  • BRscan-Skey: NO don’t install because of potential security risks.

If accidentally installed brscan-skey, uninstall with

dpkg -r brscan-skey

As usual, manage/check printer from CUPS by pointing your PC web browser to localhost:631

Scanning documents in Linux in typically done with Xsane, which works on networked or USB connected scanners.

apt install xsane

xsane

To update scanner IP address use brsaneconfig4

For native Windows (not using Windows Subsystem for Linux), download just the driver that is about 15-20 MB and add the printer by IP address and choose “have disk”, pointing to the folder where you extract the drivers files to.

Force GitHub Linguist language detection

GitHub Linguist is reasonably accurate at automatically detecting the percentages of each code language in Git tracked code. However, as with any automatic coding language detection scheme, mis-detected languages occur. This seems to happen most often with Matlab code. Despite being in the top 10 STEM / data science languages, Matlab / GNU Octave *.m files can be detected as other languages such as Objective-C or Limbo.

Fix

Add to each Matlab repo’s .gitattributes file:

*.m linguist-language=Matlab

Verify

Simply git push your code and look at the GitHub repo language graph.

If you frequently need to auto-detect code and prefer Python, consider the Linguist Python wrapper of GitHub Linguist.

Matlab HiDPI support

Matlab can adjust for HiDPI (2K, 4K, etc.) displays. HiDPI Matlab IDE resolution (icon and font size) can be manually corrected from within Matlab on any operating system.

Matlab has programmatic DPI access to get / set information on DPI as well as numerous other Matlab characteristics via settings.

Matlab DPI setting:

S = settings;
S.matlab.desktop.DisplayScaleFactor

returns existing display scale like:

ActiveValue: 1
TemporaryValue: <no value>
PersonalValue: <no value>
FactoryValue: 1

set a new persistent DisplayScaleFactor by setting:

S = settings;
S.matlab.desktop.DisplayScaleFactor.PersonalValue = 1.25  % bigger number, larger icons & text

Verify the settings has taken effect after Matlab restart by:

S = settings;
S.matlab.desktop.DisplayScaleFactor

resulting in:

     ActiveValue: 1.2500
  TemporaryValue: <no value>
   PersonalValue: 1.2500
    FactoryValue: 1

Reference

GDL GNUDataLanguage reading IDL .sav files

While GDL has CMSvlib installed to read / write IDL .sav files, here is how to install CMSvlib if desired.

Download CMSVlib and extract to “~/.local/share/cmsvlib”. Add line to ~/.profile:

export GDL_STARTUP=$HOME/.gdl/gdl-startup.pro

Copy and paste into  “~/.gdl/gdl-startup.pro”

!PATH=!PATH+':'+Expand_Path('+~/.local/share/cmsvlib/')

Reopen Terminal and GDL. Load IDL .sav file like:

restore,'myfile.sav'

Check GDL paths loaded:

print,!PATH

Autogenerated Fortran docs using FORD

This is a basic example of using FORD to document Fortran projects. FORD is a Python program:

pip install ford

Each project will have a ford.md that configures and makes description for the main project landing page.

If your code is all under the top-level directory, set ford.md like: (this example is for the fortran2018-examples repo)

src_dir: ./
output_dir: ./docs
project: Fortran 2018 examples
project_github: https://github.com/scivision/fortran2018-examples
project_website: https://scivision.github.io/fortran2018-examples
summary: Fortran 2018 examples
author: Jane Doe
author_description: SciVision, Inc.
github: https://github.com/scivision
license: by
exclude: CMakeFortranCompilerId.F
display: public
         protected
         private
source: false
graph: true
search: true

Put Markdown test of arbitrary length here.
Within the Fortran code, use `!!` on each line with Markdown, Latex, etc.

When satisfied, move the documentation files to the gh-pages branch. This avoids cluttering Git with several megabytes of frequently changing documentation history.

Notes

Flip/Rotate LaTeX images

LaTeX rotation and flipping of text and images may be accomplished via graphicx.

Optional trim=left lower right upper is flipped vertically too. Trimming “lower” actually trims “upper”.

Rotation

\rotatebox[origin=c]{90}{\includegraphics[]{myimg}}
90
degrees of rotation counter-clockwise
origin=c
rotate about center of object

Vertical flip

\scalebox{1}[-1]{\includegraphics[]{myimg}}

Horizontal flip

\scalebox{-1}[1]{\includegraphics[]{myimg}}

Minimal working example

\documentclass[]{article}

\usepackage{graphicx}

\begin{document}

90 degree rotation \rotatebox[origin=c]{90}{\includegraphics[]{myimg}}

Vertical Flip \scalebox{1}[-1]{\includegraphics[]{myimg}}

Horizontal Flip \scalebox{-1}[1]{\includegraphics[]{myimg}}

\end{document}

graphicx package docs