Scientific Computing

Writing multipage TIFF with Python

An easy, robust way to write multipage TIFF on any platform in Python is imageio.

For all examples below, assume a stack of images in an Numpy ndarray imgs, with dimensions:

imgs.shape
(Nimg, y, x) for monochrome. (Nimg, y, x, 3) for RGB color.

ImageIO is a library we have contributed code to and recommend in general for Python image IO.

pip install imageio
import imageio

imageio.mimwrite('myimgs.tiff',imgs)

tifffile

ImageIO uses tifffile internally, so most don’t need to use tifffile directly. To use tifffile directly, install tifffile.py:

pip install tifffile
import tifffile

tifffile.imsave('myimages.tiff',imgs)

tifffile.imsave() is capable of description and tags arguments and to compress losslessly.

Advanced Python TIFF multi-page writing example: archive/old_tiffile_demo.py.

Read TIFF headers

The de facto TIFF header tags. can be read from the command line with Perl Image::ExifTool

apt install libimage-exiftool-perl

exiftool myfile.tif

Note: tiffinfo doesn’t print tiff tags.

Print all TIFF tags from Python using archive/PrintTiffTags.py

Alternative multipage-Tiff method using scikit-image and FreeImage: (we recommend imageio or tifffile instead)

from skimage.io._plugins import freeimage_plugin as freeimg

freeimg.write_multipage(imgs,'myimages.tiff')

Due to the large number of image libraries invoked, sometimes scikit-image needs a little tweaking for image I/O:

Windows

if you get:

RuntimeError: Could not find a FreeImage library

Fix by downloading the FreeImage binaries and extract Dist/x64/FreeImage.dll to the directory found by:

$(python -c "import skimage; print(skimage.__path__[0])")/io/_plugins/

Linux

If you get:

freeimage had a problem: Could not find a FreeImage library in any of...

Fix by:

apt install libfreeimage3

reference

Old Numpy

To fix tifffile error caused by too-old Numpy version:

RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa

Install a newer Numpy version:

pip install numpy

pip install tifffile

Since there is a Numpy .whl binary wheel for ARM, the latest Numpy installs quickly on Raspberry Pi.

Install packages in GNU Octave

GNU Octave can install third-party packages in a friendly way, analogous to the Matlab App Store or how Linux repositories work. Regardless of operating system, Octave can install these extension packages from the Octave command line. Some packages require a compiler or libraries. If package install fails, read the log output to see if installing a system library is required.

Packages are installed at the Octave command prompt, and download automatically. Prereqs are not automatically installed, but messages are given telling which package needs to be installed first. signal is a perfect example of this, given below.

“signal” is a popular Octave package, which brings many functions found in Matlab’s DSP and Communications Toolbox. We’ll see that signal needs other packages first; let’s walk through the Octave signal install. All commands are from Octave command prompt.

Try using a command that requires signal

diric(0.2, 5)

warning: the ‘diric’ function belongs to the signal package from Octave Forge which seems to not be installed in your system.

If I had already installed signal, but forgotten to load it since I started Octave, the error would have been:

warning: the ‘diric’ function belongs to the signal package from Octave Forge which you have installed but not loaded.

Install signal Octave package from Octave prompt:

pkg install signal

This returns a warning saying that control is required.

Install control:

pkg install control

this requires the gfortran compiler.

pkg install signal

Use Octave packages in a Matlab-compatible way simply by enclosing in try end

function d = twicediric(x)
  try
    pkg load signal
  end

  d = 2*diric(x)
end

If the package isn’t installed, the message on reaching the missing function tells which package is needed.

Matplotlib datetime examples

Matplotlib can make many types of plots with a time axis. However, sometimes it takes an additional command or two to make the date/time axis work right in Matplotlib.

As seen in xarray_matplotlib.py, for imshow() datetime64 extent, you need to do something like:

import matplotlib.dates as mdates

# whatever your time vector is
t = np.arange('2010-05-04T12:05','2010-05-04T12:06', dtype='datetime64[s]').astype(datetime)

mt = mdates.date2num((t[0],t[-1]))

ax.imshow(im, extent=[mt[0],mt[1], y[0],y[-1]], aspect='auto')

In most Matplotlib plotting functions numpy.datetime64 is a first-class citizen, but not yet for imshow() perhaps due to the limits-oriented nature of imshow(). We use pcolormesh() instead of imshow() for datetime-oriented raster data.

Matlab OpenCV C++/CUDA/MEX support

To use OpenCV from Matlab via vision Support Packages as integrated by the Mathworks using MEX.

Select “Computer Vision System Toolbox OpenCV Interface by MathWorks Computer Vision System Toolbox Team” and install.

Note: the examples require particular compilers depending on Matlab version and operating system.

Examples directory contains Computer Vision Toolbox examples from the Mathworks. Find the Matlab OpenCV example directory, in Matlab:

fileparts(which('mexOpenCV'))

The examples below assume you’re starting from this directory. See the README.txt in each directory for compilation details. Some examples require a CUDA GPU.

Foreground Detector: build example

cd ForegroundDetector

mexOpenCV backgroundSubtractorOCV.cpp

If the example fails to compile due to compiler mismatch, follow the instructions given in the error message.

Run the OpenCV Matlab demo:

testBackgroundSubtractor

You will see a Video Player window pop up with cars driving by, with the cars detected outlined in white rectangles.

print to stderr from Matlab / GNU Octave

In both Matlab and GNU Octave, functions like error and warning by default print messages to stderr in non-interactive sessions, for easier capture to uncluttered log files.

To print to stderr in general, use

fprintf(2, 'Hello text')

Verify these are printing to stderr by appending to the end of the command line command: 2>err.log

From GNU Octave:

octave --eval "fprintf(2,'this is stderr')"

From Matlab:

matlab -batch "fprintf(2,'this is stderr')"

CMake builds for modern Fortran

Fortran languages standards keep long backward compatibility. Fortran 2018 finally deprecated Fortran 1958 fixed width format. In general across languages, compiler vendors take years to add the full language feature set. Automatically determining if a particular compiler supports a needed modern Fortran feature is straightforward with CMake (and Meson).

This CMake example is for error stop. Unlike C++, we do not typically need to enable modern Fortran features with specific compiler flags. Modern Fortran features are typically enabled by default.

include(CheckSourceCompiles)

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)
# save link time, only compile is needed

check_source_compiles(Fortran "program test
error stop
end" f08errorstop)

if(f08errorstop)
  ...
endif()

Fortran submodule is supported by CMake.

Install Ruby Gems without sudo

Ensure Gems are installed to home directory under ~/gems, by adding to ~/.profile:

export GEM_HOME=$HOME/gems
export PATH=$PATH:$HOME/gems/bin

This allows installing Ruby Gems without sudo.

Windows laptop to remote Windows SSH + RDP

Assumes: Windows laptop connecting to a remote Windows PC with OpenSSH server and client built into Windows.

Remote PC IP Remote PC SSH port Remote PC RDP port
1.2.3.4 22 (open TCP firewall) 3389 (blocked by remote PC firewall)

Setup: on the Windows laptop, create script file sshrdp.bat:

start /b ssh -L 3391:localhost:3389 %1

@echo off
REM enough time to enter password
timeout /nobreak 10

mstsc /v:localhost:3391

Usage: from that directory, type sshrdp myhostname to connect RDP over SSH to myhostname computer that’s setup in your ~/.ssh/config file or via the IP address or hostname directly.

sshrdp.bat:

  1. makes the SSH connection (you’ll be prompted for SSH password, or use a public key file)
  2. makes the RDP connection over the SSH tunnel (where you will be prompted for the Windows password).

Notes

You can specify some Remote Desktop parameters on the MSTSC command line. There are many more options only available via saving an .RDP profile file.

You can create an .RDP file by saving a Remote Desktop Connection and then load that specific profile by:

mstsc "c:/MyPC.RDP" /v:localhost:3391
  • use Port 3391 for local forwarding because Windows uses Port 3390 for something else.

Related: Linux to Windows PC over SSH / RDP

Where is Python site-package directory

Python libraries are installed or linked to the lib/python?.?/site-packages/ directory. The location of this directory is revealed by:

python -c "import site; print(site.getsitepackages())"

The output will be like:

~/miniconda3/lib/python3.7/site-packages

There are multiple ways to setup virtual environments, the command above will be particular to the virtual environment, for example:

~/miniconda3/envs/py36/lib/python3.6/site-packages'