Scientific Computing

Count lines of project code with CLOC

CLOC is a single-file Perl program that counts lines of code very quickly in large projects. If even more speed is needed, CLOC can run with parallel processes. Recent versions of CLOC are 10-100x faster than old versions. The Linux distro might have an old / slow CLOC version. It can be worth downloading the current version of CLOC if you use it a lot.

CLOC works on any operating system since it’s just Perl script.

For Linux / macOS: download latest cloc-*.pl to ~/.local/bin. Be sure $HOME/.local/bin is on your PATH by adding to ~/.profile:

export PATH=$HOME/.local/bin:$PATH

For convenience, make a softlink for cloc:

ln -s ~/.local/bin/cloc-*.pl ~/.local/bin/cloc

Make sure it’s executable:

chmod +x ~/.local/bin/cloc

For Windows download latest cloc-*.exe to c:\cloc\ and add c:\cloc to the user PATH.


To make CLOC run N times faster, you need the Perl Parallel::ForkManager library, which on Linux is installed by:

apt install libparallel-forkmanager-perl

then use like:

cloc --processes=4 .

For the GEMINI projects, we get a result like:

cloc --exclude-dir=vendor,archive,objects,subprojects  .
--exclude-dir
comma-separated list of directory names to exclude
     175 text files.
     168 unique files.
      22 files ignored.
github.com/AlDanial/cloc v 1.80  T=0.38 s (411.2 files/s, 69628.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Fortran 90                      35           3355           2809           9008
MATLAB                          71           1258           1202           4560
TeX                              1            266             76           1140
CMake                           20            315            119            857
Markdown                         3            175              0            366
make                             2             78             17            169
Bourne Shell                    13            113             97            160
INI                              6              1              0            126
Python                           4             65             15            122
NAnt script                      1             18              0             62
YAML                             1              7              9             19
-------------------------------------------------------------------------------
SUM:                           157           5651           4344          16589
-------------------------------------------------------------------------------

For debugging or finding what specific files are contributing to the total, use option:

cloc --categorized=cloc.log

Disable text wrapping in nano

Word wrapping in nano text editor can make scripts and files fail to run/parse properly. A simple edit can goof up a config or script by adding unwanted newlines. These settings are in general good for nano on any operating system.

Disable text wrap in nano by adding to ~/.nanorc

set nowrap

A suggested ~/.nanorc contains the following for good defaults including Python coding:

set constantshow
set tabsize 4
set nowrap
set tabstospaces
  • constantshow: show line and column of cursor position (on old nano version as in Red Hat 7, this option may not be recognized)
  • tabsize 4, tabstospaces: make tabs into 4 spaces
  • nowrap: disable long-line text wrapping

Nano syntax highlighting is enabled with include statements in ~/.nanorc.

WINE serial port links

WINE creates serial port softlinks( automatically.

Serial port mapping is managed via

WINEPREFIX=~/.wine wine regedit

Be sure to include the WINEPREFIX (default is ~/.wine)

Which WINE serial port to use: find the WINE serial port corresponding to the Linux serial port device name by watch log output from:

dmesg -w

when plugging in the serial ↔ USB adapter.

Using WINEPREFIX, start the WINE registry editor:

WINEPREFIX=~/.wine wine regedit

Configure the port–if the device is seen at /dev/ttyUSB0, and you want it to appear to WINE on COM1, edit HKEY_LOCAL_MACHINE\Software\Wine\Ports to have a new string entry named COM1 with value /dev/ttyUSB0.

Verify this setting (but do not edit) by:

ls ~/.wine/dosdevices

where again ~/.wine is the location of your WINEPREFIX.

64-bit Irfanview in WINE on Ubuntu

Irfanview is a popular Windows image viewing/editing program that works well on Linux via WINE. IrfanView requires MFC42, install by:

winetricks mfc42

Download and install IrfanView:

wine iview*x64_setup.exe

For more complex setups using winecfg, it’s best to use a separate WINEPREFIX.

FFmpeg and VLC on ReactOS

Currently ReactOS only covers through Windows XP/2003. FFmpeg has required Windows 7 or newer since 2015. Thus without some hacks or compiling FFmpeg on ReactOS yourself, FFmpeg isn’t currently available on ReactOS.

A current version of VLC is in the ReactOS Application Explorer, try doing what you need with VLC instead of FFmpeg with ReactOS until ReactOS can support Windows 7 applications better.

Map coordinate convert Python, Matlab, Fortran

Over the past few years we have created and refined open-source map coordinate conversion programs that are independently available for:

These programs use syntax similar to the $1000 Matlab Mapping and Aerospace Toolboxes, while being independently developed as open-source software. We use continuous integration on each package to help ensure quality results.

The functions available include

aer2ecef  aer2enu  aer2geodetic  aer2ned
ecef2aer  ecef2enu  ecef2enuv  ecef2geodetic  ecef2ned  ecef2nedv
ecef2eci  eci2ecef  eci2aer  aer2eci
enu2aer  enu2ecef   enu2geodetic
geodetic2aer  geodetic2ecef  geodetic2enu  geodetic2ned
ned2aer  ned2ecef   ned2geodetic
azel2radec radec2azel
vreckon vdist
lookAtSpheroid
track2

Extracting raw images from PDF

Instead of low-quality screen-shots a PDF to get the images, use Poppler to extract the original high-resolution images from the PDF. Note: only raster images can be exported with Poppler.

Examples of PDF image extraction tasks:

List all PDF images:

pdfimages -list in.pdf

Extract PDF images from all pages, dumping all images in mydoc.pdf to the same directory. Filenames start with out-. There might be a lot of images.

pdfimages -all in.pdf out

Extract PDF images from specific pages: example is for page 3 only:

pdfimages -all -f 3 -l 3 in.pdf out
-f
first page to extract
-l
last page to extract

Install Poppler:

  • Linux: apt install poppler-utils
  • macOS / Linux (Homebrew): brew install poppler
  • Windows: use WSL poppler

Related: extracting a page(s) from PDF

Joining PDF files into one

Join multiple PDF files into one:

pdfunite one.pdf two.pdf three.pdf joined.pdf

pdfunite is obtained by installing Poppler:

  • Linux: apt install poppler-utils
  • macOS: brew install poppler
  • Windows: WSL poppler

recursive convert DOC, DOCX to PDF

We have created Python scripts in LibreOffice Utils that recursively search for files matching a glob pattern (such as *.docx) and convert or print these input documents. They use LibreOffice in headless (console) mode, just a single Terminal command.

  • doc2pdf.py: recursively converts directories containing DOC, DOCX, RTF or other word processing files to PDF.
  • doc2print.py: recursively print documents to the default printer