Scientific Computing

Check HDF5 files for corruption

HDF5 files do not have an error recovery mechanism and do not journal. There is an optional per-variable error checksum Fletcher32 to detect data corruption. Checking/comparing file size alone is not an adequate check for HDF5 corruption.

Here a few easy techniques to check for corrupted HDF5 files.

Python HDF5 checking script checks HDF5 files for corruption and optionally finds the corrupted block(s) and variable(s)

HDF5 shell tools are installed by:

h5stat file.h5

Print the data values in the file

h5dump file.h5

HDFview GUI appears to use the Fletcher32 checksum to show a red question mark if corruption is detected. Another curiosity is that the Object reference is 2^32 - 1 on the corrupted variable.

HDFView bad variable


Related: HDF5 GUIs to view and edit variables in .h5 files

Spyder EOL while scanning string literal

In Spyder IDE when using Command Line Options under:

Run → Configure → General Settings → Command Line Options

be sure no line ending is in the parameter string. Otherwise, opening the Run Command may have error:

EOL while scanning string literal

Try deleting trailing spaces–also check any of the interword spaces if it’s still happening.

Share virtual machine between PCs

Virtual machine image sharing between computers works best with operating systems that are not license-bound to hardware. Linux and ReactOS are good candidates. Windows is not a good candidate for virtual machine sharing since Windows checks the hardware. The image can reside on a fast USB drive or network share.

Global license-free bands in 25-60 MHz range

Global ITU license-free frequencies

Individual countries may choose the details for their lands, or may open up additional frequencies. Generally this is a place to start.

  • 26.957-27.283 MHz ISM band (ITU RR 5.150)
  • 40.66-40.7 MHz ISM band (ITU RR 5.150)

Country-specific license-free

I don’t list every allocation because they are so numerous–just a few countries I ran across. This is a conceptual table–confirm yourself.

Country Regulation Frequencies [MHz] EIRP [dBm]
Australia 40.66–41 30
New Zealand 26.96-27.28; 29.7-30.0; 40.665-40.695 20
China/HK 26.96-27.28; 29.7-30.0; 40.665-40.695 20
Vietnam 26.96-27.28; 29.7-30.0; 40.665-40.695 20
Japan 0-322 -41.3
Korea 0-322 -41.3
Singapore 43.5 – 50.0
Malaysia 46.6 - 47
India (NFAP-2011) IND 04
UK Ofcom 27-27.3; 31-31.2; 40.66-41; 49.8-50
EU ECC Rec. 70-03 40.66-40.7 10
EU Ground pen / Wall pen radar 30-230 -44.5dBm/120kHz
Russia 26.957-27.283; 40.66-40.7; 40.025-48.5 10
Russia 40.66-40.7 30
Brazil 40.66-40.7 -35.2
Brazil 54-70 -55.2
UAE 29.7 - 47.0 10
USA P15.209 30-88 -55.2
USA 40.66-40.7 -35.2

Global ISM bands (from ITU), detailed


Related:

Get terminal window size from Fortran

The Bash shell has environment variables LINES and COLUMNS representing the current terminal window width. One might therefore incorrectly assume that Fortran 2003 standard get_environment_variable() subroutine would trivially get the current Terminal window dimensions. This is not so, since Bash passes along only the “POSIX” set of environment variables, and those that have been exported to the executable.

A working example of getting current Fortran terminal size using Ncurses is in the BlockTran program, where the variables LINES and COLS are set using the getmaxyx Ncurses macro.


Use the method above, as the method below naïvely fails.

Thus, the following will result in status code 1, indicating the environment variable was not found.

If the user runs this program as

LINES=24 ./myprog

that “works”, but we want an automatically determined value.

program noenv

use, intrinsic:: iso_fortran_env, only: error_unit

implicit none (type, external)

character(4) :: buf
integer :: h,ios

call get_environment_variable('LINES',buf,status=ios)

if (ios/=0) then
  write(error_unit,*) 'got error code',ios,'on trying to get LINES'
  stop
endif

read(buf,*) h

end program

Obsolete Fortran statement functions

Fortran 77 statement functions were thankfully made obsolete by Fortran 95 standard. Statement functions are not anonymous functions. In most cases, one should simple use standard functions instead of confusing statement functions. Other use cases are addressed by the polymorphism enabled from Fortran 2003 forward and/or pointers.

Example: Replace Fortran statement function with a standard function.

program st

integer :: f,i,j,k,n
! obsolete statement function (don't use)
f(n) = n+(i*j)**k

i=2
j=3
k=4

print *,f

print *,g(i,j,k)

contains

integer function g(n,i,j,k) ! use this instead of statement function
  integer, intent(in) :: n,i,j,k
  g = n+(i*j)**k
end function g

end program

Use VirtualBox over VNC

While for regular use one might setup a headless VM with RDP and/or SSH running on the VM itself, this procedure is for connecting to a virtual machine on a remote PC.

Prerequisites:

  • Remote: VirtualBox install of guest operating system
  • local: apt install xfreerdp-x11

on your local laptop, create a script vmrdp.sh containing:

#!/usr/bin/env bash

set -o errexit

ssh -f -L 5930:localhost:5930 sshusername@1.2.3.4 sleep 1;

xfreerdp /u:VMusername /v:localhost:5930

Optionally, add a few simple FreeRDP options to enhance performance, especially on limited data bandwidth connections

Most people can use the VirtualBox GUI, and don’t need the alternative command line option. Establish a VNC connection over SSH The very basic OpenBox desktop environment can open a Terminal on the remote PC by right clicking anywhere on the remote desktop and type

virtualbox

The usual VirtualBox program should open

From the right dropdown menu of the right arrow “start” button on the main VirtualBox window on the remote VNC PC, click headless. The status of the VM will show “Running”. Connect from laptop with a script vmrdp.sh as above. When done, shutdown your VM as usual if desired.

Most can use the GUI method above, but VirtualBox can also be managed well via shell commands.

virtualbox vmstart MyVMName --headless

starts up the VM named “MyVMName” for example. Numerous headless options exist for low server resource utilization via VBoxManage CLI.