CMake / Meson temporary directory from environment variable

We have found that some CIs and systems do not define the environment variables “TMPDIR” or “TEMP” for temporary directory. Thus we recommend using a known defined location such as the current binary directory.

CMake

set(tmpdir ${CMAKE_CURRENT_BINARY_DIR})

tmpdir variable can be passed as a parameter to user test programs in CMakeLists.txt like:

add_executable(mytest mytest.c)

add_test(NAME MyTest COMMAND $<TARGET_FILE:mytest> ${tmpdir})

where mytest.c accepts command line parameters.

Meson

tmpdir = meson.current_build_dir()

tmpdir variable can be passed as a parameter to user test programs in meson.build like:

mytest = executable('mytest', 'mytest.c')

test('MyTest', mytest,
  args: [tmpdir])

Convert Periscope video for YouTube

If YouTube won't accept a video upload, or the video never completes “Processing” on YouTube, try re-encoding the video with FFmpeg. Sometimes lossy conversion is necessary to achieve the YouTube recommended upload settings.

Periscope video downloads use MPEG TS container with .ts file extension. These .ts files can be played back in VLC or similar to confirm content. For Periscope, a lossless conversion to YouTube is possible with:

ffmpeg -i pscp.ts -bsf:a aac_adtstoasc -codec copy -max_muxing_queue_size 1000 pscp.mp4

The FFmpeg option -max_muxing_queue_size is an arbitrary parameter–the default queue size is sometimes not large enough, and error messages on conversion can result like:

Too many packets buffered for output stream 0:1.

Notes

  • Do not use .mkv extension for YouTube uploads, it will fail to Process.
  • certain formats like AV1 or FFV1 will fail to finish processing the upload
  • after completing Uploading, video should start Processing in less than 5 minutes. If not, it will probably never actually work.

Deleting uploads that won't convert

Uploads that don't finish conversion processing after upload are not viewable on YouTube. To delete these videos:

  1. go to YouTube Studio and click “Videos” in the left-hand menu
  2. click the little three dot option button next to the video and select Delete Forever

Data bandwidth for live streaming video

These live streaming methods are easily accessible via PyLivestream that lightly wraps the FFmpeg command line. It generates a command line that you can use directly, or copy / paste for future reference.

Periscope

Periscope streaming is generally of lower video quality than other services, but can be robust in data bandwidth constrained environments. Instead of just dropping / cutting off the livestream (like YouTube) if the data connection has temporary difficulty, Periscope will degrade the video quality and keep the stream going. Periscope will increase the video quality to normal once the data connection improves. The data bandwidth depends on the video content (lots of motion vs. talking head) and RF channel (WiFi, LTE) conditions. The data are given in Mbyte/minute as data plans are typically priced by the Megabyte.

1 Mbyte/minute = 133 kbps, so non-360 Periscope uses about 500 - 900 kbps data bandwidth.

Stream typemean (Mbyte/minute)median (Mbyte/minute)
Horiz #16.35.6
Horiz #25.24.3
Vert #14.34.1

960 x 540 resolution

The specified bandwidth of SD 960 x 540 pixel Periscope is 896 kbps (800 kbps video + 96 kbps audio) or 6.7 Mbyte/minute max. Our experience across numerous mobile broadcasts have been in the range of 3.5..6.5 MByte/minute. The mobile Periscope app defaults to this mode.

1280 x 720 resolution

HD 1280 x 720 pixel streaming is available via Periscope Producer API. The recommended video data bandwidth is 2500 kbps + 128 kbps audio for 19.7 MByte / minute max.

360 video

Periscope Producer 360 API and a 360 camera with 2160 x 1080 resolution, video data bandwidth of 7000 kbps + 128 kbps audio for 53.5 Mbyte / minute max.

YouTube Live

YouTube Live has several streaming modes from 240p to 4K @ 60 fps. I would recommend considering 360p to start, as many livestream viewers are on mobile, where more than 480p streaming is often redirected by the carrier to 480p (for those on cellular). YouTube Live 360p: 640x360 @ 30fps resolution, with data bandwidth of 400 - 1,000 kbps corresponding to 7.5 Mbyte/minute.

Notes

Import Python user modules in Matlab

Matlab can call user Python modules on Linux, Mac and Windows. It goes without saying, the Python program must work first with Python alone for it to work in Matlab. For concurrent Python modules using asyncio, you may need to create a shim function to allow Matlab to call the Python module. Anaconda / Miniconda Python works fine from Matlab.

Configure Matlab for Python

Matlab is designed to work with specific Python versions for each Matlab version. Matlab will not specifically tell you when you're using an incompatible Python version, but you may get unstable operation or errors. Matlab R2019b introduced the pyenv Python environment manager.

The Python executable is found from Terminal / Command Prompt:

python -c "import sys; print(sys.executable)"

For Windows, it may be like C:/Miniconda3/python.exe and for MacOS / Linux it may be like ~/Miniconda3/python.

This Matlab command is persistent–Matlab remembers this Python choice even after restarting Matlab.

pyenv('Version', 'C:/Miniconda3/python.exe')

Verify Matlab → Python config by typing pyenv from within Matlab.

Troubleshooting

If only Python standard library modules work, and even commonly used modules like Numpy will not work, errors may occur from Matlab like:

py.numpy.arange(1)
Unable to resolve the name py.numpy.arange

Try diagnosing the issue from Matlab like:

pyenv

help('numpy')

Verify the desired Python install is being used, and that there isn't an error as below. If such an error occurs, check the PATH as seen by Matlab by:

getenv('PATH')

If the Python directories are missing, this may be why user modules are not importable in Python. These paths can be added to Matlab. On Windows, the necessary path is like C:/Miniconda3/Library/bin, the directory with lots of *.{so,dylib,dll} files and the fix is like:

setenv('PATH', ['c:/Miniconda3/Library/bin', pathsep, getenv('PATH')])

On Linux / Mac, the needed path is like ~/miniconda3/lib.

This command can be issued after a Python command was attempted, it's not necessary to restart Matlab. Note that this needs to be in PATH, since PYTHONPATH does not help in this case. This change does not persist across Matlab sessions. If it works for you, put it in the ~/Documents/MATLAB/startup.m file to have it reload each time Matlab is started.

Here's the error that's fixed by the procedure above:

problem in numpy - ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
  1. Check that you expected to use Python3.7 from "C:/Miniconda3/python.exe",
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy version "1.17.4" you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.

Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.

Original error was: DLL load failed: The specified module could not be found.

Notes

  • Python module import is implicit in the Matlab → Python module function call. There is no need to import numpy etc. from Matlab
  • Python executable choice persists across Matlab sessions–Matlab “remembers” even after you restart Matlab or reboot the computer.
  • editing imported Python module code requires restarting Matlab to take effect.

Older versions of Matlab back to R2014b can also call Python from Matlab. However, the pre-Matlab R2018b Python interface was more difficult to use and slower for arrays / matrices. Matlab ≥ R2018b is strongly recommended for use with Python.

Generate vectors of datetime in Python and Matlab

Generating a range of datetime data is a common data analysis and simulation task. Here we show examples of generating datetime vectors for:

  • Python datetime and numpy.datetime64
  • Matlab ≥ R2014b datetime
  • Matlab ≤ R2014a and GNU Octave

Python

Python datetime is often used as timezone-naïve with UTC as the assumed timezone. This custom avoids ambiguities when working with Pandas and Numpy, which are foundational for Python data science.

Generate a range of Python datetime like:

from datetime import datetime, timedelta
import typing


def datetime_range(start: datetime, end: datetime,
                   step: timedelta) -> typing.List[datetime]:
    """like range() for datetime"""
    return [start + i * step for i in range((end - start) // step)]


dt = datetime_range(datetime(2019, 12, 1), datetime(2020, 4, 1), timedelta(days=1))

With Numpy datetime64 generate a range of times like:

dt = numpy.arange('2019-12-01', '2020-04-01', dtype='datetime64[D]')

Pandas has the date_range function to generate time vectors.

Matlab ≥ R2014b

Matlab R2014b introduced datetime that is strongly encouraged over the ambiguous datenum() supported by old Matlab and GNU Octave.

dt = datetime(2019, 12, 1):datetime(2020, 4, 1)

Matlab ≤ R2014a and GNU Octave

GNU Octave does yet have a datetime class. To be compatible with old Matlab versions and GNU Octave, generate a range of datenum times like:

function times = datetimerange(tstart, tend, ts)
% generate a vector range of time using datenum time
% ts: seconds

narginchk(3,3)
validateattributes(tstart, {'numeric'}, {'scalar', 'positive'})
validateattributes(tend, {'numeric'}, {'scalar', 'positive'})
validateattributes(ts, {'numeric'}, {'scalar', 'positive'})


t0 = datenum(t0);
t1 = datenum(t1);

times = t0:ts/86400:t1;

end

Matlab pcolor datetime plots

Matlab datetime was added in Matlab R2014b. Until Matlab R2019, pcolor() did not understand datetime. It is sometimes necessary to manipulate Matlab plots involving datetime for the desired result.

dt = datetime(2019, 12, 1):datetime(2020, 3, 1);

dat = rand([100, length(dt)]);
y = 1:size(dat, 1);

pcolor(dt, y, dat)

If the plot doesn't have the desired datetime axis formatting, try datetick.

datetick('x', 'yyyy-mm-dd', 'keepticks')

Notes

Fix Matlab graphics error after driver update

After updating an operating system graphics driver while Matlab was open, an error may occur upon plotting in Matlab like:

MATLAB has experienced a low-level graphics error, and may not have drawn correctly. Read about what you can do to prevent this issue at Resolving Low-Level Graphics Issues then restart MATLAB.

The solution to this issue typically is, as the message suggests, just restarting Matlab. A full reboot is not always necessary, but try that if the error persists.

Check last error code across OS

From Terminal / Command Prompt, the return code from the last command is printed by:

  • Mac / Linux: echo $?
  • Windows command prompt: echo %errorlevel%
  • PowerShell: echo $lasterrorcode

Skip test with return code 77

Skipping a test with Meson or CMake by returning error code 77 is a de facto practice. Sometimes it is only feasible to know a test should be skipped by attempting to run that test.

Meson build system accepts return code 77 by default as a signal to skip the test. CMake can be configured to do this like:

add_test(NAME MyTest COMMAND mytest)  # arbitrary
set_tests_properties(MyTest PROPERTIES SKIP_RETURN_CODE 77)

Check temperature of Raspberry Pi CPU

The vcgencmd utility allows reading a few dozen hardware measurements on the Raspberry Pi boards.

CPU Temperature

vcgencmd measure_temp

Typical temperatures in office environment, with case:

Pi Modelheatsinkusagetemp [C]
2Byeslight-moderate40..45
4Bnolight-moderate65..70
4BnoYouTube 720p6085

A red thermometer icon GPU-superimposed on the Raspberry Pi display output means the Raspberry Pi is overheating and is throttling the CPU and GPU to avoid self-destruction.

Raspberry Pi 0, 1, 2, 3

CPU temp. [C]iconthrottle
< 80nonenone
80 - 85
half full thermometer red
CPU
> 85
full thermometer red
CPU & GPU

Raspberry Pi 4

CPU temp. [C]iconthrottle
80 - 85noneCPU: 1000 MHz
85 - 90
half full thermometer red
CPU
> 90
full thermometer red
CPU & GPU

CPU clock

The current Raspberry Pi CPU speed (actual speed, not limits or requested speed) is obtained from

vcgencmd measure_clock arm

The output is in units of Hertz:

frequency(45)=1199998000

The Raspberry Pi CPU clock speed and power consumption is dynamic:

clock speed [MHz]Raspi 2Raspi 3Raspi 3+Raspi 4
idle600600600600
100% one or more cores900120014001500

Log measurements

crontab -e can periodically log temperature and CPU frequency, e.g. add a line like:

@hourly  vcgencmd measure_temp | /usr/bin/logger

logger writes the measured parameters into the system log.

Monitor Raspberry Pi DC input voltage