Where “files” is set appropriately for your project.
Making a per-project files is strongly recommended to ensure files aren’t missed in the type check.
One can make a system-wide ~/.mypy.ini, that is overridden by the per-project .mypy.ini.
isolate problem packages
Sometimes an external package adds type hinting that is incompatible with the current MyPy release.
This is relatively rare, but was the case with Xarray.
To ignore a package’s type hinting, add the following to .mypy.ini, where we assume we want to ignore xarray type checking.
We use this function to capture the number of physical CPU cores available on a computer from Matlab.
Like any such function it doesn’t always work, but we try 3 separate methods to help improve accuracy.
Knowing the CPU count is useful when running mpiexec from Matlab.
function N = get_cpu_count()
%% get apparent number of physical CPU coresif isoctave
N = idivide(nproc, 2); % assume hyperthreadingelse
N = maxNumCompThreads;
if N <2% happens on some HPC
N = feature('NumCores');
endif N <2&& usejava('jvm')
% assume hyperthreading
N = java.lang.Runtime.getRuntime().availableProcessors /2;
N = max(N, 1);
endfunction isoct = isoctave()
isoct = exist('OCTAVE_VERSION', 'builtin') ==5;
CMake (via CTest) can run tests in parallel via the option
ctest --parallel 4
where “4” is the number of tests to run in parallel–set it approximately equal to the number of CPU cores in your system typically.
Some tests need to be run not in parallel, for example tests using MPI that use lots of CPU cores, or tests that use a lot of RAM, or tests that must access a common file or hardware device.
We have found that using the
makes whole groups of tests run sequentially instead of individually running sequentially when fixtures are used.
That is, all the FIXTURES_SETUP run, then all FIXTURES_REQUIRED that have RUN_SERIAL.
This is not necessarily desired, because we had consuming fixtures that didn’t have to wait for all the fixtures to be setup.
We found that using
did not suffer from this issue, and allows the proper test dependencies and the expected parallelism.
For simplicity we omit the necessary add_test() and just show the properties.
The test has an MPI-using quick setup “Quick1” and then a long test “Long1” also using MPI.
Finally, we have a quick Python script “Script1” checking the output.
In the real setup, we have Quick1, Quick2, … QuickN and so on.
When we used RUN_SERIAL, we had to wait for ALL Quick* before Long* would start.
With RESOURCE_LOCK the tests intermingle, making better use of CPU particularly on large CPU count systems, and with lots of tests.
The name “cpu_mpi” is arbitrary like the other names.
lacks features needed for blackbox interfacing with executables, including lack of stdin pipe.
We have demonstrated via
how to exchange data to/from executables and run them with timeout using either:
Traceability of a binary artifact such as an executable or library can be improved by writing information about the Git repository status into the artifact itself.
This is a finer-grained implementation of the version number we are accustomed to seeing in the command line interface of executables.
This example doesn’t cover every possible thing to be traced, for example non-version controlled artifacts that are linked in.
This example just covers the Git repo of the current CMake project.
Nonetheless, those needing more advanced traceability can build upon this example.
Usually for easier reuse across projects, we put this in a separate CMake script file like gitrev.cmake and include it from the main CMake project.
allows specifying details options to control HTTP behavior via
Typical options that are modified include Timeout and SSL Certificate checking bypass.
While SSL certificate checking adds security to web operations, some HPC systems have old or broken certificates.
Other systems may simply need environment variable SSL_CERT_FILE set to tell Matlab’s vendored cURL where the cert file is.
As a last resort, certificate checking can be turned off, but this opens up code / file integrity and concomitant security issues.
A generally better solution than disabling certificate checking is to configuration your user profile to tell cURL and Git the location of the system certificates.
For this example we assume the certificate file is at “/etc/ssl/certs/ca-bundle.crt”.
cURL SSL config
set environment variable by editing ~/.bashrc
This for example can fix issues with Matlab websave() that uses Matlab’s vendored cURL.