Sometimes cryptic errors occur if there was a syntax error in a .bib BibTeX bibliography file that doesn’t disappear even when the .bib syntax is corrected.
The fix for this is often to delete the auto-generated files:
Suppose the top-level LaTeX file is “main.tex”.
Then the compilation generates main.bbl and main.aux among several others.
If you keep getting errors when you know the syntax is correct then try:
rm main.aux main.bbl
pdflatex main
bibtex main
pdflatex main
pdflatex main
A general rule for resurrecting old code in any language is first try to get it working for known input/output via emulation or virtual machine.
Many old codes were made before linters and other correctness checks we’ve long taken for granted.
The codes may use non-standard tricks no one has thought about for years to squeeze tiny fractions of efficiency that haven’t been significant for decades.
After correct operation is established, Fortran 66 code often needs a few tweaks to work with modern compilers.
The Fortran Wiki has an excellent article on modernizing Fortran syntax http://fortranwiki.org/fortran/show/Modernizing+Old+Fortran.
Here are a few specific techniques we use.
Regex operations
Obviously make a copy of the file first. Sometimes 10,000+ lines will match so it can be tedious to check.
Similar operations may be needed if a mix of tabs and spaces are used.
move line numbers left
Run this regex and replace with null (nothing):
^\s+(?=\d+\s)
make code start in column 7
Recall that fixed format (Fortran <= 77) code has line numbers in columns 1-5 of each line numbers
After the prior operation, push the actual code right of column 6 by replacing this regex with appropriate number of spaces:
(?<=^\d+\s+)
multiline codes
You may see $ used like a semicolon in CDC Cyber Fortran
Duplicated line numbers
Put procedures in separate files for duplicated line numbers.
If needing to re-encode video anyway, for example if coming from an a scientific camera capture with uncompressed video, the
suggested settings for YouTube upload
using FFmpeg are:
This function recursively zips files in directories, storing only the relative path due to the use of the “arcname” parameter.
Otherwise, you get the path relative to the root of your filesystem, which is rarely useful.
import zipfile
from pathlib import Path
defzip_dirs(path: Path, pattern: str) -> T.List[Path]:
"""
recursively .zip a directory
"""
path = Path(path).expanduser().resolve()
dlist = [d for d in path.glob(pattern) if d.is_dir()]
if len(dlist) ==0:
raise FileNotFoundError(f"no directories to zip under {path} with {pattern}")
for d in dlist:
zip_name = d.with_suffix(".zip")
with zipfile.ZipFile(zip_name, mode="w", compression=zipfile.ZIP_LZMA) as z:
for root, _, files in os.walk(d):
for file in files:
fn = Path(root, file)
afn = fn.relative_to(path)
z.write(fn, arcname=afn)
print("write", zip_name)
Matlab plotting can be quite slow, as can Python Matplotlib plotting.
Sometimes, Matlab
parfor
can be used to
plot in parallel,
when all parfor restrictions are met.
However, parallel plotting in Matlab doesn’t always work, or may work on some operating systems but not others.
So use great caution if making a “parfor” plotting loop–it may not work for others.
Errors noted
Here are some errors we’ve gotten when trying to do relatively simple plots in parallel:
Warning: A worker aborted during execution of the parfor loop. The parfor loop will now run again
on the remaining workers.
> In distcomp/remoteparfor/handleIntervalErrorResult (line 240)
In distcomp/remoteparfor/getCompleteIntervals (line 387)
In parallel_function>distributed_execution (line 741)
In parallel_function (line 573)
Error using distcomp.remoteparfor/rebuildParforController (line 194)
All workers aborted during execution of the parfor loop.
Error in distcomp.remoteparfor/handleIntervalErrorResult (line 253)
obj.rebuildParforController();
Error in distcomp.remoteparfor/getCompleteIntervals (line 387)
[r, err] = obj.handleIntervalErrorResult(r);
Warning: worker(s) crashed while executing code in the current parallel pool. MATLAB may attempt
to run the code again on the remaining workers of the pool, unless an spmd block has run. View the
crash dump files to determine what caused the workers to crash.
Matlab does not have a native means to pass Matlab data via a stdin pipe to programs called with
system.
This is not an issue with
Python subprocess
that is quite robust and featureful.
“stdout” and “stderr” are joined by Matlab, available at program completion like:
[ret, msg] = system(...);
where “msg” is the data from stdout and stderr joined together.
The benefits of this method is that is can be faster than using temporary files, and helps avoid filesystem clashes when running many external processes in parallel or asynchronously.
This method avoids the need to write additional code directly interfacing memory between Fortran or C and Matlab by using file-based or pipe-based API for data streaming.
The penalty for this simplicity is execution speed, but that can be resolved later if I/O speed is a genuine bottleneck by using Matlab Engine or MEX.
Python
tempfile
is normally robust but has some corner cases on Windows.
A common issue is that on exiting a tempfile context manager, PermissionError is thrown on Windows.
This can happen when non-writable files are placed into a tempfile.TemporaryDirectory such as cloning a Git directory into the TemporaryDirectory.
Workaround
A common workaround is to simply not use the tempfile context manager, instead manually deleting the temporary directory when done:
from pathlib import Path
import tempfile
temp_dir = tempfile.TemporaryDirectory()
temp_path = Path(temp_dir.name)
# do stuff in temporary directory# when done with temporary directorytry:
temp_dir.cleanup()
except PermissionError:
pass
If there is a PermissionError, the temporary directory remains until the operating system cleans up the temporary directory.
Matlab “.m” script functionality can be readily extended with Java and Python code.
It may seem obvious to check if Python is available.
It is also important to check if the
Matlab JVM interface
is available as Matlab may be running in
-nojvm
mode.
Starting up Matlab without the JVM disables plotting and Java functions and can be as much as 10x faster to startup Matlab:
matlab -nojvm
Detect if JVM is available from within a Matlab script by:
hasjvm = usejava('jvm');
% boolean
GNU Octave
Octave also has a
JVM interface
that extends Octave functionality.
As with Matlab, the Octave JVM interface availability is checked by
When desired and permitted by the computer filesystem, it’s possible to change file ownership.
This may be necessary when a file was inadvertently created with root / admin ownership by mistake, and it’s necessary for a general user to edit or access the file.
Changing file ownership can have unexpected consequences, like removing the ability of others to access the file or run a program depending on the file.
Therefore, file ownership changes should be done only when necessary and with consideration for others who may depend on the file.
Examples
These examples assume the file “my.txt” is in the current directory and the user logged in should own the file.
These tasks are similarly done in other languages such as: