Thomas Weise (汤卫思)

A professor at Hefei University.

3 minute read

Sometimes we want to execute a command or program that may use too many CPU cores or too much memory and thus potentially crash our system. Instead of waiting for this to happen, we can also limit the number of CPU cores as well as the memory that a process can use/allocate. If the program can smoothly run within such limits, it will complete normally. If it tries to allocate more memory than what we permit, it will most likely simply crash. If we have 8 logical CPU cores, but we permit the program to only use 4, then it can also never happen that the computer becomes too slow because of the computational load. Regardless what happens, our computer will remain fully usable.

Here I provide the little script runWithLimits.sh, which does this in the terminal. It takes as parameters

  • The number of CPU cores that the command can use. If you specify 0, then as default we will use max(1, (N-1)/2-1) as default, where N is the number of available cores.
  • The maximum permitted amount of memory that can be used. Valid units are B, K, M, G, T, standing for byte, kilobyte, megabyte, gigabyte, and terabyte, respectively.
  • The command to execute and its arguments.

For example, runWithLimits.sh 1 2M echo "Hello World!" will run the program echo with parameter Hello World! on at most 1 CPU core and permitting the program to use at most 2 megabytes of memory. This will normally work, but if you try runWithLimits.sh 1 2B echo "Hello World!", it will likely crash, because 2 bytes of memory are not enough even for echo…

Here you can download this script and the complete collection of my personal scripts is available here.

#!/bin/bash

# This script run a command while limiting its number of CPU cores and memory consumption.
#
# Parameters:
# 1. The maximum number of CPU cores to use.
# 2. The maximum amount of memory to use, units are B for bytes,
#    K for kilobytes, M for megabytes, G for gigabytes, T for terabytes
#    and so on.
# After these two parameters, the command and all of its arguments follow.

# strict error handling
set -o pipefail  # trace ERR through pipes
set -o errtrace  # trace ERR through 'time command' and other functions
set -o nounset   # set -u : exit the script if you try to use an uninitialized variable
set -o errexit   # set -e : exit the script if any statement returns a non-true return value

if [ $# -lt 3 ]; then
    echo "$(date +'%0Y-%0m-%0d %0R:%0S'): Execute command with limits on CPU cores and memory."
    echo "Parameters:"
    echo " 1. maximum number of CPU cores; 0 for reasonable default"
    echo " 2. maximum amount of memory (in B, K, M, or G (bytes))"
    echo " 3., 4., 5., and so on: the command and its arguments"
    exit 1
fi

echo "$(date +'%0Y-%0m-%0d %0R:%0S'): Welcome to the program execution script for command '${@:3}'."
cpus="$1"
if [ "$cpus" -lt 1 ]; then
    if command -v nproc &> /dev/null; then
        cpus="$(nproc --all)"
        cpus="$((cpus - 1))"
        cpus="$((cpus / 2))"
        cpus="$((cpus - 1))"
        if [ $cpus -le 1 ]; then
            cpus=1
        fi
    else
        cpus=1
    fi
fi
echo "$(date +'%0Y-%0m-%0d %0R:%0S'): We will permit the use of at most '$cpus' CPU cores."

memory="$2"
echo "$(date +'%0Y-%0m-%0d %0R:%0S'): We will permit the use of at most '$memory' of memory."

set +o errexit
nice -n 19 taskset -c "$cpus" systemd-run --scope -p MemoryMax="$memory" --user "${@:3}"
retcode="$?"
set -o errexit

echo "$(date +'%0Y-%0m-%0d %0R:%0S'): The program has ended with exit code '$retcode'."
exit "$retcode"

You May Also Enjoy

Finding Non-ASCII Characters in Text Documents

1 minute read

Sometimes we want to find whether a text documents contains only ASCII characters or whether there is some non-ASCII character in it. You see, ASCII is an old 7 bit-based encoding for text. Hence, it supports a very limited range of different characters and certainly not fancy stuff like “ä” let alone “你好”. Most tools today can understand the Unicode character set encoded with UTF-8, which is more or less compatible to that and can store all such characters. Some software, however, still takes offense if characters appear that are outside the ASCII range, e.g., in names of functions in some programming languages. Other software can deal with Unicode well, but dislikes only certain characters in certain places. If we have some strange LaTeX error when compiling a paper draft, sometimes it can help to check whether we accidentally had maybe a “。” instead of a “.” slip in somewhere.

Converting Vector Graphics Formats

3 minute read

Most graphics that we use in publications or reports are vector graphics, such as SVGs, PDFs, sometimes also WMFs, EMFs, EPSs, or PS files. It is often necessary to convert between them, most often between PDF and SVG. This can be done via Inkscape. Here I provide the little script vec2vec.sh, which does this in the terminal. It basically just executes Inkscape with the correct parameters.

(Down)scaling Images in PDF Documents

3 minute read

Sometimes we have PDF documents and want to submit through a web form, but it is too big. Often, this size is caused by the images inside the PDF. We may reduce their resolution, their “dots per inch” (DPI) value, and then the PDF gets smaller. This can be done via Ghostscript. Here I provide the little script scalePdf.sh, which does this in the terminal. It basically just executes Ghostscript with the correct parameters.

Converting MS Office Documents to PDF

3 minute read

Sometimes you have a Microsoft Office document, say an MS Word or MS PowerPoint file, and want to convert it to PDF. If you do not have Microsoft Office installed, then you can do this via LibreOffice, which is free open source. Here I provide the little script office2pdf.sh, which does this in the terminal. It basically just executes LibreOffice with the correct parameters.