Parallel monitoring of CPU usage with Python's psutil

I wrote a script that periodically monitors CPU utilization while running some process in Python.

Multithreading allows periodic execution without stopping the main process. It may be useful when you want to report time series data of heavy computer load. The full code is at the bottom.

Library Installation

The library psutil can be used to get the CPU utilization in Python. You can install it via pip.

pip install psutil

The basic usage is as follows.

# percpu=True returns a list of utilization (%) per core
>>> psutil.cpu_percent(interval=1, percpu=True)
[7.0, 1.0, 8.9, 0.0, 5.0, 1.0, 4.0, 0.0]

# percpu=False returns an average utilization (%)
>>> psutil.cpu_percent(interval=1, percpu=False)
8.9

# Smaller interval(sec) shortens the measurement time, but also increases the error.
>>> psutil.cpu_percent(interval=0.5, percpu=True) 
[6.0, 0.0, 7.8, 1.9, 4.0, 0.0, 5.8, 0.0]

Parallel processing

This time, we will implement parallel processing using the native library threading.
Thread instances can be created and executed as follows.

m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
m.start()

As arguments for threading.Thread(), target is a method to execute and args is the argument for the method.

monitor_cpu is the method for monitoring.

Monitoring method

Manage monitoring threads using threading.Event.
threading.Event is used to make a thread wait until an event occurs, and then resume the waiting thread when an event is raised by another thread.

We use 2 events.

• wait(timeout) : make the current thread wait until an event occurs or timeout seconds elapse.
• set() : fires an event.

def monitor_cpu(initial_time):
    print("START monitor_cpu")
    while not event.wait(1):  # Wait for 1 second, if no event occurs, execute inside the loop and wait again
        elapsed_time = time.time() - initial_time
        cpu_percent = psutil.cpu_percent(percpu=True)
        cpu_percent = '\t'.join(["{:10.4f}".format(v) for v in cpu_percent])
        print("time:", int(elapsed_time), cpu_percent)
    print("END monitor_cpu")  # When an event occurs, exit the loop and terminate execution


if __name__=="__main__":
    event = threading.Event()
    initial_time = time.time()
    m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
    m.start()

    # Main process

    event.set()  # Fires an event

Inside monitor_cpu, an infinite loop runs with an execution interval of 1 second. When an event occurs, it breaks the loop and terminates execution.

def monitor_cpu(initial_time):
    print("START monitor_cpu")
    while flag:
        time.sleep(1)
        elapsed_time = time.time() - initial_time
        cpu_percent = psutil.cpu_percent(percpu=True)
        cpu_percent = '\t'.join(["{:10.4f}".format(v) for v in cpu_percent])
        print("time:", int(elapsed_time), cpu_percent)
    print("END monitor_cpu")

if __name__=="__main__":
    event = threading.Event()
    initial_time = time.time()
    flag = True
    m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
    m.start()
    tmp = 0
    for i in range(100000000):
        tmp = i+i
    flag = False

Entire code

import threading
import time
import psutil

def monitor_cpu(initial_time):
    print("START monitor_cpu")
    while not event.wait(1):
        elapsed_time = time.time() - initial_time
        cpu_percent = psutil.cpu_percent(percpu=True)
        cpu_percent = '\t'.join(["{:10.4f}".format(v) for v in cpu_percent])
        print("time:", int(elapsed_time), cpu_percent)
    print("END monitor_cpu")

if __name__=="__main__":
    event = threading.Event()
    initial_time = time.time()
    m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
    m.start()
    tmp = 0
    for i in range(100000000):
        tmp = i+i
    event.set()

Results

From left to right: time (seconds), CPU1 utilization (%), CPU2 utilization ... CPU8 utilization.

START monitor_cpu
time: 1  52.0     1.0    29.7     1.0    28.7     2.0    30.4     0.0
time: 2  45.0     3.0    36.3     3.0    32.4     1.0    29.4     1.0
time: 3  43.6     2.0    31.0     1.0    33.7     0.0    27.7     2.0
time: 4  45.5     2.0    25.2     1.0    22.8     0.0    19.8     0.0
time: 5  41.6     1.0    26.0     1.0    26.7     1.0    20.8     1.0
time: 6  46.1     5.0    34.7     3.0    38.0     3.0    31.7     4.0
time: 7  63.0    10.0    52.5    10.0    55.4    10.9    55.0    10.0
time: 8  51.5     6.9    36.0     4.9    41.2     5.0    39.6     4.9
time: 9  55.0     0.0    20.6     1.0    26.0     0.0    14.9     0.0
time: 10  49.5    2.9    22.8     1.0    25.2     1.0    23.5     1.0
time: 11  43.6    1.0    32.7     0.0    28.3     0.0    23.8     1.0
time: 12  47.5    2.9    22.8     1.0    20.6     2.0    25.0     2.0
END monitor_cpu