A CPU resize can also change how your application runs. If the launcher starts one worker per visible CPU and each worker’s numerical library also selects its thread count from that number, both settings can grow after the resize.
I’d record those settings before moving a Python analytics service to a larger dedicated CPU server. Otherwise, a timing comparison mixes added capacity with a different process and thread configuration.
The developer needs to account for parallelism at each layer. A worker process can call a numerical routine that uses several native threads to complete the calculation. BLAS libraries perform underlying linear algebra for tools such as NumPy.
Follow the CPU count through the launcher
Consider a hypothetical batch service that performs matrix calculations for independent customer datasets. Its launcher starts one process per detected vCPU. Each process initializes one OpenBLAS pool, with a thread limit equal to that same vCPU count. Assume no framework reduces those limits across the workers.
After a resize, those hypothetical settings would look like this:
| Available vCPUs | Worker processes | BLAS limit per process | Sum of BLAS limits |
|---|---|---|---|
| 8 | 8 | 8 | 64 |
| 16 | 16 | 16 | 256 |
The VM’s vCPU allocation doubles while the sum of the configured limits quadruples. These illustrative settings describe configured limits. A library can use fewer threads for a particular operation, and workers may enter their calculations at different times.
If the workers run multithreaded calculations together, their pools can compete for the available CPU capacity. Scheduling overhead can consume time that the developer expected to spend on useful calculations.
Scikit-learn’s parallelism documentation distinguishes joblib workers from OpenMP and BLAS threads. An estimator’s n_jobs setting controls the joblib layer. It does not independently set every native library’s thread count.
That distinction gives me a reason to inspect the running configuration before increasing the VM size.
Inspect what the worker loads
With threadpoolctl, NumPy and SciPy installed in the application environment, the thread-pool inspection tool can report the libraries initialized by imports:
python3 -m threadpoolctl -i numpy scipy.linalg
The JSON identifies the native backend, version, threading layer and configured num_threads. I would retain those fields with the worker count. A configured pool size describes the runtime’s settings; CPU utilization needs separate observation.
A fresh command-line process also leaves out changes a framework applies later. For the application, call threadpool_info() inside an initialized worker after the relevant imports and record its output. Check the configuration where the calculation runs.
The pool’s scope affects the arithmetic. Some native libraries share one pool across a process’s Python threads. Others create worker pools for individual calling threads. Switching the application from processes to threads requires checking those semantics again.
In an August pairwise-distance benchmark discussion, cakedev0 reported better scaling in several configurations after limiting BLAS to one thread. The author left the exact cause open and also described a tested case where a single-worker variant won. I’d use that report to choose configurations to compare with my own batch sizes.
Set an explicit policy for the batch
For many independent jobs, I would begin with a fixed process count and one BLAS thread per process. Then I’d compare fewer processes with larger native pools. One large matrix calculation may benefit from a different allocation than many smaller calculations running together.
OpenBLAS documents its startup thread controls. For a pthreads build, set OPENBLAS_NUM_THREADS in the worker launcher’s environment before the library initializes. OpenMP builds use OMP_NUM_THREADS, which can also affect other OpenMP code. Other BLAS backends have their own controls.
I would verify the loaded limits after changing the launcher. A setting copied from a deployment file is insufficient evidence that the worker inherited it or that its framework retained it.
Joblib’s loky backend already limits native threads in child processes, while manually supplied thread environment variables can override that policy. Inspect the effective configuration before adding a second set of controls.
For the comparison, keep the input datasets and installed packages fixed. Record batch completion time and check the calculated results. Include worker startup consistently, or record it separately if the service reuses workers. The trial should represent how that application operates.
I would also watch memory consumption as the process count rises. Each worker’s application state needs room in the VM. A faster calculation that makes the full batch run out of memory would fail this trial.
Carry the policy into the larger VM
Fluence’s dedicated CPU virtual servers support CPU-intensive workers and analytics jobs, with daily billing and API-driven deployment. I’d consider that capacity for steady batch work and carry the recorded worker configuration into the deployment trial.
On a larger VM, first repeat the batch with the same explicit process count and native thread limits. That establishes how the unchanged configuration behaves on the new allocation. Then increase one parallelism setting at a time and inspect the initialized workers again.
If a smaller native pool improves completion time on the current VM, use that result when deciding whether to resize. If the tuned configuration still needs more capacity, repeat the comparison on the candidate server. Neither result establishes a universally best worker count.
Before the next resize, save the vCPU allocation, process count, native backend and thread limits alongside the batch timing. Those records let the next developer see which software settings changed when the server grew.
Top comments (0)