DEV Community

PixLab | Symisc Systems profile image Vincent
Vincent for PixLab | Symisc Systems

Posted on

Making C++ and Python Work Together a Little Better with NumPy Files

#cpp #python #numpy #interoperability

SyNumpy

A lot of modern systems are split between two worlds:

  • Python for experimentation, model training, data science workflows, and notebooks
  • C++ for performance-sensitive code, native applications, embedded systems, or production pipelines

That split is completely normal. It is also where a lot of unnecessary friction shows up.

You prototype an ML workflow in Python, generate arrays with NumPy, and then at some point you need those same arrays inside a C++ application. Suddenly the clean notebook pipeline turns into format conversion, custom parsers, awkward glue code, or an extra serialization layer you did not really want.

That is the kind of problem syNumpy tries to solve.

The Interoperability Problem

Python and C++ are often used in the same stack, but they do not naturally speak the same data format conventions out of the box.

When the Python side already uses NumPy, the simplest answer is often to keep using NumPy’s own file format rather than introducing something heavier. The .npy format is compact, well understood, and practical for moving numeric arrays between environments.

That becomes useful in cases like:

  • exporting embeddings from Python and consuming them in C++
  • moving preprocessed tensors into native inference code
  • exchanging OCR or vision-related feature vectors across services
  • saving intermediate numerical results from one environment and loading them in another

The goal is not to replace every other interchange format. The goal is to make one common workflow much less painful.

Where syNumpy Fits

syNumpy is a standalone C++17 library for reading and writing NumPy .npy files.

It gives native C++ code a direct way to consume or produce NumPy arrays without adding a large dependency stack or building a custom bridge around Python.

A few things I like about this approach:

  • it stays close to an already common Python workflow
  • it keeps the integration path simple
  • it is easy to vendor into an existing C++ project
  • it avoids inventing a new format just to move arrays around

If your Python code already emits NumPy arrays, letting your C++ side read the same .npy files directly is often the lowest-friction option.

A Small Example

Here is a simple C++ example using syNumpy:

#include "synumpy.hpp"
#include <vector>

int main() {
    // Save a small array to disk
    std::vector<float> values = {1.0f, 2.0f, 3.0f};
    syNumpy::saveNpy("floats.npy", values);

    // Load it back
    syNumpy::NpyArray arr = syNumpy::loadNpy("floats.npy");
    std::vector<float> roundtrip = arr.asVector<float>();

    return roundtrip.size() == 3 ? 0 : 1;
}
Enter fullscreen mode Exit fullscreen mode

That may look small, but it is exactly the kind of thing that helps when your Python tooling and native C++ code need to cooperate without ceremony.

Why This Matters in Practice

This kind of interoperability is not theoretical.

syNumpy is used internally by FACEIO for facial feature extraction workflows, and also inside PixLab production systems tied to vision, document processing, and related internal workflows.

That matters because it suggests the library was shaped by actual production needs, not just by a synthetic “hello world” use case.

In practice, when a library is used in real pipelines, the important details tend to be the boring ones:

  • predictable behavior
  • explicit failures on malformed input
  • small API surface
  • easy integration into existing build systems

That is usually what makes interoperability tools worth keeping around.

Why .npy Is a Good Middle Ground

There are many ways to move structured data between systems, but arrays are a special case. When the core unit of exchange is already a NumPy array, using .npy directly is often more natural than wrapping it in something more generic.

It is a nice middle ground between:

  • staying inside Python-only tooling
  • building a custom binary format
  • introducing a bigger serialization framework than the problem requires

For teams working across both ecosystems, simple things like this compound over time. Less conversion code means less maintenance, fewer failure points, and fewer hidden assumptions about layout or dtype handling.

If You Want to Explore It

You can find syNumpy here:

And if you want to see where this kind of tooling shows up in real products:

Closing

C++ and Python are going to keep living side by side for a long time. That is not a problem by itself. The real problem is when moving data between them becomes harder than it should be.

A small library that reads and writes NumPy .npy files will not solve every interoperability problem. But for a very common one, it can remove a surprising amount of friction.

Top comments (0)