DEV Community: Yevhen Krasnokutsky

Giving Odin Intelligence

Yevhen Krasnokutsky — Tue, 21 May 2024 16:37:17 +0000

My adventure with Odin keeps going. I decided to try something unexpected with Odin - deep learning and AI! "But how?" you might ask. Fortunately, there is a well-known deep-learning framework for running AI models. I'm about ONNX.

Prerequisites

Familiarity with ONNX
Familiarity with C
Odin Overview page has been read

ONNX has a well-documented C API, which makes it easy to port to Odin. At least, I hoped it would be easy. But this is another story.

The plan:

chose the ONNX sample to translate to Odin
make binds to ONNX
???
run model on GPU with the use of Odin

ONNX Sample

I've found a suitable for my idea ONNX example. I'm going to use this example as a strong foundation for the project. But to make things more interesting I'll add just a few enhancements:

listing all available providers
checking if CUDA is available
using CUDA (if available) for running a model

This example code shows the basic usage of ONNX:

Make OrtApi instance
Initialize OrtEnv variable
Configure OrtSessionOptions and OrtSession
Run session (all computations happen here)
Process results and release resources

ONNX Bindings

Long story short. I've already made Odin bindings for ONNX. And I'll use those bindings in the project. I have to say that this post is not about bindings generation. Still, we'll compare the C and Odin versions of the code to get an idea about the usage of C APIs in Odin.

???

In this paragraph, I'll describe only the key parts of the code. A link to the full code is available in the references.

Make `OrtApi` instance

In C OrtApi instance initialized as follows:

#include <onnxruntime_c_api.h>

const OrtApi* g_ort = OrtGetApiBase()->GetApi(ORT_API_VERSION);

On the Odin side, the OrtApi structure is already defined in the bindings file, so we can simply initialize the OrtApi instance and even check if it was initialized successfully:

g_ort: ^OrtApi
if g_ort = OrtGetApiBase().GetApi(ORT_API_VERSION); cast(rawptr)g_ort == nil {
    fmt.eprintln(">>> OrtApi is nil")
    os.exit(1)
}

Listing all available providers

To get a list of all available providers, in C we use:

int providers_count;
char **providers;

CheckStatus(g_ort->GetAvailableProviders(&providers, &providers_count));

printf(">>> Num Providers: %d\n", providers_count);
printf(">>> Providers:\n");

for (int i = 0; i < providers_count; ++i) {
    printf(">>> %d) %s\n", i, providers[i]);
}
CheckStatus(g_ort->ReleaseAvailableProviders(providers, providers_count));

Here, you can see a regular pattern used by the ONNX C API:

Declare variables.
Initialize variables by reference in a function.
Check the status of the operation.
Release allocated resources (if any).

CheckStatus is just a helper function to check the status:

void CheckStatus(OrtStatus *status) {
  if (status != NULL) {
    const char *msg = g_ort->GetErrorMessage(status);
    fprintf(stderr, "%s\n", msg);
    g_ort->ReleaseStatus(status);
    exit(1);
  }
}

The equivalent code in Odin is as follows:

//// Get available providers:
providers_count: c.int
providers: [^]cstring

g_ort.GetAvailableProviders(cast(^^^c.char)(&providers), &providers_count)

defer g_ort.ReleaseAvailableProviders(providers, providers_count)

fmt.println(">>> Available providers:")
for i: c.int = 0; i < providers_count; i += 1 {
    fmt.printfln("\t%d) %s", i, providers[i])
}
/*
    0) TensorrtExecutionProvider
    1) CUDAExecutionProvider
    2) CPUExecutionProvider
*/

The most interesting part of the code snippets above (as well as in the whole project) is how Odin types are mapped to C types.

Mapping between C's int and Odin's c.int is quite straightforward. But what about

char **providers;
OrtStatus* GetAvailableProviders(char*** out_ptr, int* provider_length);

This is quite tricky. On the Odin side, we could use providers: ^^c.char, but we shouldn't. This is because we use providers as a 1D array of C-strings (null-terminated char arrays), but not just double pointer to char. To express array nature of providers, we use multi-pointers. Multi-pointers support indexing in Odin. So, instead of providers: ^^c.char, we use providers: [^]cstring.

But how do I pass providers: [^]cstring to a function with the following signature:

GetAvailableProviders : proc(out_ptr: ^^^c.char, provider_length: ^c.int) -> OrtStatusPtr

To pass providers into GetAvailableProviders() we have to cast providers to ^^^c.char:

cast(^^^c.char)(&providers)

Initialize `OrtEnv` variable

According to the C API, to initialize the OrtEnv we use the following code:

OrtEnv *env;
CheckStatus(g_ort->CreateEnv(ORT_LOGGING_LEVEL_WARNING, "test", &env));

On the Odin side, the equivalent code is quite similar:

env: ^OrtEnv
status: OrtStatusPtr = g_ort.CreateEnv(OrtLoggingLevel.ORT_LOGGING_LEVEL_WARNING, "test", &env)
CheckStatus(g_ort, status)
defer g_ort.ReleaseEnv(env)

CheckStatus() is as follows:

CheckStatus :: proc(ort: ^OrtApi, status: OrtStatusPtr) {
    if status != nil {
        msg: cstring = ort.GetErrorMessage(status)
        fmt.eprintln(msg)
        ort.ReleaseStatus(status)
        os.exit(1)
    }
}

Configure `OrtSessionOptions` and `OrtSession`

`OrtSessionOptions`

To initialize OrtSessionOptions we use the OrtApi instance:

OrtSessionOptions *session_options;
CheckStatus(g_ort->CreateSessionOptions(&session_options));
g_ort->SetIntraOpNumThreads(session_options, 1);
g_ort->SetSessionGraphOptimizationLevel(session_options, ORT_ENABLE_BASIC);

Compare it with Odin's code:

session_options: ^OrtSessionOptions
status = g_ort.CreateSessionOptions(&session_options)
CheckStatus(g_ort, status)
defer g_ort.ReleaseSessionOptions(session_options)

status = g_ort.SetIntraOpNumThreads(session_options, 1)
CheckStatus(g_ort, status)

status = g_ort.SetSessionGraphOptimizationLevel(
    session_options,
    GraphOptimizationLevel.ORT_ENABLE_BASIC,
)
CheckStatus(g_ort, status)

Enable CUDA if available

After OrtSessionOptions is initialized, we can check if CUDA is available and configure ONNX to use CUDA as the provider. Here I show only Odin code because as you already have seen, the difference between Odin and C is minimal.

Let's find out if CUDA is available:

is_cuda_available: bool
for i: c.int = 0; i < providers_count; i += 1 {
    if providers[i] == "CUDAExecutionProvider" {
        is_cuda_available = true
        break
    }
}
fmt.printfln(">>> CUDA is available: %t", is_cuda_available)

And use it as acceleration provider:

if is_cuda_available {
    fmt.println(">>> Setting up CUDA...")
    status = OrtSessionOptionsAppendExecutionProvider_CUDA(session_options, 0)
    CheckStatus(g_ort, status)
}

`OrtSession`

OrtSession is initialized with OrtEnv, model path, and OrtSessionOptions:

session: ^OrtSession
model_path :: "squeezenet1.0-8.onnx"

status = g_ort.CreateSession(env, model_path, session_options, &session)
CheckStatus(g_ort, status)
defer g_ort.ReleaseSession(session)

Run session

So, we've made all necessary preparations to configure OrtSession. We're almost ready to run the mode in GPU.

To run the model, we have to:

specify the model's input and output node names
allocate input tensor
run session
get computation results

Specify model's input and output node names

ONNX model is a computational graph where nodes represent operations, and edges represent the data flow.

We have to specify which node we'd like to put data in and which node we'd like to get the results from. The specification is done by name.

Without diving deep into details, I'll postulate that the input node name is data_0, the output node name is softmaxout_1, and the input node dimensions are 1x3x224x224. There are a few ways to get this information:

read model specification or source code
use handy visualizers, for example, https://netron.app/
use ONNX API functionality to get the information about the model in runtime

Here is a screenshot of SqueezeNet model properties taken on the netron.app:

The syntax of node name declaration is straightforward, so I'll show only output_node_names in C (to be more precise, in C++ as it was implemented in the ONNX example):

std::vector<const char *> output_node_names = {"softmaxout_1"};

On the Odin side, input node name, output node name, and dimensions are initialized as follows:

input_node_dims := make([dynamic]c.int64_t)
defer delete(input_node_dims)
append(&input_node_dims, 1, 3, 224, 224)

input_node_names := make([dynamic]cstring)
defer delete(input_node_names)
append(&input_node_names, "data_0")

output_node_names := make([dynamic]cstring)
defer delete(output_node_names)
append(&output_node_names, "softmaxout_1")

Allocate input tensor

In a real-world scenario, we'd use images to pass to the model and get predictions. But for the sake of the demo, we'll use dummy data to be passed to the model.

SqueezeNet is the model for image classification. The model is trained on the ImageNet dataset. Images in the ImageNet dataset have a specific size of 224x224x3 pixels (224 pixels height, 224 pixels width, and 3 channels). So, we have to allocate and populate a vector of 224*224*3 elements.

size_t input_tensor_size = 224 * 224 * 3;
std::vector<float> input_tensor_values(input_tensor_size);
// initialize input data with values in [0.0, 1.0] (dummy data)
for (size_t i = 0; i < input_tensor_size; i++) {
    input_tensor_values[i] = (float)i / (input_tensor_size + 1);
}

// create input tensor object from data values
OrtMemoryInfo *memory_info;
CheckStatus(g_ort->CreateCpuMemoryInfo(OrtArenaAllocator, OrtMemTypeDefault, &memory_info));

OrtValue *input_tensor = NULL;
CheckStatus(g_ort->CreateTensorWithDataAsOrtValue(
    memory_info, input_tensor_values.data(),
    input_tensor_size * sizeof(float), input_node_dims.data(), 4,
    ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT, &input_tensor));

int is_tensor;
CheckStatus(g_ort->IsTensor(input_tensor, &is_tensor));
assert(is_tensor);
g_ort->ReleaseMemoryInfo(memory_info);

In the C code above (again, it's C++, but who cares), we allocate the input_tensor_values vector of floats with 224*224*3 elements and populate it with dummy data. Then, we "transfer" the data to the ONNX world by calling CreateTensorWithDataAsOrtValue() and allocating input_tensor. As the final step, we check if input_tensor is a reference to the real tensor, and we're good to go.

Here is the same thing in Odin:

input_tensor_size: c.size_t = 224 * 224 * 3

input_tensor_values := make([dynamic]c.float, input_tensor_size)
defer delete(input_tensor_values)

// initialize input data with values in [0.0, 1.0] (dummy data)
for i: c.size_t = 0; i < input_tensor_size; i += 1 {
    input_tensor_values[i] = cast(c.float)i / (cast(c.float)input_tensor_size + 1)
}

// create input tensor object from data values
memory_info: ^OrtMemoryInfo
status = g_ort.CreateCpuMemoryInfo(
    OrtAllocatorType.OrtArenaAllocator,
    OrtMemType.OrtMemTypeDefault,
    &memory_info,
)
CheckStatus(g_ort, status)
defer g_ort.ReleaseMemoryInfo(memory_info)

input_tensor: ^OrtValue
status = g_ort.CreateTensorWithDataAsOrtValue(
    memory_info,
    cast(rawptr)raw_data(input_tensor_values),
    input_tensor_size * size_of(c.float),
    cast(^c.int64_t)raw_data(input_node_dims),
    len(input_node_dims),
    ONNXTensorElementDataType.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT,
    &input_tensor,
)
CheckStatus(g_ort, status)
defer g_ort.ReleaseValue(input_tensor)

is_tensor: c.int
status = g_ort.IsTensor(input_tensor, &is_tensor)
CheckStatus(g_ort, status)
assert(is_tensor == 1, "input_tensor not a tensor")

Pay attention to how we initialize dynamic arrays in Odin, and how we get a pointer to its inner data. To get a pointer to the inner data of the dynamic array we use raw_data() Odin's intrinsic. And then, we cast it to the raw pointer cast(rawptr) which is equivalent to casting to the void* in C.

Run session

At the moment, we've prepared a session and allocated input tensor as well as names and dimensions of input and output nodes. It was a long, tedious, but necessary preparation.

To store inference results, we use the OrtValue pointer. After computations are done, we check if that pointer is a valid tensor. Here is a C sample:

OrtValue *output_tensor = NULL;
CheckStatus(g_ort->Run(session, NULL, input_node_names.data(),
                        (const OrtValue *const *)&input_tensor, 1,
                        output_node_names.data(), 1, &output_tensor));
CheckStatus(g_ort->IsTensor(output_tensor, &is_tensor));
assert(is_tensor);

At this point, Odin's code should be clear and readable. I hope no extra explanations are required at the moment:

output_tensor: ^OrtValue
run_options: ^OrtRunOptions
status = g_ort.Run(
    session,
    run_options,
    raw_data(input_node_names),
    &input_tensor,
    len(input_node_names),
    raw_data(output_node_names),
    len(output_node_names),
    &output_tensor,
)
defer g_ort.ReleaseValue(output_tensor)
CheckStatus(g_ort, status)

status = g_ort.IsTensor(output_tensor, &is_tensor)
CheckStatus(g_ort, status)
assert(is_tensor == 1, "output_tensor not a tensor")

Get computation results

The time has come to get computation results from ONNX world to our world back. For these purposes, the GetTensorMutableData() function is used.

The result of the computations is a float array that has a length of 1000. "Why 1000?" you may ask. This is because there are 1000 classes of images in the ImageNet dataset. The model's output is the vector of probabilities of the image (dummy data in our case) depicting a specific class.

Here is a C sample:

float* floatarr;
CheckStatus(g_ort->GetTensorMutableData(output_tensor, (void**)&floatarr));
assert(std::abs(floatarr[0] - 0.000045) < 1e-6);

// score the model, and print scores for first 5 classes
for (int i = 0; i < 5; i++)
    printf("Score for class [%d] =  %f\n", i, floatarr[i]);

You already know that for array pointers we use multi-pointers in Odin:

floatarr: [^]c.float
status = g_ort.GetTensorMutableData(output_tensor, cast(^rawptr)&floatarr)
CheckStatus(g_ort, status)
assert(abs(floatarr[0]) - 0.000045 < 1e-6, "computition failed")

for i := 0; i < 5; i += 1 {
    fmt.printfln(">>> Score for class [%d] =  %.6f", i, floatarr[i])
}

Run model on GPU with the use of Odin

The full code described in the post is available on GitHub: https://github.com/yevhen-k/onnx-odin-squeezenet-inference-demo

To run the code, you have to:

Use Linux
Have ONNX Runtime on your machine (in the /thirdparty/onnxruntime folder in the current example)
Have GPU with CUDA (optional)

Prepare, compile and run the code.

Clone the repo

git clone https://github.com/yevhen-k/onnx-odin-squeezenet-inference-demo.git
cd onnx-odin-squeezenet-inference-demo

Get SqueezeNet model (using squeezenet version 1.3)

curl https://github.com/onnx/models/raw/main/validated/vision/classification/squeezenet/model/squeezenet1.0-8.onnx -Lso squeezenet1.0-8.onnx

Edit onnxbinding.odin if necessary to adjust package name or foreign import of libonnxruntime.so

// ...
package onnx_bindings
// ...
when ODIN_OS == .Linux do foreign import onnx "/thirdparty/onnxruntime/lib/libonnxruntime.so"
// ...

Build

cd ..
odin build onnx-odin-squeezenet-inference-demo -extra-linker-flags:"-Wl,-rpath=/thirdparty/onnxruntime/lib/" -out:onnx-odin-squeezenet-inference-demo/odin_onnx_example

Run

cd onnx-odin-squeezenet-inference-demo && ./odin_onnx_example

Special Thanks

Special thanks to the Odin community for answering my questions and helping me understand the mechanisms of passing data between C and Odin.

References

Inference of SqueezeNet.onnx model on CUDA with Odin: https://github.com/yevhen-k/onnx-odin-squeezenet-inference-demo/
Odin bindings to the ONNX Runtime (Linux): https://github.com/yevhen-k/onnx-odin-bindings
ONNX SqueezeNet example: https://github.com/microsoft/onnxruntime/blob/v1.4.0/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/C_Api_Sample.cpp
ONNX C API: https://github.com/microsoft/onnxruntime/blob/v1.17.3/include/onnxruntime/core/session/onnxruntime_c_api.h
SqueezeNet: https://paperswithcode.com/method/squeezenet
Model viewer: https://netron.app/

Giving Odin Vision

Yevhen Krasnokutsky — Wed, 08 May 2024 16:52:54 +0000

Disclaimer

This post is about my experience with Odin programming language. So, I won't talk about its features and advantages and provide basic tutorials. There are plenty of materials on those topics.

Prologue

Recently, I was browsing YouTube unconsciously and fell into a pit of recommendations about programming languages. Odin was one of them. Many famous tech YouTubers either talked about Odin or interviewed Ginger Bill (Odin's creator). So, I checked Odin's overwiew page and found the language interesting to give it a try.

Prerequisites

Familiarity with C/C++
Familiarity with OpenCV
Odin's Overview page has been read

Choosing a Project

What kind of project I'd like to make? Hellope (Odin's "Hello World")? Or making a game? Why even a game you may ask. There are some YouTube tutorials and articles on making games with Odin. It's because Odin gives you the ability to use raylib, sdl2, glfw, OpenGL, directx, and even vulkan right out of the box with its Vendor Library Collection.

Given my experience with computer vision, I decided to make a project with the use of OpenCV.

OpenCV Makes Odin See

Project idea: make a simple app that reads frames from a webcam and shows frames in a window.

This is the famous VideoCapture "Hello World" implemented many times in C++ and Python:

C++ example: https://docs.opencv.org/4.x/d8/dfe/classcv_1_1VideoCapture.html
Python example: https://docs.opencv.org/4.x/dd/d43/tutorial_py_video_display.html

But how do I combine Odin and OpenCV to read camera frames? There is no such package in Odin that makes binds with OpenCV. At least I haven't found it.

Fortunately, Odin supports FFI and bindings to C. Unfortunately, OpenCV doesn't expose C API.

So, here was my plan:

Make C wrappers for all necessary C++ functions
Compile wrapper lib to .a or .so file
Describe all wrapper functions in Odin foreign block
Implement app
Run app

1. Make C wrappers for all necessary C++ functions

To make different programming languages and OS able to talk with each other, they should share the same language. C is such a lingua franca. It might not be perfect, but it's better than nothing.

The goal of the section is to express all C++ types and functionality in plane C.

To implement all required functionality of the app we'll need the following functions and types:

cv::Mat type to store read data
cv::imread() reads an image from the file and returns cv::Mat object
empty() method of cv::Mat object to check if it's empty
cv::namedWindow() function to create a named window
cv::imshow() function to show the image on named window
cv::destroyWindow() function to destroy the window and release its resources
cv::waitKey() function to wait for a pressed key
cv::imwrite() function to write an image to a file
cv::VideoCapture type to manage video camera capture
open() method of cv::VideoCapture to open a camera for video capturing
release() method of cv::VideoCapture to release and deallocate capture resources
isOpened() method of cv::VideoCapture to check if capture is opened
read() method of cv::VideoCapture to read a frame from the camera and store it in cv::Mat object

Let's make the corresponding header imageprocessing.h and implementation imageprocessing.cpp files.

// `imageprocessing.h`

#ifndef IMPROC_H
#define IMPROC_H
#include <stdbool.h>

typedef void *Mat;
typedef void *VideoCapture;

#ifdef __cplusplus
extern "C" {
#endif

Mat cv_new_mat();

Mat cv_image_read(const char *file, int flags);

bool cv_mat_isempty(Mat mat);

void cv_named_window(const char *name);

void cv_image_show(const char *name, Mat img);

int cv_wait_key(int delay);

void cv_destroy_window(const char *name);

bool cv_image_write(const char *filename, Mat img);

void cv_free_mem(void *data);

VideoCapture cv_new_videocapture();

bool cv_videocapture_open(VideoCapture cap, int device_id, int api_id);

void cv_videocapture_release(VideoCapture cap);

bool cv_videocapture_isopened(VideoCapture cap);

bool cv_videocapture_read(VideoCapture cap, Mat frame);

#ifdef __cplusplus
}
#endif

#endif

// `imageprocessing.cpp`
#include "imageprocessing.h"
#include <opencv2/highgui.hpp>
#include <opencv2/imgcodecs.hpp>

Mat cv_new_mat() { return new cv::Mat(); }

Mat cv_image_read(const char *file, int flags) {
  cv::Mat image = cv::imread(file, flags);
  return new cv::Mat(image);
}

bool cv_mat_isempty(Mat mat) {
  cv::Mat *m = static_cast<cv::Mat *>(mat);
  return m->empty();
}

void cv_named_window(const char *name) {
  cv::namedWindow(name);
}

void cv_image_show(const char *name, Mat img) {
  cv::Mat *image = static_cast<cv::Mat *>(img);
  std::string win_name{name};
  cv::imshow(name, *image);
}

int cv_wait_key(int delay) { return cv::waitKey(delay); }

void cv_destroy_window(const char *name) {
  cv::destroyWindow(name);
}

bool cv_image_write(const char *filename, Mat img) {
  cv::Mat *image = static_cast<cv::Mat *>(img);
  return cv::imwrite(filename, *image);
}

void cv_free_mem(void *data) { free(data); }

VideoCapture cv_new_videocapture() { return new cv::VideoCapture(); }

bool cv_videocapture_open(VideoCapture cap, int device_id, int api_id) {
  cv::VideoCapture *capture = static_cast<cv::VideoCapture *>(cap);
  return capture->open(device_id, api_id);
}

void cv_videocapture_release(VideoCapture cap) {
  cv::VideoCapture *capture = static_cast<cv::VideoCapture *>(cap);
  capture->release();
}

bool cv_videocapture_isopened(VideoCapture cap) {
  cv::VideoCapture *capture = static_cast<cv::VideoCapture *>(cap);
  return capture->isOpened();
}

bool cv_videocapture_read(VideoCapture cap, Mat frame) {
  cv::Mat *image = static_cast<cv::Mat *>(frame);
  cv::VideoCapture *capture = static_cast<cv::VideoCapture *>(cap);
  return capture->read(*image);
}

As you can see, we don't operate with OpenCV's cv::Mat and cv::VideoCapture types directly. Instead, we use the so-called opaque pointer pattern (see typedef void *Mat; and typedef void *VideoCapture;) that helps us encapsulate (hide) C++ details of implementation and use only C types.

2. Compile wrapper lib to `.a` or `.so` file

All operations below assume:

you're running Linux
OpenCV is installed

Compiling static library

$ gcc -c -Wall -Werror -fpic -I/usr/include/opencv4 -I/usr/include/opencv4/opencv2 `pkg-config --libs opencv4` -lstdc++ imageprocessing.cpp
$ ar rcs imageprocessing.a imageprocessing.o

Compiling shared library

$ gcc -c -Wall -Werror -fpic -I/usr/include/opencv4 -I/usr/include/opencv4/opencv2 `pkg-config --libs opencv4` -lstdc++ imageprocessing.cpp
$ gcc -shared -o libimageprocessing.so imageprocessing.o

3. Describe all wrapper functions in Odin foreign block

Odin has IMO a great feature that significantly simplifies usage of both static and shared libraries. Yes, I'm about foreign import! The foreign import block simply expects a path to the shared or static library. If I understood correctly, the path to the static library should be specified relative to the package, and the path to the shared library should be specified relative to a compiled executable.

Let's describe wrapped-in C functionality in Odin lang. In the example below I use linking with the static library in the foreign import block.

// imageprocessing_bindings.odin

package imageprocessing

import "core:c"

when ODIN_OS == .Linux do foreign import cv "imageprocessing.a"
// when ODIN_OS == .Linux do foreign import cv "libimageprocessing.so"

Mat :: distinct rawptr
VideoCapture :: distinct rawptr

// Just copy-paste from https://docs.opencv.org/4.9.0/d8/d6a/group__imgcodecs__flags.html#ga61d9b0126a3e57d9277ac48327799c80
ImageReadModes :: enum c.int {
    IMREAD_UNCHANGED           = -1, //!< If set, return the loaded image as is (with alpha channel,
    //!< otherwise it gets cropped). Ignore EXIF orientation.
    IMREAD_GRAYSCALE           = 0, //!< If set, always convert image to the single channel
    //!< grayscale image (codec internal conversion).
    IMREAD_COLOR               = 1, //!< If set, always convert image to the 3 channel BGR color image.
    IMREAD_ANYDEPTH            = 2, //!< If set, return 16-bit/32-bit image when the input has the
    //!< corresponding depth, otherwise convert it to 8-bit.
    IMREAD_ANYCOLOR            = 4, //!< If set, the image is read in any possible color format.
    IMREAD_LOAD_GDAL           = 8, //!< If set, use the gdal driver for loading the image.
    IMREAD_REDUCED_GRAYSCALE_2 = 16, //!< If set, always convert image to the single channel grayscale
    //!< image and the image size reduced 1/2.
    IMREAD_REDUCED_COLOR_2     = 17, //!< If set, always convert image to the 3 channel BGR color image
    //!< and the image size reduced 1/2.
    IMREAD_REDUCED_GRAYSCALE_4 = 32, //!< If set, always convert image to the single channel grayscale
    //!< image and the image size reduced 1/4.
    IMREAD_REDUCED_COLOR_4     = 33, //!< If set, always convert image to the 3 channel BGR color image
    //!< and the image size reduced 1/4.
    IMREAD_REDUCED_GRAYSCALE_8 = 64, //!< If set, always convert image to the single channel grayscale
    //!< image and the image size reduced 1/8.
    IMREAD_REDUCED_COLOR_8     = 65, //!< If set, always convert image to the 3 channel BGR color image
    //!< and the image size reduced 1/8.
    IMREAD_IGNORE_ORIENTATION  = 128, //!< If set, do not rotate the image
    //!< according to EXIF's orientation flag.
}


// Just copy-paste https://docs.opencv.org/4.9.0/d4/d15/group__videoio__flags__base.html#ga023786be1ee68a9105bf2e48c700294d
VideoCaptureAPIs :: enum c.int {
    CAP_ANY           = 0, //!< Auto detect == 0
    CAP_VFW           = 200, //!< Video For Windows (obsolete, removed)
    CAP_V4L           = 200, //!< V4L/V4L2 capturing support
    CAP_V4L2          = CAP_V4L, //!< Same as CAP_V4L
    CAP_FIREWIRE      = 300, //!< IEEE 1394 drivers
    CAP_FIREWARE      = CAP_FIREWIRE, //!< Same value as CAP_FIREWIRE
    CAP_IEEE1394      = CAP_FIREWIRE, //!< Same value as CAP_FIREWIRE
    CAP_DC1394        = CAP_FIREWIRE, //!< Same value as CAP_FIREWIRE
    CAP_CMU1394       = CAP_FIREWIRE, //!< Same value as CAP_FIREWIRE
    CAP_QT            = 500, //!< QuickTime (obsolete, removed)
    CAP_UNICAP        = 600, //!< Unicap drivers (obsolete, removed)
    CAP_DSHOW         = 700, //!< DirectShow (via videoInput)
    CAP_PVAPI         = 800, //!< PvAPI, Prosilica GigE SDK
    CAP_OPENNI        = 900, //!< OpenNI (for Kinect)
    CAP_OPENNI_ASUS   = 910, //!< OpenNI (for Asus Xtion)
    CAP_ANDROID       = 1000, //!< MediaNDK (API Level 21+) and NDK Camera (API level 24+) for Android
    CAP_XIAPI         = 1100, //!< XIMEA Camera API
    CAP_AVFOUNDATION  = 1200, //!< AVFoundation framework for iOS (OS X Lion will have the same API)
    CAP_GIGANETIX     = 1300, //!< Smartek Giganetix GigEVisionSDK
    CAP_MSMF          = 1400, //!< Microsoft Media Foundation (via videoInput). See platform specific notes above.
    CAP_WINRT         = 1410, //!< Microsoft Windows Runtime using Media Foundation
    CAP_INTELPERC     = 1500, //!< RealSense (former Intel Perceptual Computing SDK)
    CAP_REALSENSE     = 1500, //!< Synonym for CAP_INTELPERC
    CAP_OPENNI2       = 1600, //!< OpenNI2 (for Kinect)
    CAP_OPENNI2_ASUS  = 1610, //!< OpenNI2 (for Asus Xtion and Occipital Structure sensors)
    CAP_OPENNI2_ASTRA = 1620, //!< OpenNI2 (for Orbbec Astra)
    CAP_GPHOTO2       = 1700, //!< gPhoto2 connection
    CAP_GSTREAMER     = 1800, //!< GStreamer
    CAP_FFMPEG        = 1900, //!< Open and record video file or stream using the FFMPEG library
    CAP_IMAGES        = 2000, //!< OpenCV Image Sequence (e.g. img_%02d.jpg)
    CAP_ARAVIS        = 2100, //!< Aravis SDK
    CAP_OPENCV_MJPEG  = 2200, //!< Built-in OpenCV MotionJPEG codec
    CAP_INTEL_MFX     = 2300, //!< Intel MediaSDK
    CAP_XINE          = 2400, //!< XINE engine (Linux)
    CAP_UEYE          = 2500, //!< uEye Camera API
    CAP_OBSENSOR      = 2600, //!< For Orbbec 3D-Sensor device/module (Astra+, Femto, Astra2, Gemini2, Gemini2L, Gemini2XL, Femto Mega) attention: Astra2, Gemini2, and Gemini2L cameras currently only support Windows and Linux kernel versions no higher than 4.15, and higher versions of Linux kernel may have exceptions.
}

@(default_calling_convention = "c")
foreign cv {
    cv_new_mat :: proc() -> Mat ---
    cv_image_read :: proc(file: cstring, flags: c.int) -> Mat ---
    cv_mat_isempty :: proc(mat: Mat) -> c.bool ---
    cv_named_window :: proc(name: cstring) ---
    cv_image_show :: proc(name: cstring, img: Mat) ---
    cv_wait_key :: proc(delay: c.int) -> c.int ---
    cv_destroy_window :: proc(name: cstring) ---
    cv_image_write :: proc(filename: cstring, img: Mat) -> c.bool ---
    cv_free_mem :: proc(data: rawptr) ---

    cv_new_videocapture :: proc() -> VideoCapture ---
    cv_videocapture_open :: proc(capture: VideoCapture, device_id, api_id: c.int) -> c.bool ---
    cv_videocapture_release :: proc(capture: VideoCapture) ---
    cv_videocapture_isopened :: proc(capture: VideoCapture) -> c.bool ---
    cv_videocapture_read :: proc(capture: VideoCapture, frame: Mat) -> c.bool ---
}

As you can see, foreign import points to the library where to look for symbols (functions), and the foreign block describes all functions with their signatures that could be found in the library with Odin lang. Each function declaration ends with --- which means that the function body is located elsewhere (in a shared or static library in our case).

To "emulate" opaque types, I use Mat :: distinct rawptr construction, which is the same as typedef void *Mat; on C's side.

Odin's types are compatible with C's types. The compatibility makes communication between Odin and C smooth. "core:c" package is especially helpful in the task of type compatibility with C. Here is a simple type conversion table between C types and corresponing Odin types.

Odin Type	C Type
cstring	const char *
c.int	int
c.bool	bool
rawptr	void*

4. Implement app

Here is an implementation of the VideoCapture demo. As a bonus, you can find functionality to read and show images.


package imageprocessing

import "core:c"
import "core:fmt"
import "core:os"


demo_read_image :: proc() {
    image_path :: "path/to/image.png"
    image := cv_image_read(image_path, cast(c.int)(ImageReadModes.IMREAD_COLOR))
    defer cv_free_mem(image)

    if cv_mat_isempty(image) {
        fmt.eprintfln("Error: can't find or open an image: %s", image_path)
        os.exit(1)
    }

    window_name :: "The Image"
    cv_named_window(window_name)
    defer cv_destroy_window(window_name)

    cv_image_show(window_name, image)
    cv_wait_key(0)
}

demo_read_camera :: proc() {

    frame := cv_new_mat()
    defer cv_free_mem(frame)

    capture := cv_new_videocapture()
    defer {
        cv_videocapture_release(capture)
        cv_free_mem(capture)
    }

    device_id: c.int = 0
    api_id: c.int = cast(c.int)VideoCaptureAPIs.CAP_ANY

    cv_videocapture_open(capture, device_id, api_id)

    window_name :: "Camera Video"
    cv_named_window(window_name)
    defer cv_destroy_window(window_name)

    if !cv_videocapture_isopened(capture) {
        fmt.eprintfln(
            "Error: can't open camera stream for device_id=%d and api_id=%d",
            device_id,
            api_id,
        )
        os.exit(1)
    }

    fmt.println(">>> Reading frames...")
    for {
        cv_videocapture_read(capture, frame)
        if cv_mat_isempty(frame) {
            fmt.eprintln("Error: empty frame... exiting")
            break
        }
        cv_image_show(window_name, frame)
        c := cv_wait_key(25)
        if c == 27 {    // ESC key pressed
            break
        }
    }

}

main :: proc() {
    fmt.println(">>> OpenCV Odin Started...")
    // demo_read_image()
    demo_read_camera()

    fmt.println(">>> OpenCV Odin Ended...")
}

The code is quite self-descriptive. We have main proc. It is the starting point of the application. Moreover, all of Odin's code is a word-for-word translation from C. In the main proc, there are two functions:

demo_read_image() for reading images (commented out)
demo_read_camera() for reading camera frames

5. Run app

For both static and dynamic linking, the build is done with the following code:

odin build ../opencv-odin-demo -extra-linker-flags:"`pkg-config --libs opencv4` -lstdc++"

As you can see, the Odin compiler can interact with the linker and its flags.

This will generate opencv-odin-demo executable. And then you can run the app:

./opencv-odin-demo

That's it! It just works!

Full Code

The code is available here: https://github.com/yevhen-k/opencv-odin-demo/

Epilogue

Odin has convenient and neat features alongside its simplicity. Linking with FFI was easier than I thought before starting the project.

The code I gave most likely isn't idiomatic and good from Odin's perspective, but it gives you a scratch to start from. I hope it might help you to make your own OpenCV+Odin projects soon.

Mom look! I'm a real programmer, not a frameworker!

DEV Community: Yevhen Krasnokutsky

Giving Odin Intelligence

Prerequisites

ONNX Sample

ONNX Bindings

???

Make OrtApi instance

Listing all available providers

Initialize OrtEnv variable

Configure OrtSessionOptions and OrtSession

OrtSessionOptions

Enable CUDA if available

OrtSession

Run session

Specify model's input and output node names

Allocate input tensor

Run session

Get computation results

Run model on GPU with the use of Odin

Special Thanks

References

Giving Odin Vision

Disclaimer

Prologue

Prerequisites

Choosing a Project

OpenCV Makes Odin See

1. Make C wrappers for all necessary C++ functions

2. Compile wrapper lib to .a or .so file

Compiling static library

Compiling shared library

3. Describe all wrapper functions in Odin foreign block

4. Implement app

5. Run app

Full Code

Epilogue

Make `OrtApi` instance

Initialize `OrtEnv` variable

Configure `OrtSessionOptions` and `OrtSession`

`OrtSessionOptions`

`OrtSession`

2. Compile wrapper lib to `.a` or `.so` file