DEV Community

x0rw
x0rw

Posted on • Edited on

IpcArc<T>: A Design Document for Inter-Process Communication in Rust

#rust #opensource #performance #linux

(This document is a preliminary draft and has not been implemented or finished yet. Feedback and comments are welcome)
Linked github repo: https://github.com/x0rw/ipc-arc

Abstract

Rust provides safe and ergonomic primitives such as Arc<T> for sharing data between threads within the same process. However, there is no standard mechanism for safely and transparently sharing data across process boundaries. While developers can use operating system facilities like shared memory, pipes, and sockets to achieve inter-process communication (IPC), these tools are often low-level, error-prone, and platform-specific.

The purpose of this design document is to propose and outline a new crate: IpcArc<T>. This crate introduces a high-level abstraction that allows developers to share data across processes with a familiar and ergonomic interface, similar to Arc. It leverages shared memory under the hood and aims to provide zero-copy, lock-free (when possible), and safe inter-process sharing of Rust values

Goals

The main goal of this crate is to provide a way to safetly and ergonomically share data between processes leveraging shared memory.

  • Safe inter-process communication: preventing common IPC errors like memory corruption and data races,
  • Ownership managment: reference counting.
  • Synchonisation mechanisms: inter-process mutexes.
  • Ergonomic API:
  • zero-copy access: on POD types.

High level design

IpcArc<T> will primarily leverage memory-mapped files:

  • When you create IpcArc<T> instance a shared-memory region will be allocated large enough to hold our type T and its metadata
  • Other processes will be able to attach to this memory region by a unique identifier specified by the owning process
  • Each shared type will be reference counted in the shared memory itself
  • Synchronization: through pshared mutex

Proposed API design:

The owning process can do

let shared_data: IpcArc<u64> = IpcArc::new("shared_identifier_1", 77u64).expect("failed to create a shared memory region...");
Enter fullscreen mode Exit fullscreen mode

On another process:

let shared_data: IpcArc<u64> = IpcArc::load("shared_identifier_1").expect("Memory region doesn't exist...");
Enter fullscreen mode Exit fullscreen mode

then on either processes we can aquire and mutate:

let lock = shared_data.aquire().unwrap();   // aquire the lock 
lock.as_ref_mut() = 344;                    // mutate
lock.drop().unwrap();                       // drop the lock
Enter fullscreen mode Exit fullscreen mode

Suggested structure design: 

struct IpcArc<T> {
    ptr: *mut T,                  // ptr to the data in shared mem
    mutex: IpcArcMutex,           // inter-process mmutex in shared mem
    ref_count: *const AtomicUsize,// atomic ref count in mem too
    _phantom: PhantomData<T>,
}
Enter fullscreen mode Exit fullscreen mode

Shared memory layout 

+-----------------------------------+
|       SHARED MEMORY SEGMENT       |
|   (Accessible by all processes)   |
|      through an identifier        |
+-----------------------------------+
|                                   |
|  +-----------------------------+  |
|  |           HEADER            |  |
|  | (Magic Number / size???)    |  |
|  +-----------------------------+  |
|                                   |
|  +-----------------------------+  |
|  |     INTER-PROCESS ATOMIC    |  |
|  |       REFERENCE COUNT       |  |
|  +-----------------------------+  |
|                                   |
|  +-----------------------------+  |
|  |     INTER-PROCESS MUTEX     |  |
|  |         STATE               |  |
|  +-----------------------------+  |
|                                   |
|  +-----------------------------+  |
|  |           Type T            |  |
|  +-----------------------------+  |
|                                   |
+-----------------------------------+
Enter fullscreen mode Exit fullscreen mode

Crates and Syscall:

  • shm_open: Create or/and open a shared memory object.
  • ftruncate: Resize the requested memory.
  • mmap: maps the shared memory object to the current process.
  • munmap, close: close and unmap the memory
  • shm_unlink: delete the shared memory object when the ref count reaches 0.

we will mainly use nix crate for a safe and ergonomic wrapper around these syscalls.

Top comments (0)