WWDC 2025 - What’s new in BNNS Graph

At WWDC 2025, Apple unveiled BNNSGraphBuilder, a powerful new Swift API that revolutionizes how iOS developers implement machine learning inference directly on the CPU. This enhancement to the existing BNNSGraph framework represents a significant leap forward in making high-performance ML accessible to Swift developers.

Understanding BNNS and BNNSGraph

Basic Neural Network Subroutines (BNNS) is Apple's machine learning library designed for CPU-based inference. Part of the Accelerate framework, BNNS excels in scenarios requiring:

• Real-time audio processing
• Low-latency inference
• Energy-efficient computation
• High-performance image processing

BNNSGraph, introduced in 2024, elevated BNNS by treating entire neural networks as single graph objects rather than discrete layers. This holistic approach unlocks several critical optimizations:

Key BNNSGraph Optimizations

• Mathematical Transformations: Reorders operations like slice operations to compute only necessary subsets
• Layer Fusion: Combines operations (e.g., convolution + activation) into single optimized operations
• Copy Elision: Eliminates unnecessary data copying by using memory references
• Memory Sharing: Optimizes tensor memory allocation across the graph
• Weight Repacking: Reorganizes weights for better cache locality

These optimizations deliver improved performance, reduced memory usage, and better energy efficiency—all automatically without additional code.

Introducing BNNSGraphBuilder

BNNSGraphBuilder addresses a key limitation of the original file-based BNNSGraph API. While the existing approach works excellently for importing PyTorch models via CoreML packages, it requires intermediate compilation steps and external tooling.

Core Advantages of BNNSGraphBuilder

Native Swift Integration

• Write ML graphs directly in Swift code
• No intermediate compilation or external files required
• Type-safe operations with compile-time error checking

Runtime Flexibility

• Share values between Swift code and graph operations
• Pass runtime-known tensor shapes for optimized static sizing
• Query intermediate tensors for debugging and validation

Simplified Workflow

• Single function call (makeContext) converts Swift code to executable graph
• Inline graph definition with application logic
• Xcode autocomplete and IntelliSense support

Basic Implementation Pattern

The fundamental BNNSGraphBuilder workflow follows this structure:

let context = try BNNSGraph.makeContext { builder in
    // Define input arguments
    let x = builder.argument(name: "x", dataType: Float.self, shape: [8])
    let y = builder.argument(name: "y", dataType: Float.self, shape: [8])

    // Define operations
    let product = x * y
    let mean = product.mean(axes: [0], keepDimensions: true)

    // Return outputs as array
    return [product, mean]
}

Memory Management Pattern

// Query argument names and create tensors
var args = context.argumentNames().map { name in
    return context.tensor(argument: name, fillKnownDynamicShapes: false)!
}

// Allocate output tensors
args[0].allocate(as: Float.self, count: 8)
args[1].allocate(as: Float.self, count: 1)

// Initialize input tensors from Swift arrays
args[2].allocate(initializingFrom: [1, 2, 3, 4, 5, 6, 7, 8] as [Float])
args[3].allocate(initializingFrom: [8, 7, 6, 5, 4, 3, 2, 1] as [Float])

// Execute graph
try context.executeFunction(arguments: &args)

Advanced Features

Strong Typing and Compile-Time Safety

BNNSGraphBuilder enforces type safety at compile time, preventing runtime errors:

// Requires explicit casting - won't compile without it
let result = bases.pow(y: exponents.cast(to: Float16.self))
let masked = result * condition.cast(to: Float16.self)

Swift-Native Slicing Operations

Tensor slicing uses familiar Swift subscript syntax:

let cropped = src[
    BNNSGraph.Builder.SliceRange(startIndex: verticalMargin, endIndex: -verticalMargin),
    BNNSGraph.Builder.SliceRange(startIndex: horizontalMargin, endIndex: -horizontalMargin),
    BNNSGraph.Builder.SliceRange.fillAll
]

Comprehensive Operation Support

BNNSGraphBuilder includes extensive operation coverage:

Mathematical Operations

• Matrix multiplication, convolution
• Element-wise arithmetic (+, -, *, /)
• Activation functions (tanh, sigmoid, etc.)

Data Manipulation

• Reduction operations (mean, sum)
• Reshape, transpose, padding
• Gather and scatter operations

Comparison and Logic

• Element-wise comparisons (<, >, ==)
• Logical operations (&&, ||, !)

Practical Use Cases

Image Preprocessing

Threshold conversion from grayscale to binary:

let mean = src.mean(axes: [0, 1], keepDimensions: false)
let thresholded = src .> mean
let result = thresholded.cast(to: Float16.self)

ML Model Postprocessing

Softmax and TopK operations:

let softmax = x.softmax(axis: 1)
let topk = softmax.topK(k, axis: 1, findLargest: true)
return [topk.values, topk.indices]

Real-Time Audio Processing

Audio bitcrusher effect with saturation and quantization:

// Saturation
var destination = source * saturationGain
destination = destination.tanh()

// Quantization  
destination = destination * resolution
destination = destination.round()
destination = destination / resolution

Performance Considerations

Precision Optimization

BNNSGraphBuilder supports multiple precision levels. FP16 operations often deliver significant performance improvements over FP32:

typealias PRECISION = Float16  // Switch between Float16 and Float32

vImage Integration

Seamless integration with vImage for zero-copy image processing:

source.withBNNSTensor { src in
    destination.withBNNSTensor { dst in
        var args = [dst, src]
        try! context.executeFunction(arguments: &args)
    }
}

When to Use BNNSGraphBuilder

Ideal Scenarios

• Small custom ML models written from scratch
• Pre/post-processing pipelines for existing models
• Real-time audio and image processing
• Prototyping and experimentation

Continue Using File-Based API For

• Large, complex models trained in PyTorch/TensorFlow
• Models requiring frequent updates from external sources
• Complex architectures with hundreds of layers

Key Takeaways

The API particularly shines in scenarios requiring real-time performance, custom preprocessing pipelines, and tight integration with existing Swift codebases. For iOS developers building ML-powered applications, BNNSGraphBuilder provides a powerful tool for achieving optimal performance while maintaining code clarity and safety.

Comments

[ArshTechPro]

Write ML graphs directly in Swift code
No intermediate compilation or external files required
Type-safe operations with compile-time error checking