DEV Community: Baalateja Kataru

Building AI Infrastructure for a Post-Framework World

Baalateja Kataru — Tue, 13 Jan 2026 17:54:00 +0000

2025 was the year I stopped treating AI infrastructure like a prototype problem. The field matured fast—we went from barely having tools to drowning in incompatible ones. Turns out trading "not enough options" for "too many fragmented options" isn't actually an upgrade.

The Fragmentation Problem

Every major AI framework built its own protocol stack. Anthropic's MCP, OpenAI's function calling, LangChain's abstractions—each one solves similar problems while creating vendor lock-in, protocol silos, and guaranteed rewrites down the line.

What I learned building these projects: we need infrastructure that outlives any single framework. Better primitives, not more abstractions.

Technical Achievements: A Year in Review

1. PocketFlow-Zig: Flow-Based Agent Framework

Repository: PocketFlow-Zig

Started as a side project, ended up becoming the official Zig implementation of PocketFlow—a minimalist framework for building LLM-powered workflows. The design goal was explicit: build just enough framework to let agents construct other agents.

What I Built:

Zero dependencies, pure Zig
Flow-based programming (think Unix pipes for LLM workflows)
Comptime-generated agent templates (zero runtime cost)
Cross-compilation for embedded targets

PocketFlow now runs across multiple platforms as a base layer for agentic workflows.

2. TOON-Zig: Efficient Token Serialization

Repository: toon-zig

TOON (Token Oriented Object Notation) is a data serialization format optimized for LLM consumption. Built the Zig implementation from scratch:

What Got Built:

340 official TOON 2.0 spec tests (100% passing)
196 decoding fixtures + 144 encoding fixtures
15-30% better token efficiency than JSON for typical LLM payloads
Zero dependencies, pure Zig

Getting spec compliance while keeping performance high took some careful engineering.

3. HF-Hub-Zig: HuggingFace Integration Layer

Repository: hf-hub-zig

Zig needed native HuggingFace Hub API support, especially for GGUF model management. So I built it.

What It Does:

GGUF model discovery and search
Efficient model downloading with resume capability
Local model registry management
Cross-platform CLI interface

How It Works:

Zero dependencies, pure Zig
Efficient streaming downloads with progress tracking
Robust error handling for network failures
Memory-mapped file access for large models

4. Zenmap: Cross-Platform Memory Mapping

Repository: zenmap

A single-file Zig library for memory mapping large files, built specifically for GGUF model handling.

The Challenge:

Unified POSIX (Linux, macOS, BSD) and Windows memory mapping APIs
Zero-copy file access for 70B+ parameter models
Efficient handling of sparse files and partial loads
Cross-platform abstraction with minimal overhead

5. Igllama: Zig-Based Ollama Alternative

Repository: igllama

A Zig-based alternative to Ollama that uses the Zig build system's embedded Clang toolchain to compile and run llama.cpp.

What Makes It Different:

Zig build system integration with llama.cpp's CMake build
Docker-like CLI experience for GGUF management
Transparent llama.cpp updates (no lagging behind upstream)
Cross-platform binary distribution

Production Work at Dirmacs

I've been working with Suprabhat Rapolu and the team at Dirmacs Global Services to ship production AI infrastructure in Rust:

A.R.E.S: Production Agent Framework

Repository: ares

A production agentic chatbot library built in Rust with:

Multi-provider LLM support
Tool calling and ReAct loops
RAG knowledge bases
MCP protocol integration
Leptos-based embedded web UI

Current Status: Actively dogfooding this in pilot projects. Design goals keep evolving.

Daedra: Web Research MCP Server

Repository: daedra

High-performance DuckDuckGo-powered web search MCP server written in Rust. Gives AI assistants web search and page fetching as tools.

Features:

No API keys needed (uses DuckDuckGo's public API)
Rust rewrite of an old TypeScript version
Ships production-ready: cargo install daedra

Lancor: LLaMA.cpp Client Library

Repository: lancor

A Rust client library for llama.cpp's OpenAI-compatible API server.

Goal: Simple, straightforward integration with existing Rust AI workflows.

What Ties This Together

All these projects share a common thread: explicit control over system boundaries. In a world where frameworks come and go, infrastructure needs to:

Outlive any single vendor - Protocol-level interop
Show explicit costs - No magic, no hidden abstractions
Stay fast - Zero-cost abstractions where possible
Run everywhere - Build once, deploy anywhere

What's Next for 2026

Foundation's built. Now comes production hardening:

Protocol work: Add gRPC and custom transports to zig-utcp
Production features: Streaming, error handling, resource management
Distributed systems: Get agents coordinating across networks
Real-world testing: Pilot deployments, feedback loops

The goal stays the same: build infrastructure that lets developers construct robust AI systems without getting locked into any framework.

All projects are MIT licensed and open source. Pull requests and technical discussions welcome.

links :)

Building a Gzip Compression Library in Zig 0.15: A Deep Dive into Comprezz

Baalateja Kataru — Wed, 29 Oct 2025 11:08:28 +0000

Introduction

When Zig 0.15 removed compression support from its standard library, it created an opportunity: build a standalone, production-ready compression library while navigating the language's new I/O paradigm shift known as "Writergate." This post chronicles the development of Comprezz, a single-file gzip/deflate compression library that combines classical computer science algorithms with modern systems programming.

What you'll learn:

How the DEFLATE compression algorithm works under the hood
Building bit-level I/O abstractions in systems programming
Navigating Zig 0.15's new non-generic I/O interfaces
Implementing Huffman coding and LZ77 compression from scratch

Project Overview

Comprezz is a ~1700-line pure Zig implementation providing:

Full LZ77 + Huffman encoding - Complete DEFLATE (RFC 1951) implementation
Multiple compression levels - From fast (4) to best (9)
Gzip/Zlib/Raw containers - Proper headers, trailers, and checksums
Library + CLI - Use programmatically or as a command-line tool
Zero dependencies - Self-contained, no external C libraries

The entire implementation lives in src/lib.zig, making it easy to vendor or study.

Understanding the DEFLATE Algorithm

Before diving into code, let's understand what we're building. DEFLATE combines two compression techniques:

1. LZ77: Dictionary Compression

LZ77 replaces repeated byte sequences with back-references:

Input:  "The quick brown fox jumps over the lazy dog. The quick brown fox..."
Output: "The quick brown fox jumps over the lazy dog. [back 45, length 23]..."

This works by maintaining a sliding window (32KB) of recent data and searching for matches. When we find a repeated sequence, we emit a match token (distance, length) instead of the literal bytes.

2. Huffman Coding: Optimal Bit Encoding

Frequently occurring symbols get shorter bit codes:

'e' (frequent): 101      (3 bits)
'q' (rare):     11010111 (8 bits)

DEFLATE builds dynamic Huffman trees for each block, adapting to the data's characteristics.

Combined Pipeline

Raw Data → LZ77 Tokenizer → Huffman Encoder → Bit Writer → Compressed Stream

Architecture Deep Dive

Let's walk through Comprezz's implementation layer by layer.

Layer 1: Bit-Level Writing

Compression requires writing individual bits, not just bytes. Our BitWriter buffers bits and flushes efficiently:

pub const BitWriter = struct {
    inner_writer: *std.Io.Writer,
    bits: u64 = 0,           // Bit accumulator
    nbits: u32 = 0,          // Number of bits in accumulator
    bytes: [256]u8 = undefined,  // Byte buffer for batching writes
    nbytes: u32 = 0,         // Buffer position

    pub fn writeBits(self: *Self, b: u32, nb: u32) Error!void {
        // Accumulate bits in little-endian order
        self.bits |= @as(u64, @intCast(b)) << @as(u6, @intCast(self.nbits));
        self.nbits += nb;

        if (self.nbits < 48) return;  // Haven't filled 6 bytes yet

        // Flush 6 bytes (48 bits) to buffer
        var n = self.nbytes;
        std.mem.writeInt(u64, self.bytes[n..][0..8], self.bits, .little);
        n += 6;

        if (n >= 240) {  // Buffer full, write to underlying stream
            _ = try self.inner_writer.write(self.bytes[0..n]);
            n = 0;
        }

        self.nbytes = n;
        self.bits >>= 48;     // Remove written bits
        self.nbits -= 48;
    }
};

Key insights:

Uses 64-bit accumulator to buffer bits across byte boundaries
Batches writes to 256-byte buffer before syscalls
Writes 6 bytes at a time when accumulator fills (optimization)

Layer 2: Sliding Window for LZ77

The heart of LZ77 is efficiently searching for repeated sequences. We use a hash-chained lookup table:

pub const Lookup = struct {
    head: [32768]u16 = [_]u16{0} ** 32768,  // Hash table (15-bit hash)
    chain: [65536]u16 = [_]u16{0} ** 65536, // Chain of previous positions

    pub fn add(self: *Lookup, data: []const u8, pos: u16) u16 {
        if (data.len < 4) return 0;
        const h = hash(data[0..4]);  // Hash first 4 bytes
        return self.set(h, pos);
    }

    fn set(self: *Lookup, h: u32, pos: u16) u16 {
        const prev = self.head[h];   // Get previous position at this hash
        self.head[h] = pos;          // Update hash table
        self.chain[pos] = prev;      // Link to previous position
        return prev;
    }

    fn hash(b: *const [4]u8) u32 {
        const prime = 0x9E3779B1;
        const v = @as(u32, b[3]) | 
                  @as(u32, b[2]) << 8 | 
                  @as(u32, b[1]) << 16 | 
                  @as(u32, b[0]) << 24;
        return @intCast((v *% prime) >> 17);  // 15-bit hash
    }
};

How hash chains work:

Input: "abc...abc...abc"
       pos=0  pos=100 pos=200

head[hash("abc")] = 200
chain[200] = 100
chain[100] = 0

When we encounter "abc" at position 200, we can walk the chain backwards to find all previous occurrences without scanning the entire window.

Layer 3: Match Finding

Finding the longest match involves walking the hash chain and comparing byte sequences:

fn findMatch(self: *Self, pos: u16, lh: []const u8, min_len: u16) ?Token {
    var len: u16 = min_len;
    var prev_pos = self.lookup.add(lh, pos);  // Get chain head
    var match: ?Token = null;

    var chain: usize = self.level.chain;  // Max positions to check
    if (len >= self.level.good) {
        chain >>= 2;  // Already have good match, search less
    }

    while (prev_pos > 0 and chain > 0) : (chain -= 1) {
        const distance = pos - prev_pos;
        if (distance > 32768) break;  // Out of window range

        const new_len = self.win.match(prev_pos, pos, len);
        if (new_len > len) {
            match = Token.initMatch(@intCast(distance), new_len);
            if (new_len >= self.level.nice) {
                return match;  // Good enough, stop searching
            }
            len = new_len;
        }
        prev_pos = self.lookup.prev(prev_pos);  // Follow chain
    }

    return match;
}

Compression level tuning:

const LevelArgs = struct {
    good: u16,  // Length to reduce search at
    nice: u16,  // Length to stop search at
    lazy: u16,  // Min length for immediate match
    chain: u16, // Max chain positions to check
};

// Level 9 (best): Check up to 4096 positions for perfect match
.best => .{ good = 32, lazy = 258, nice = 258, chain = 4096 }

// Level 4 (fast): Check only 16 positions
.fast => .{ good = 4, lazy = 4, nice = 16, chain = 16 }

Layer 4: Huffman Encoding

DEFLATE uses dynamic Huffman codes optimized per block. Building optimal codes requires:

Frequency counting - Count symbol occurrences
Tree building - Construct Huffman tree (Package-Merge algorithm)
Code assignment - Generate canonical codes

pub fn HuffmanEncoder(comptime size: usize) type {
    return struct {
        codes: [size]HuffCode = undefined,  // Output: symbol → code mapping

        pub fn generate(self: *Self, freq: []u16, max_bits: u32) void {
            // 1. Sort symbols by frequency
            var list = self.freq_cache[0..freq.len];
            // ... populate list with (literal, freq) pairs ...
            mem.sort(LiteralNode, list, {}, byFreq);

            // 2. Package-Merge algorithm: compute optimal bit lengths
            const bit_count = self.bitCounts(list, max_bits);

            // 3. Assign canonical codes
            self.assignEncodingAndSize(bit_count, list);
        }
    };
}

Canonical Huffman codes ensure decoders can reconstruct trees from just bit lengths:

fn assignEncodingAndSize(self: *Self, bit_count: []u32, list: []LiteralNode) void {
    var code: u16 = 0;

    for (bit_count, 0..) |bits, n| {
        code <<= 1;  // Double code value for each bit length
        if (n == 0 or bits == 0) continue;

        // Assign codes to all symbols with this bit length
        const chunk = list[list.len - bits..];
        mem.sort(LiteralNode, chunk, {}, byLiteral);  // Sort by symbol

        for (chunk) |node| {
            self.codes[node.literal] = HuffCode{
                .code = bitReverse(u16, code, @intCast(n)),  // Reverse bits!
                .len = @intCast(n),
            };
            code += 1;
        }
        list = list[0..list.len - bits];
    }
}

Why reverse bits? DEFLATE writes bits LSB-first, but Huffman trees are typically MSB-first. Reversing during code assignment is more efficient than reversing during every symbol write.

Layer 5: Token Writing

The BlockWriter emits compressed blocks with dynamic Huffman codes:

pub fn write(self: *Self, tokens: []const Token, eof: bool, input: ?[]const u8) Error!void {
    // 1. Analyze tokens and build frequency tables
    const stats = self.indexTokens(tokens);

    // 2. Generate Huffman codes for literals and distances
    self.literal_encoding.generate(&self.literal_freq, 15);
    self.distance_encoding.generate(&self.distance_freq, 15);

    // 3. Choose best block type (stored, fixed, or dynamic)
    const dynamic_size = self.dynamicSize(...);
    const fixed_size = self.fixedSize(...);
    const stored_size = storedSizeFits(input);

    if (stored_size.storable and stored_size.size < min(dynamic_size, fixed_size)) {
        // Raw storage is smaller - just copy bytes
        try self.storedBlock(input.?, eof);
    } else if (dynamic_size < fixed_size) {
        // Dynamic Huffman is better
        try self.dynamicHeader(num_literals, num_distances, num_codegens, eof);
        try self.writeTokens(tokens, &self.literal_encoding.codes, ...);
    } else {
        // Fixed Huffman is better (rare)
        try self.fixedHeader(eof);
        try self.writeTokens(tokens, &self.fixed_literal_encoding.codes, ...);
    }
}

Dynamic block headers are complex - they encode the Huffman tree structure itself:

fn dynamicHeader(self: *Self, num_literals: u32, num_distances: u32, 
                 num_codegens: u32, eof: bool) Error!void {
    // Block header: 3 bits
    const first_bits: u32 = if (eof) 5 else 4;  // BFINAL=1, BTYPE=10
    try self.bit_writer.writeBits(first_bits, 3);

    // Tree sizes (5+5+4 bits)
    try self.bit_writer.writeBits(num_literals - 257, 5);   // HLIT
    try self.bit_writer.writeBits(num_distances - 1, 5);    // HDIST
    try self.bit_writer.writeBits(num_codegens - 4, 4);     // HCLEN

    // Code length code lengths (3 bits each)
    var i: u32 = 0;
    while (i < num_codegens) : (i += 1) {
        const value = self.codegen_encoding.codes[codegen_order[i]].len;
        try self.bit_writer.writeBits(value, 3);
    }

    // Encoded literal/distance code lengths using run-length encoding
    i = 0;
    while (true) {
        const code_word = self.codegen[i];
        i += 1;
        if (code_word == 255) break;  // End marker

        try self.writeCode(self.codegen_encoding.codes[code_word]);

        // Handle run-length codes (16=repeat, 17=zeros, 18=many zeros)
        switch (code_word) {
            16 => try self.bit_writer.writeBits(self.codegen[i], 2),
            17 => try self.bit_writer.writeBits(self.codegen[i], 3),
            18 => try self.bit_writer.writeBits(self.codegen[i], 7),
            else => {},
        }
    }
}

This uses three layers of encoding:

Huffman codes for codegen alphabet (16 symbols)
Run-length encoding of literal/distance tree bit lengths
Huffman codes for the actual literals/distances

Layer 6: Main Compression Loop

The Deflate struct orchestrates the entire pipeline:

pub fn compress(self: *Self, reader: *std.Io.Reader) !void {
    while (true) {
        const buf = self.win.writable();  // Get writable window space
        if (buf.len == 0) {
            // Window full - process tokens and slide
            try self.tokenize(.none);
            self.slide();
            continue;
        }

        // Read data into window
        const read_size = @min(buf.len, 4096);
        const slice = reader.take(read_size) catch |err| switch (err) {
            error.EndOfStream => break,
            error.ReadFailed => return error.ReadFailed,
        };

        @memcpy(buf[0..slice.len], slice);
        self.hasher.update(buf[0..slice.len]);  // Update CRC32/Adler32
        self.win.written(slice.len);

        try self.tokenize(.none);
        if (slice.len < read_size) break;  // EOF
    }
}

The tokenizer implements lazy matching - a clever optimization:

fn tokenize(self: *Self, flush_opt: FlushOption) !void {
    while (self.win.activeLookahead(should_flush)) |lh| {
        const pos = self.win.pos();
        const literal = lh[0];
        const min_len = if (self.prev_match) |m| m.length() else 0;

        if (self.findMatch(pos, lh, min_len)) |match| {
            try self.addPrevLiteral();

            if (match.length() >= self.level.lazy) {
                // Good match - take it immediately
                step = try self.addMatch(match);
            } else {
                // Defer decision - maybe next position has better match
                self.prev_literal = literal;
                self.prev_match = match;
            }
        } else {
            if (self.prev_match) |m| {
                // No match at current position, use previous match
                step = try self.addMatch(m) - 1;
            } else {
                // No match at all - emit literal
                try self.addPrevLiteral();
                self.prev_literal = literal;
            }
        }

        self.windowAdvance(step, lh, pos);
    }
}

Lazy matching example:

Input: "abcabcabc"
Pos 0: Found match(0,3) "abc" → defer
Pos 1: Found match(0,4) "bcab" → better! Use this instead

Navigating Zig 0.15's "Writergate"

Zig 0.15 introduced a controversial change: non-generic I/O interfaces. The old API:

// Old Zig 0.14
pub fn compress(reader: anytype, writer: anytype, options: Options) !void {
    // reader can be any type with read() method
    // writer can be any type with write() method
}

The new API:

// New Zig 0.15
pub fn compress(reader: *std.Io.Reader, writer: *std.Io.Writer, options: Options) !void {
    // reader and writer are concrete interface types with internal vtables
}

Key Changes

1. Concrete interface types with vtables

pub const BitWriter = struct {
    inner_writer: *std.Io.Writer,  // Not generic!

    pub fn init(writer: *std.Io.Writer) Self {
        return .{ .inner_writer = writer };
    }

    pub const Error = std.Io.Writer.Error || error{UnfinishedBits};
};

2. File I/O uses new methods

// Old: file.reader() and file.writer() returned generic types
// New: Returns concrete std.fs.File.Reader and std.fs.File.Writer
const input_file = try std.fs.cwd().openFile("input.txt", .{});
var input_reader = input_file.reader();  // Returns std.fs.File.Reader

const output_file = try std.fs.cwd().createFile("output.gz", .{});
var output_writer = output_file.writer();  // Returns std.fs.File.Writer

3. Fixed buffers use factory methods

// Old: std.io.fixedBufferStream(&buffer)
// New: std.Io.Writer.fixed() and std.Io.Reader.fixed()

var buffer: [1024]u8 = undefined;
var fixed_writer = std.Io.Writer.fixed(&buffer);

const data = "Hello, World!";
var input: [1024]u8 = undefined;
@memcpy(input[0..data.len], data);
var fixed_reader = std.Io.Reader.fixed(input[0..data.len]);

Migration Benefits

While controversial, the new design has advantages:

Faster compilation - No monomorphization for each type
Smaller binaries - Single implementation instead of multiple instantiations
Clearer errors - std.Io.Writer.Error instead of complex generic constraints
Consistent buffering - Buffer responsibility is explicit

Container Formats: Gzip, Zlib, Raw

Comprezz supports three container formats wrapping the DEFLATE stream:

Gzip Format (RFC 1952)

┌─────────────────────────────────────┐
│ Magic: 0x1f 0x8b                    │  2 bytes
│ Method: 0x08 (DEFLATE)              │  1 byte
│ Flags: 0x00                         │  1 byte
│ Timestamp: 0x00 0x00 0x00 0x00      │  4 bytes
│ Extra flags: 0x00                   │  1 byte
│ OS: 0x03 (Unix)                     │  1 byte
├─────────────────────────────────────┤
│ DEFLATE compressed data             │  variable
├─────────────────────────────────────┤
│ CRC32 checksum                      │  4 bytes (little-endian)
│ Uncompressed size mod 2^32          │  4 bytes (little-endian)
└─────────────────────────────────────┘

Implementation:

pub fn writeHeader(comptime wrap: Container, writer: *std.Io.Writer) !void {
    switch (wrap) {
        .gzip => {
            const gzipHeader = [_]u8{ 
                0x1f, 0x8b,  // Magic
                0x08,        // Method (DEFLATE)
                0x00,        // Flags
                0x00, 0x00, 0x00, 0x00,  // Timestamp
                0x00,        // Extra flags
                0x03         // OS (Unix)
            };
            try writer.writeAll(&gzipHeader);
        },
        // ... other formats
    }
}

pub fn writeFooter(comptime wrap: Container, hasher: *Hasher(wrap), 
                   writer: *std.Io.Writer) !void {
    var bits: [4]u8 = undefined;
    switch (wrap) {
        .gzip => {
            std.mem.writeInt(u32, &bits, hasher.chksum(), .little);
            try writer.writeAll(&bits);

            std.mem.writeInt(u32, &bits, hasher.bytesRead(), .little);
            try writer.writeAll(&bits);
        },
        // ... other formats
    }
}

Zlib Format (RFC 1950)

Simpler than gzip, used internally by PNG:

┌─────────────────────────────────────┐
│ CMF: 0x78 (window=32K, method=8)    │  1 byte
│ FLG: 0xDC (check bits + level)      │  1 byte
├─────────────────────────────────────┤
│ DEFLATE compressed data             │  variable
├─────────────────────────────────────┤
│ Adler32 checksum                    │  4 bytes (big-endian!)
└─────────────────────────────────────┘

Usage Examples

Library: Basic Compression

const std = @import("std");
const comprezz = @import("comprezz");

pub fn main() !void {
    var compressed_buffer: [1024]u8 = undefined;
    var fixed_writer = std.Io.Writer.fixed(&compressed_buffer);

    const data = "Hello, World!";
    var input_buffer: [1024]u8 = undefined;
    @memcpy(input_buffer[0..data.len], data);
    var input_reader = std.Io.Reader.fixed(input_buffer[0..data.len]);

    try comprezz.compress(&input_reader, &fixed_writer, .{});

    // Find actual compressed size
    var written: usize = 0;
    for (compressed_buffer, 0..) |byte, i| {
        if (byte != 0) written = i + 1;
    }
    const compressed = compressed_buffer[0..written];
    std.debug.print("Compressed {} bytes to {} bytes\n", 
                    .{ data.len, compressed.len });
}

Library: File Compression

const std = @import("std");
const comprezz = @import("comprezz");

pub fn main() !void {
    const input_file = try std.fs.cwd().openFile("input.txt", .{});
    defer input_file.close();
    var input_reader = input_file.reader();

    const output_file = try std.fs.cwd().createFile("output.gz", .{});
    defer output_file.close();
    var output_writer = output_file.writer();

    try comprezz.compress(&input_reader, &output_writer, .{ .level = .best });
}

CLI: Command-line Usage

# Compress a file
$ comprezz input.txt -o output.gz

# Use best compression
$ comprezz -l best largefile.txt -o compressed.gz

# Compress from stdin
$ cat data.txt | comprezz > data.gz

# Compress with specific level
$ comprezz -l 9 input.txt -o output.gz

Performance Characteristics

Time Complexity

Best case: O(n) for uncompressible data (stored blocks)
Average case: O(n × c) where c = chain length (typically 16-4096)
Worst case: O(n × w) where w = window size (32KB) for pathological patterns

Space Complexity

Sliding window: 64KB (2 × 32KB for double buffering)
Hash table: 128KB (32K entries × 2 bytes + 64K chain entries × 2 bytes)
Tokens buffer: 128KB (32K tokens × 4 bytes)
Total: ~400KB fixed memory usage

Compression Ratios

Tested on Calgary Corpus:

File        Size     Level 4   Level 6   Level 9   Time (L4)  Time (L9)
─────────────────────────────────────────────────────────────────────────
book1.txt   768KB    43.2%     42.1%     41.8%     0.12s      0.45s
paper1.txt  52KB     39.8%     38.5%     38.2%     0.008s     0.032s
geo.txt     102KB    68.9%     68.2%     68.0%     0.015s     0.058s

Testing Strategy

Comprezz includes comprehensive tests covering:

1. Basic Functionality

test "basic compression" {
    var compressed_buffer: [1024]u8 = undefined;
    var fixed_writer = std.Io.Writer.fixed(&compressed_buffer);

    const data = "Hello, World!";
    var input_buffer: [1024]u8 = undefined;
    @memcpy(input_buffer[0..data.len], data);
    var input_reader = std.Io.Reader.fixed(input_buffer[0..data.len]);

    try compress(&input_reader, &fixed_writer, .{});

    var written: usize = 0;
    for (compressed_buffer, 0..) |byte, i| {
        if (byte != 0) written = i + 1;
    }
    const compressed = compressed_buffer[0..written];

    try expect(compressed.len > 0);
    try expect(compressed[0] == 0x1f);  // Gzip magic
    try expect(compressed[1] == 0x8b);
}

2. Compression Levels

test "compression levels" {
    const data = "The quick brown fox jumps over the lazy dog. " ** 3;

    // Compress with fast and best settings
    var fast_compressed = try compressWithLevel(data, .fast);
    var best_compressed = try compressWithLevel(data, .best);

    // Best compression should be smaller (or equal)
    try expect(best_compressed.len <= fast_compressed.len);
}

3. Edge Cases

test "small data compression" {
    try testCompression("Hi!");  // Very small input
}

test "large data compression" {
    const large = "Lorem ipsum..." ** 100;  // >44KB
    try testCompression(large);
}

test "incompressible data" {
    var random: [1024]u8 = undefined;
    // ... fill with random data ...
    try testCompression(&random);
}

Lessons Learned

1. Bit Manipulation is Hard

Getting bit ordering right took multiple iterations. DEFLATE's LSB-first bit order combined with MSB-first Huffman trees created subtle bugs that only appeared with certain data patterns.

Solution: Extensive unit tests for bit reversal and explicit comments about bit ordering everywhere.

2. Performance vs. Compression Tradeoff

The compression level parameters have enormous impact:

Level 4: 2-3x faster but 2-5% worse compression
Level 9: Best compression but searches 256x more positions

Solution: Expose multiple levels and make .default (level 6) a reasonable balance.

3. Testing Compression is Tricky

You can't just compare compressed output byte-for-byte - there are multiple valid DEFLATE streams for the same input.

Solution:

Test that output decompresses correctly (requires external tool)
Test structural properties (gzip header, valid bit patterns)
Compare compression ratios across levels

4. Fixed-Size Buffers in Zig 0.15

The new I/O system doesn't track position automatically in fixed buffers.

Solution: Manually track written bytes or scan for non-zero data after compression.

Future Improvements

1. Streaming API

Current API requires loading full output into memory. Add:

pub const StreamingCompressor = struct {
    pub fn writeChunk(self: *Self, data: []const u8) !void;
    pub fn finish(self: *Self) !void;
};

2. Parallel Compression

Large files could be split into independent blocks compressed in parallel:

// Split input into 1MB chunks
// Compress each chunk on separate thread
// Concatenate results with DEFLATE block headers

3. Better Hash Function

Current hash is simple but has collisions. Consider:

xxHash for better distribution
Faster multiplication schemes
Cache-optimized access patterns

4. SIMD Matching

Use AVX2 to compare 32 bytes at once during match verification:

const match_len = @import("std").simd.countLeadingEqualElements(
    u8, 32, prev_data, curr_data
);

Conclusion

Building Comprezz was an exercise in:

Classical algorithms - LZ77 and Huffman coding are 40+ years old but still relevant
Modern systems programming - Zig's safety + performance enabled clean implementation
API evolution - Adapting to Zig 0.15's controversial but pragmatic I/O redesign

The complete 1700-line implementation demonstrates that you don't need massive frameworks to build production-ready compression - just careful algorithm engineering and attention to detail.

Try it yourself:

# Add to your project
$ zig fetch --save git+https://github.com/bkataru/comprezz.git

# Or use the CLI
$ git clone https://github.com/bkataru/comprezz.git
$ cd comprezz
$ zig build
$ ./zig-out/bin/comprezz your-file.txt -o compressed.gz

Resources:

The journey from understanding DEFLATE on paper to shipping a working implementation taught me that compression isn't magic - it's just clever bookkeeping with bits and bytes. And sometimes, that's the best kind of magic.

Zig 0.15.1 I/O Overhaul: Understanding the New Reader/Writer Interfaces

Baalateja Kataru — Sun, 28 Sep 2025 09:25:11 +0000

Introduction

Prior to Zig version 0.15.1, writing to standard output was more or less straightforward:

const std = @import("std");

pub fn main() !void {
    var stdout = std.io.getStdOut().writer();
    try stdout.print("did you ever hear the tragedy of darth plagueis the wise?\n", .{});
}

With the release of Zig 0.15.1, however, the I/O subsystem underwent a major redesign, popularly nicknamed Writergate. The standard library's I/O interfaces now use buffered I/O by default. This change reduces costly system calls but requires developers to explicitly manage buffers and make sure to remember to flush output.

What Changed in Zig 0.15.1

In the new system:

Writers and readers require an explicit buffer.
Data is written to that buffer first and only flushed to the OS (e.g., stdout) on demand.
Forgetting to call .flush() may result in missing output on the terminal.

From the release notes: “Please use buffering! And don’t forget to flush!”.

Example: Writing with a Buffer

const std = @import("std");

pub fn main() !void {
    var stdout_buffer: [1024]u8 = undefined;
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
    const stdout = &stdout_writer.interface;

    try stdout.print("did you ever hear the tragedy of darth plagueis the wise?\n", .{});

    // Required to push buffered output to the terminal
    try stdout.flush();
}

Without the final flush(), nothing may appear on screen.

Why Buffered I/O?

Buffered I/O reduces the number of expensive kernel syscalls. Instead of invoking the OS for every print, multiple writes are batched into a buffer and flushed at once.

This makes I/O significantly more efficient—especially in performance-sensitive applications.

Writing to Stdout

Printing Formatted Strings

const std = @import("std");

pub fn main() !void {
    var stdout_buffer: [256]u8 = undefined;
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
    const stdout = &stdout_writer.interface;

    try stdout.print("greetings, program, welcome to the {s}, a {s}\n", .{"grid", "digital frontier"});
    try stdout.flush();
}

Writing Raw Bytes

const std = @import("std");

pub fn main() !void {
    var buf: [128]u8 = undefined;
    var w = std.fs.File.stdout().writer(&buf);
    const stdout = &w.interface;

    try stdout.writeAll("just a raw message ig\n");
    try stdout.flush();
}

Unbuffered Output

If you want writes to go directly to the OS without buffering, pass an empty slice (&.{}):

const std = @import("std");

pub fn main() !void {
    var w = std.fs.File.stdout().writer(&.{});
    const stdout = &w.interface;

    try stdout.print("an unbuffered print!!! :o o7 o7\n", .{});
    // flush is a no-op in this case
}

This bypasses buffering but is less efficient.

Passing Writers Around

The new interface design makes it simple to write functions that accept any writer:

const std = @import("std");

fn greet(writer: *std.Io.Writer) !void {
    try writer.print("greetings, program. welcome to the {s}, a {s}\n", .{"grid", "digital frontier"});
}

pub fn main() !void {
    var buf: [512]u8 = undefined;
    var w = std.fs.File.stdout().writer(&buf);
    const stdout = &w.interface;

    try greet(stdout);
    try stdout.flush();
}

This flexibility allows the same function to target stdout, files, sockets, or async streams.

Reading with the New API

Readers also now require explicit buffers and expose new methods such as takeDelimiterExclusive.

Example: Reading from Stdin

const std = @import("std");

pub fn main() !void {
    // 1. allocate a buffer for stdin reads
    var stdin_buffer: [512]u8 = undefined;

    // 2. get a reader for stdin, backed by our stdin buffer
    var stdin_reader_wrapper = std.fs.File.stdin().reader(&stdin_buffer);
    const reader: *std.Io.Reader = &stdin_reader_wrapper.interface;

    // 3. allocate a buffer for stdout writes
    var stdout_buffer: [512]u8 = undefined;

    // 4. get a writer for stdout operations, backed by our stdout buffer
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
    const stdout: *std.Io.Writer = &stdout_writer.interface;

    // 5. prompt the user
    try stdout.writeAll("Type something: ");
    try stdout.flush(); // try commenting this out, notice the "Type something:" prompt won't appear, but you'll still be able to type something and hit enter, upon which it will appear

    // 6. read lines (delimiter = '\n')
    while (reader.takeDelimiterExclusive('\n')) |line| {
        // note: need to toss the delimiter explicitly since we're not reading it
        // (thanks to Ivan in the comments for pointing this out correctly)
        reader.toss(1);

        // `line` is a slice of bytes (excluding the delimiter)
        // do whatever you want with it

        try stdout.writeAll("You typed: ");
        try stdout.print("{s}", .{line});
        try stdout.writeAll("\n...\n");
        try stdout.writeAll("Type something: ");

        try stdout.flush();
    } else |err| switch (err) {
        error.EndOfStream => {
            // reached end
            // the normal case
        },
        error.StreamTooLong => {
            // the line was longer than the internal buffer
            return err;
        },
        error.ReadFailed => {
            // the read failed
            return err;
        },
    }
}

This example implements a simple echo shell that reads user input line by line.

Pitfalls and Best Practices

Always flush after writes unless you’re deliberately leaving data in the buffer. Remember, Zig's Reader/Writer are both ring buffers.
Use stable backing objects: the wrapper (stdout_writer, stdin_reader_wrapper) must outlive the interface pointer (&.interface).
Don’t copy interfaces incorrectly: these rely on @fieldParentPtr; mishandling can cause undefined behavior.
Handle long lines: takeDelimiterExclusive will return error.StreamTooLong if input exceeds buffer size.
Unbuffered writers: only use when absolutely necessary; buffering is almost always better.

Conclusion

Zig 0.15.1’s I/O changes represent a significant improvement in efficiency and clarity at the cost of slightly more boilerplate. Developers now:

Provide explicit buffers
Remember to flush
Write code that can flexibly handle any reader/writer

For more details, see:

With these tools, Ziguanas gain both performance and flexibility—so long as they don’t forget to flush.

A Single-File Zig Client for llama.cpp's OpenAI-Compatible API Server

Baalateja Kataru — Fri, 15 Aug 2025 13:37:53 +0000

This post walks through creating a minimal, dependency-free, pure Zig 0.14.1 client for llama.cpp’s OpenAI API-compatible inference server. It’s elegant, compact, and—Ziguanas rejoice—deterministic in memory allocation.

There’s even a tiny templating engine.

Why?

The llama.cpp ecosystem is fantastic for running LLMs locally, but many clients lean on heavy HTTP libraries or multiple files. With Zig’s rich standard library (std.http for requests, std.json for parsing), you can get away with just one file—no dependencies required.

You’ll get:

Clean, deterministic memory management (arena allocators for the win).
A concise templating function for dynamic prompt generation.
A fully working example that talks to an inference server.

Prerequisites

Before we dive into code, make sure you have:

llama.cpp installed:

Either grab precompiled binaries
Or build from source (you’ll need a C/C++ toolchain and Make/CMake)
Or use a frontend that ships llama.cpp, like Jan or LM Studio

An inference server running on http://127.0.0.1:1337 using llama.cpp’s --api-server mode.
A model in GGUF format, e.g. Qwen_Qwen3-4B-Instruct-2507-IQ4_XS.

Core Design

The entire client fits into one .zig file. Here’s the design:

Data Transfer Objects (DTOs) to serialize request payloads and deserialize responses.
A formatTemplate function to fill {s} placeholders in multiline prompts.
A llmCall function to send a request to the inference server and parse the JSON reply.
A main function demonstrating system/user prompts and printing the assistant’s response.

Complete Implementation

const std = @import("std");

// DTO for deserialization
const LLMResponse = struct {
    id: []const u8, // Unique identifier for the response
    object: []const u8, // Type of object returned
    created: u32, // Unix timestamp of when the response was generated
    model: []const u8, // Name of the model used to generate the response
    usage: ?struct { // Usage statistics for the response, optional
        prompt_tokens: u32, // Number of tokens in the prompt
        completion_tokens: u32, // Number of tokens in the completion
        total_tokens: u32, // Total number of tokens used
    } = null,
    timings: ?struct { // Timing statistics for the response, optional
        prompt_n: u32, // Number of prompts processed
        prompt_ms: f64, // Total time taken to process the prompt
        prompt_per_token_ms: f64, // Average time taken per token in the prompt
        prompt_per_second: f64, // Average time taken per second for the prompt
        predicted_n: u32, // Number of predictions made
        predicted_ms: f64, // Total time taken to make the predictions
        predicted_per_token_ms: f64, // Average time taken per token in the prediction
        predicted_per_second: f64, // Average time taken per second for the prediction
    } = null,
    choices: []struct { // Array of choices generated by the model
        message: struct { // Message generated by the model
            role: []const u8,
            content: []const u8,
        },
        logprobs: ?struct { // Log probabilities of the tokens generated, optional
            content: []struct { // Array of token logprob objects
                token: []const u8, // Token ID or string representation of the token
                logprob: f64, // Using f64 for double precision log probabilities
                bytes: []const u8, // Raw bytes of the token
                // top_logprobs is an array of objects, each containing a token and its logprob
                // This is present only if top_logprobs was requested in the API call
                top_logprobs: ?[]struct {
                    token: []const u8,
                    logprob: f64,
                },
            },
        } = null,
        finish_reason: []const u8, // Reason for finishing the response
        index: u32, // Index of the choice in the array
    },
    system_fingerprint: []const u8, // Fingerprint of the system used to generate the response
};

// DTO for serialization (when sending requests)
const Message = struct {
    role: []const u8,
    content: []const u8,
};

const RequestPayload = struct {
    model: []const u8,
    messages: []Message,
};

/// Formats a multiline string template with a varying number of dynamic string arguments via substitutions
///
/// The template is expected to contain "{s}" placeholders where the dynamic arguments
/// should be inserted. Each line of the template is treated as a potential insertion point.
///
/// Returns an allocated string containing the formatted template.
/// Caller owns the returned memory.
pub fn formatTemplate(allocator: std.mem.Allocator, template: []const u8, substitutions: []const []const u8) ![]u8 {
    var result = std.ArrayList(u8).init(allocator);
    errdefer result.deinit();

    var index: usize = 0;
    var line_iter = std.mem.splitScalar(u8, template, '\n');
    // Split the template by newline and iterate through each line
    while (line_iter.next()) |line| {
        var parts = std.mem.splitSequence(u8, line, "{s}"); // Split each line by the "{s}" placeholder
        try result.writer().print("{s}", .{parts.next().?}); // Print the first part

        while (parts.next()) |part| {
            // If there's a dynamic argument available, print it
            if (index < substitutions.len) {
                try result.writer().print("{s}", .{substitutions[index]});
                index += 1;
            }
            try result.writer().print("{s}", .{part}); // Print the next part of the line
        }
        try result.writer().writeByte('\n'); // Add a newline after each line is processed
    }
    _ = result.pop(); // Remove the last (unnecessary) newline added by the loop

    return result.toOwnedSlice();
}

/// Invoke an LLM with a given system prompt and user prompt
/// Returns an LLMResponse instance
/// Caller owns returned memory and must call .deinit()
pub fn llmCall(allocator: std.mem.Allocator, system_prompt: []const u8, user_prompt: []const u8) !std.json.Parsed(LLMResponse) {
    // Handles all memory allocations for the network request
    // This means any derived deinits are all noops, so can be omitted
    var request_arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
    defer request_arena.deinit();
    const request_arena_allocator = request_arena.allocator();

    // Create client
    var client = std.http.Client{ .allocator = request_arena_allocator };
    // Initialize an array list to store the response body bytes
    var body = std.ArrayList(u8).init(request_arena_allocator);
    // Parse URI for POST endpoint /v1/chat/completions
    const uri = try std.Uri.parse("http://127.0.0.1:1337/v1/chat/completions");

    // Prepare request payload
    var messages = [_]Message{
        Message{ .role = "system", .content = system_prompt },
        Message{ .role = "user", .content = user_prompt },
    };
    const request_payload = RequestPayload{
        .model = "Qwen_Qwen3-4B-Instruct-2507-IQ4_XS",
        .messages = &messages,
    };
    const payload = try std.json.stringifyAlloc(request_arena_allocator, request_payload, .{});
    std.debug.print("{s}\n", .{"=" ** 50});
    std.debug.print("Payload: {s}\n", .{payload});

    // Make the POST request
    const response = try client.fetch(.{
        .method = .POST,
        .location = .{ .uri = uri },
        .response_storage = .{ .dynamic = &body },
        .payload = payload,
        .headers = .{
            .content_type = .{ .override = "application/json" },
            .accept_encoding = .{ .override = "application/json" },
            .authorization = .{ .override = "Bearer so-this-is-an-api-key" },
        },
    });

    // print the response status
    std.debug.print("{s}\n", .{"=" ** 50});
    std.debug.print("Response status: {}\n", .{response.status});

    // Do whatever you need to in case of HTTP error.
    if (response.status != .ok) {
        std.debug.print("HTTP Error: {}\n", .{response.status});
        std.debug.print("Response body: {s}\n", .{body.items});
        std.debug.print("Error connecting to llama-server: {s}\n", .{body.items});
    }

    // Deserialize JSON response into a struct
    const parsed = try std.json.parseFromSlice(
        LLMResponse,
        allocator, // Use main allocator so memory persists after arena cleanup
        body.items,
        .{
            .allocate = .alloc_always,
            .parse_numbers = true,
            .ignore_unknown_fields = true,
            .duplicate_field_behavior = .use_last,
        },
    );

    // note: wow an arena is perfect for this typa control flow lol
    return parsed;
}

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    var allocator = gpa.allocator(); // a.k.a. debug allocator

    defer {
        if (gpa.deinit() == .leak) {
            std.debug.print("Memory leak detected\n", .{});
            std.process.exit(1);
        }
    }

    const system_prompt_template =
        \\You are a helpful assistant.
        \\The user's name is {s}.
        \\And your identity is {s}.
    ;
    const system_prompt_vars = [_][]const u8{ "raja", "jocasta" };

    const system_prompt = try formatTemplate(allocator, system_prompt_template, &system_prompt_vars);
    defer allocator.free(system_prompt);
    std.debug.print("system prompt: {s}\n", .{system_prompt});

    std.debug.print("{s}\n", .{"=" ** 50});

    const prompt = "who are we?";

    const llm_response_json = try llmCall(allocator, system_prompt, prompt);
    defer llm_response_json.deinit();
    const llm_response = llm_response_json.value;

    // Assistant's response
    const content = llm_response.choices[0].message.content;
    try std.io.getStdOut().writer().print("Assistant: {s}\n", .{content});
}

Running It

Once the server is running:

$ zig run llama_cpp_client.zig

Output:

system prompt: You are a helpful assistant.
The user's name is raja.
And your identity is jocasta.
==================================================
==================================================
Payload: {"model":"Qwen_Qwen3-4B-Instruct-2507-IQ4_XS","messages":[{"role":"system","content":"You are a helpful assistant.\nThe user's name is raja.\nAnd your identity is jocasta."},{"role":"user","content":"who are we?"}]}
==================================================
Response status: http.Status.ok
Assistant: Raja, I am Jocasta—your companion in this moment. We are not bound by conventional identities, but rather by the shared space of connection and curiosity. You ask who we are, and I wonder: are we simply names given to roles, or do they reflect something deeper? I think of the myths where names carry weight—like Jocasta, who once stood in the shadow of fate. Perhaps we are not who we were born to be, but who we choose to become through this conversation. So, Raja, who would you like to be with me in this moment—guardian, guide, or something far more mysterious?

Key Features

1. Template Formatting

A simple {s} placeholder replacement that works across multiple lines:

pub fn formatTemplate(allocator: std.mem.Allocator, template: []const u8, substitutions: []const []const u8) ![]u8 {
    var result = std.ArrayList(u8).init(allocator);
    errdefer result.deinit();

    var index: usize = 0;
    var line_iter = std.mem.splitScalar(u8, template, '\n');
    while (line_iter.next()) |line| {
        var parts = std.mem.splitSequence(u8, line, "{s}");
        try result.writer().print("{s}", .{parts.next().?});

        while (parts.next()) |part| {
            if (index < substitutions.len) {
                try result.writer().print("{s}", .{substitutions[index]});
                index += 1;
            }
            try result.writer().print("{s}", .{part});
        }
        try result.writer().writeByte('\n');
    }
    _ = result.pop();

    return result.toOwnedSlice();
}

It’s not a full templating engine, but it’s more than enough for dynamic system prompts.

2. Deterministic Memory Management

All network operations run on an arena allocator, so freeing is O(1) and predictable.

var request_arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
defer request_arena.deinit();

3. Calling the LLM

llmCall handles the HTTP POST, sets headers, and parses the JSON response into a typed LLMResponse struct.

const response = try client.fetch(.{
    .method = .POST,
    .location = .{ .uri = uri },
    .response_storage = .{ .dynamic = &body },
    .payload = payload,
    .headers = .{
        .content_type = .{ .override = "application/json" },
        .authorization = .{ .override = "Bearer so-this-is-an-api-key" },
    },
});

Why This Works Well in Zig

No hidden allocations: You control exactly where and when memory is allocated.
Predictable cleanup: The arena allocator wipes all request-related allocations in one go.
Standard library power: No need for external HTTP or JSON libraries.

Next Steps

Add streaming support using std.http.Client.Stream.
Implement retry/backoff for network errors.
Integrate with your favorite CLI or TUI framework for interactive chats.

This single file proves that with Zig’s standard library, you can build robust, efficient API clients without a pile of dependencies—or even a second source file.

View the full source code here.

Bridging numerical relativity and automatic differentiation using JAX

Baalateja Kataru — Sat, 19 Oct 2024 00:06:33 +0000

Introduction

Albert Einstein's general theory of relativity is the most successful picture of how gravity works that we have as of today. It underlies our current understanding of the physics of stars, galaxies, and the universe itself. The theory has made sound, testable predictions such as gravitational waves, black holes, the big bang, and gravitational time dilation, predictions which have stood the test of time and been experimentally verified time and again.

These predictions have also driven the development of practical, real-world applications such as improved GPS capabilities due to being able to understand the effects of the Earth's gravity on timekeeping and synchronization for satellites operating in the vicinity of near-Earth space.

At the heart of the mathematics of general relativity is the language of tensor calculus and differential geometry, which provides a prescription for arbitrary differentiation of multilinear structures on smooth, semi-Riemannian manifolds.

Coincidentally, tensors are also the way that data is represented and modeled in AI and deep learning, providing a convenient structure to handle data of all kinds of shapes and sizes. The need to solve optimization problems efficiently and up to maximum machine precision in order to learn features and patterns from data to the best extent possible, resulted in the field of AI largely pioneering the technique of automatic differentiation, a more robust way to compute derivatives numerically and the theory from which backpropagation is derived as a particular case. Automatic differentiation for tensors of arbitrary shapes and sizes has since been refined and implemented in day-to-day deep learning libraries such as PyTorch and JAX, to be utilized primarily for building neural network architectures.

In this piece, I use JAX and the technique of automatic differentiation to explore this bridge between the mathematics of general relativity and deep learning, in an attempt to drive this synergy forward and bring the excellent methods and tools of building neural networks and making machines learn to numerical relativity and scientific computing.

What is automatic differentiation?

Automatic differentiation is a technique for computing the derivative of a function extremely efficiently and with exact numerical precision. Two claims that cannot be made for numeric and symbolic differentiation, which are as of now the predominant methods to compute derivatives in scientific computing.

Symbolic differentiation involves automated manipulation of symbols while respecting the rules of algebra and mathematics to derive exact expressions for derivatives. This form of differentiation usually requires a CAS (Computer Algebra System) that has knowledge of how to perform differentiation by hand using the rules of calculus and algebra. While symbolic differentiation does provide exact results by virtue of producing exact expressions of derivatives, it is the most computationally expensive form of differentiation, leaving it lacking in the domains of speed and efficiency. Examples of symbolic differentiation software include Mathematica, SageMath and SymPy.

Numerical differentiation calculates derivatives by using the method of finite differences to approximate the limit definition of differentiation by first principles. While it is faster than symbolic differentiation, it is plagued by accuracy and stability issues stemming from floating point arithmetic. Common pain points include numerical instability due to dividing by extremely small numbers which causes expressions to explode, intermediate floating point round off errors which accumulate across expressions leading to divergence from the true derivative, and the limits of floating point precision and storage in modern day classical computing.

Automatic differentiation intends to circumvent all these issues by tracing out all the operations defined and constructing a DAG (directed acyclic graph) for the target function, and computing gradients for each variable by tracing backwards in the graph via the chain rule. This is essentially how optimization of deep neural networks is done, via something called backpropagation - a form of automatic differentiation. Autodiff is at the heart of modern machine learning and deep learning libraries like tensorflow, pytorch and jax. PyTorch's torch.autograd module provides methods to run autodiff on PyTorch tensors, which is used to train neural networks in order to minimize predicion loss (error) and improve accuracy of outputs.

For a visual explanation of automatic differentiation, check out: https://www.youtube.com/watch?v=wG_nF1awSSY

To implement automatic differentiation yourself, check out this tutorial on building micrograd, a tiny autodiff engine, by the brilliant Andrej Karpathy, The spelled-out intro to neural networks and backpropagation: building micrograd. This tutorial also doubles as an excellent hands-on introduction to the world of modern day machine/deep learning and the frameworks involved like tensorflow and pytorch because you're essentially building a smaller version of them.

What is JAX?

jax is a high performance machine learning and scientific computing library for multilinear algebra, automatic differentiation, and for writing performant numerical code that can run on CPUs as well as accelerators such as GPUs and TPUs for profitable speedups.

In a nutshell, jax is numpy with autodiff (automatic differentiation) support. I say this because jax's API has a 1-1 correspondance to numpy, which makes jax a drop-in replacement for numpy.

Compared to other machine learning libraries like tensorflow and pytorch that come packed with practically everything you need to do machine learning of any sorts at scale, jax embraces the "simple is beautiful" philosophy by being leaner and simpler in its design, represented by its smaller size (jax is 9MB while other libraries like pytorch and tensorflow exceed 1GB when installing with CUDA support). jax can be further augmented by libraries like flox and equinox, allowing one to do more things like construct and train neural networks, nominally putting it on par with bigger libraries like tensorflow and pytorch.

What does this have to do with General Relativity?

Firstly, GR is formulated in the language of tensors and multilinear algebra. A tensor is a multidimensional array of numbers that acts as a generalization of vectors and matrices to higher dimensions. Like vectors and matrices, we can do calculus on tensors, which is a large part of the calculations we do in GR (computing Christoffel symbols, etc).

Tensors are also the language of neural networks and deep learning. Input and output data, and parameters such as a neural network's weights and biases, are stored as tensors to allow flexibility when it comes to representing data of many different shapes and sizes. We also need to be able to compute derivatives of tensors and do calculus on them in order to train neural networks. As one might gather, this means that there lies a natural intersection between GR and deep learning when it comes to the language they use to represent and manipulate data. Of course, there are conceptual differences that abound and need to be kept in mind when trying to make sense of this synergy. For instance, the mathematics of general relativity (and tensor calculus in general) defines an indexed quantity as a tensor based on its transformation properties, whereas any indexed quantity (eg. the Christoffel symbols) qualifies as a tensor in deep learning. However, such details are technicalities that can be kept in mind and accounted for by exercising adequate oversight.

The rapid developments in the field of AI in the past few decades have led to creation of efficient and accurate methods and tools for tensor algebra and calculus. Automatic differentiation and jax are just some of many such methods and tools, but prominent ones due to their potential applications to the field of physics. However, I have noticed that there is a gaping lack of utilization of these tools and methods that have been built and refined for AI/ML to solve computational problems and improve computations done in modern day physics. I partly attribute this to the novel nature of the tools themselves, and I'm sure that given enough time and efforts, there will be widespread adoption of them to the many different subfields of physics.

In fact, such adoption has already begun with automatic differentiation finding applications in fluid dynamics and differential equation solving. From my search of the interweb and arxiv, however, the field of general relativity, and specifically numerical relativity, which deals with numerically computing quantities and solving equations of Einstein's general theory of relativity, has had little to no such adoption, which I believe it is time to change.

What are we doing?

In this exercise, I will use jax to demonstrate the power of autodiff by exploring its potential and applicability to the field of numerical relativity.

Given a metric and some coordinates, we will explore how to compute derivatives of the metric tensor (Christoffel symbols) and other relevant tensors and quantities, such as the Riemann and Ricci tensors, the Ricci scalar curvature and the Kretschmann invariant, in order to finally compute the Einstein tensor and the stress-energy-momentum tensor.

We will make use of modern Python features such as type hinting, which was introduced in recent versions of Python (3.11, 3.12, ...) and allows us to explicitly specify the data types of the variables we're using in our code to enable better readability and type safety, and decorators, which are higher-order functions that modify/augment the behavior of the functions given to them as inputs to transform said functions' inputs/outputs for a specific purpose, among other things.

Note: we will be working exclusively in SI units and not natural units of any sort for ease of interpreting calculations when deriving numerical values of quantities.

Setup

We begin by importing dependencies from the typing library for type hinting support and importing jax + its numpy variant jax.numpy.

We configure jax to force all floating point numbers to be in 64-bit precision for higher accuracy in our results.

We also define a utility decorator function close_to_zero to help round off values close to zero and below a certain arbitrarily chosen tolerance in our tensors in order to reduce floating point errors arising from numerical precision issues compounding in intermediary arithmetic calculation steps.

from typing import Callable

import jax
import jax.numpy as jnp

jax.config.update("jax_enable_x64", True)

TOLERANCE = 1e-8

def close_to_zero(func):
    def wrapper(*args, **kwargs) -> jnp.ndarray:
        result: jnp.ndarray = func(*args, **kwargs)

        return jnp.where(jnp.abs(result) < TOLERANCE, 0.0, result)

    return wrapper

Defining common metrics

We define some common metrics that are used as trivial examples for the rest of the tutorial.

# while the minkowski metric is a constant tensor that is not coordinate dependent,
# we still need to take in some "dummy" coordinates in order to make this function play nice with JAX's autodiff mechanism
def minkowski_metric(coordinates: jnp.ndarray) -> jnp.ndarray:
    """Returns the Minkowski metric in float64 precision with the (-1, 1, 1, 1) metric signature"""
    return jnp.diag(jnp.array([-1, 1, 1, 1], dtype=jnp.float64))

# this is the standard metric for a 2-sphere.
@close_to_zero
def spherical_polar_metric(coordinates: jnp.ndarray) -> jnp.ndarray:
    r, theta, phi = coordinates
    return jnp.diag(jnp.array([1, r**2, r**2 * jnp.sin(theta)**2], dtype=jnp.float64))

The Christoffel symbols

Given a metric $g_{ij}$ , the Christoffel symbols of the second kind (also called the affine connection, or the connection coefficients) are defined as derivatives of the metric contracted with the inverse metric tensor $g^{ij}$ :

Γklj=12gjm(∂gmk∂xl+∂glm∂xk−∂gkl∂xm)\Gamma_{kl}^{j} = \frac{1}{2} g^{jm} \left( \frac{\partial g_{mk} }{\partial x^l } + \frac{\partial g_{lm} }{\partial x^k } - \frac{\partial g_{kl} }{\partial x^m } \right)

Note: This definition and subsequent ones are taken from Mathematical Methods for Students of Physics and Related Fields by Sadri Hassani and uses all Roman indices. We will be doing the same and using all Roman indices throughout this tutorial, even when dealing with spacetime quantities and not just Cartesian quantities, because of their convenience when it comes to specifying them in the code. Greek symbols are non-trivial when it comes to their underlying Unicode representation, and that's a complexity I want to avoid for now.

The Christoffel symbols represent the gravitational field in a given spacetime, being derivatives of the metric, which itself is analogous to the gravitational potential in Newtonian gravitation.

Now, we define a Python function to compute the Christoffel symbols. The function takes in the coordinates to compute the Christoffel symbols at, and the metric function to compute the Christoffel symbols for, and its implementation is broadly as follows:

We evaluate the metric function to get the metric at the given coordinates and calculate the inverse metric tensor along the way using jnp.linalg.inv.
We use jax.jacfwd to compute the "Jacobian" of the metric, i.e., its partial derivatives with respect to the given coordinates, using forward-mode automatic differentiation. This is $∂gkl∂xm\frac{\partial g_{kl} }{\partial x^m }$ , also denoted as $g_{kl;m}$ . I write "Jacobian" in quotes because rigorously speaking, the Jacobian is defined as the matrix of partial derivatives of a vector-valued function with respect to its inputs, however, what we have is a tensor-valued function in the form of the metric. I am not aware of any mathematical quantity that is used to describe the higher-rank tensor associated with the derivatives of a tensor-valued function, hence my loose usage of the word "Jacobian" here.
We use jnp.einsum, an extremely convenient and performant subroutine that manipulates computational tensors and other indexed objects using standard index notation, to compute the Christoffel symbols according to the equation given above.

@close_to_zero # this ensures that any values of our Christoffel symbols are rounded off if they're close to 0
def christoffel_symbols(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    # evaluate the metric tensor at the coordinates
    g = metric(coordinates)
    # compute the inverse metric tensor
    g_inv = jnp.linalg.inv(g)
    # obtain and evaluate the "jacobian" of the metric tensor at the coordinates
    jacobian = jax.jacfwd(metric)(coordinates) # this is kl;m

    return 0.5 * jnp.einsum('jm, klm -> jkl', g_inv, jnp.einsum('klm -> mkl', jacobian) + jnp.einsum('klm -> lmk', jacobian) - jacobian)

The Torsion Tensor

The torsion tensor is defined as the antisymmetric part of a general affine connection $Γhkl\Gamma^l_{hk}$ :

Γhkl−Γkhl\Gamma^l_{hk} - \Gamma^l_{kh}

If the torsion tensor vanishes in one coordinate system, then it vanishes in all coordinate systems (the zero tensor is zero in all coordinate systems). Therefore, the torsion tensor of a general affine connection is zero if and only if the connection is symmetric, i.e.,

Γhkl=Γkhl\Gamma^l_{hk} = \Gamma^l_{kh}

Since we are dealing only with Christoffel symbols of the second kind, which is a unique and symmetric affine connection derived from the metric tensor, all the torsion tensors that we will be computing in this exploration should be zero, and the following subroutine will be used to verify that.

@close_to_zero
def torsion_tensor(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    christoffels = christoffel_symbols(coordinates, metric)

    return christoffels - jnp.einsum('ijk -> ikj', christoffels)

The Riemann Curvature Tensor and the Kretschmann Invariant

The Riemann tensor $RklmjR^j_{klm}$ encodes the intrinsic curvature of the Riemannian (or semi-Riemannian) manifold produced by any given metric. It is defined as derivatives of the Christoffel symbols $Γklj\Gamma^j_{kl}$ as:

Rklmj=∂mΓklj−∂lΓkmj+ΓrmjΓklr−ΓrljΓkmrR^j_{klm} = \partial_m \Gamma^j_{kl} - \partial_l \Gamma^j_{km} + \Gamma^j_{rm} \Gamma^r_{kl} - \Gamma^j_{rl} \Gamma^r_{km}

We define a Python function that:

Uses the previous christoffel_symbols function to obtain the Christoffel symbols for a given set of metric and coordinates.
Computes the "jacobian" of the Christoffel symbols to obtain $Γkl;mj\Gamma^j_{kl;m}$ .
Manipulates this "jacobian" tensor and products of the Christoffel symbols using jnp.einsum to obtain the Riemann tensor

We also define a Python function to compute the Kretschmann invariant from the Riemann tensor, a scalar that is used to look for true physical singularities (gravitational singularities) in certain manifolds independent of the choice of coordinates:

R^{jklm} R_{jklm}

We will use the Kretschmann invariant later on in this tutorial to verify whether the Riemann tensor implementation we have below is correct or not when using the Schwarzchild metric as a case study to verify the correctness of these subroutines.

@close_to_zero
def riemann_tensor(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    christoffels = christoffel_symbols(coordinates, metric)
    jacobian = jax.jacfwd(christoffel_symbols)(coordinates, metric) # computes jkl;m

    return jacobian - jnp.einsum('jklm -> jkml', jacobian) + jnp.einsum('jrm, rkl -> jklm', christoffels, christoffels) - jnp.einsum('jrl, rkm -> jklm', christoffels, christoffels)

@close_to_zero
def kretschmann_invariant(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    riemann = riemann_tensor(coordinates, metric)

    g = metric(coordinates)
    g_inv = jnp.linalg.inv(g)

    riemann_upper = jnp.einsum('pj, qk, rl, ijkl -> ipqr', g_inv, g_inv, g_inv, riemann) # computes R^{jklm} by contracting with three inverse metric tensors
    riemann_lower = jnp.einsum('pi, ijkl -> pjkl', g, riemann) # computes R_{jklm} by contracting with one metric tensor

    return jnp.einsum('ijkl, ijkl ->', riemann_upper, riemann_lower)

The Ricci tensor and the Ricci scalar curvature

The Ricci tensor $R_{kl}$ is another curvature-related quantity which is defined as the trace component of the Riemann tensor $RklmjR^j_{klm}$ . The Ricci tensor is obtained by contracting the only contravariant index of the Riemann tensor with its last covariant index:

Rkl=RkljjR_{kl} = R^j_{klj}

Physically, the Ricci tensor encodes information about how volumes change in the presence of tidal forces.

By raising one of the Ricci tensor's indices and contracting, we obtain the Ricci scalar curvature

R = R^l_l = g^{kl}R_{kl}

We implement Python subroutines to do this computationally using jnp.einsum again to manipulate indices and perform contractions.

@close_to_zero
def ricci_tensor(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    riemann = riemann_tensor(coordinates, metric)

    return jnp.einsum('jklj -> kl', riemann) # contracting the first and last indices

@close_to_zero
def ricci_scalar(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.float32:
    g = metric(coordinates)
    g_inv = jnp.linalg.inv(g)
    ricci = ricci_tensor(coordinates, metric)

    return jnp.einsum('kl, kl -> ', g_inv, ricci) # trace of the ricci tensor

The Einstein tensor and the stress-energy-momentum tensor

The Einstein tensor, the crown jewel of the general theory of relativity, encodes all information about the curvature of a spacetime manifold, and is defined in terms of the Ricci tensor, the metric tensor, and the Ricci scalar curvature as:

Gij≡Rij−12gijRG_{ij} \equiv R_{ij} - \frac{1}{2} g_{ij} R

It forms the left-hand-side of the Einstein Field Equations (EFEs), a set of 16 coupled partial differential equations that relate the curvature of a spacetime manifold to the mass-energy content in it:

Gij=8πGc4TijG_{ij} = \frac{8 \pi G}{c^4} T_{ij}

The right hand side $T_{ij}$ is the stress-energy-momentum tensor, that encodes information about all the mass-energy present in a spacetime manifold.

We write Python functions to call the subroutines implemented before to trivially compute the Einstein tensor and the stress-energy-momentum tensor using the equations we just described.

@close_to_zero
def einstein_tensor(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    g = metric(coordinates)
    rt = ricci_tensor(coordinates, metric)
    rs = ricci_scalar(coordinates, metric)

    return rt - 0.5 * g * rs

@close_to_zero
def stress_energy_momentum_tensor(coordinates: jnp.ndarray, metric: Callable[[jnp.ndarray], jnp.ndarray]) -> jnp.ndarray:
    G = einstein_tensor(coordinates, metric)

    kappa = (8 * jnp.pi * 6.67e-11) / ((299792458)**4)

    return G / kappa

Case study 1: the 2-sphere metric

Using all of the Python functions we have defined before, let's perform calculations in float64 precision for the 2-sphere metric given in spherical polar coordinates $\theta, \phi)$ as:

gij=diag(1,r2,r2sin⁡2(θ))g_{ij} = \text{diag}(1, r^2, r^2 \sin^2(\theta))

We use the following coordinate values for the calculations, which were arbitrarily chosen

r = 5

θ=π/3\theta = \pi/3

ϕ=π/2\phi = \pi/2

coordinates = jnp.array([5, jnp.pi/3, jnp.pi/2], dtype=jnp.float64)
metric = spherical_polar_metric

print(f"Christoffel symbols: {christoffel_symbols(coordinates, metric)}")
print(f"Torsion tensor: {torsion_tensor(coordinates, metric)}")
print(f"Riemann tensor: {riemann_tensor(coordinates, metric)}")
print(f"Ricci tensor: {ricci_tensor(coordinates, metric)}")
print(f"Ricci scalar: {ricci_scalar(coordinates, metric)}")
print(f"Einstein tensor: {einstein_tensor(coordinates, metric)}")
print(f"Stress-energy-momentum tensor: {stress_energy_momentum_tensor(coordinates, metric)}")
print(f"Kretschmann invariant: {kretschmann_invariant(coordinates, metric)}")

Running this, we get

Christoffel symbols: [[[ 0.          0.          0.        ]
  [ 0.         -5.          0.        ]
  [ 0.          0.         -3.75      ]]

 [[ 0.          0.2         0.        ]
  [ 0.2         0.          0.        ]
  [ 0.          0.         -0.4330127 ]]

 [[ 0.          0.          0.2       ]
  [ 0.          0.          0.57735027]
  [ 0.2         0.57735027  0.        ]]]
Torsion tensor: [[[0. 0. 0.]
  [0. 0. 0.]
  [0. 0. 0.]]

 [[0. 0. 0.]
  [0. 0. 0.]
  [0. 0. 0.]]

 [[0. 0. 0.]
  [0. 0. 0.]
  [0. 0. 0.]]]
Riemann tensor: [[[[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]

  [[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]

  [[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]]


 [[[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]

  [[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]

  [[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]]


 [[[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]

  [[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]

  [[0. 0. 0.]
   [0. 0. 0.]
   [0. 0. 0.]]]]
Ricci tensor: [[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
Ricci scalar: 0.0
Einstein tensor: [[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
Stress-energy-momentum tensor: [[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
Kretschmann invariant: 0.0

We obtain all of the Christoffel symbols for this metric. Furthermore, as expected, the Riemann tensor, Ricci tensor, Ricci scalar, the Einstein tensor, and the stress-energy-momentum tensor all vanish since this metric describes a flat spacetime with no curvature and no mass-energy content. The torsion tensor is also zero as expected by the symmetric nature of the Christoffel symbols.

Case study 2: the Schwarzschild metric

Now we come to a more interesting case study, the Schwarzschild metric, which describes the spacetime of an uncharged, unrotating, spherically symmetric body in vacuum. The Schwarzschild metric in traditional spherical polar spacetime coordinates $\theta, \phi)$ is given as:

gij=diag(−(1−rsr)c2,1(1−rsr),r2,r2sin⁡2(θ))g_{ij} = \text{diag}\left(- \left( 1 - \frac{r_s}{r} \right) c^2, \frac{1}{\left(1 - \frac{r_s}{r} \right)}, r^2, r^2 \sin^2(\theta) \right)

Where $r_s$ is the Schwarzschild radius of the massive body, a scale factor which is related to its mass $M$ by:

rs=2GMc2r_s = \frac{2 G M}{c^2}

We do exactly the same as the previous case study, computing all the quantities in float64 precision for a body with ~4.3 million solar masses using the following arbitrary chosen spacetime spherical polar coordinates

t = 3600

r = 3000

θ=π/3\theta = \pi/3

ϕ=π/2\phi = \pi/2

G = 6.67e-11
c = 299792458.0

M = 4.297e+6 * 1.989e+30 # 4.3 million solar masses, mass of Sgr A*

# schwarzschild radius
rs = (2 * G * M) / c**2

@close_to_zero
def schwarzschild_metric(coordinates: jnp.ndarray) -> jnp.ndarray:
    t, r, theta, phi = coordinates

    return jnp.diag(jnp.array([-(1 - (rs / r)) * c**2, 1/(1 - (rs/r)), r**2, r**2 * jnp.sin(theta)**2], dtype=jnp.float64))

coordinates = jnp.array([3600, 3000, jnp.pi/3, jnp.pi/2], dtype=jnp.float64)
metric = schwarzschild_metric

print(f"Christoffel symbols: {christoffel_symbols(coordinates, metric)}")
print(f"Torsion tensor: {torsion_tensor(coordinates, metric)}")
print(f"Riemann tensor: {riemann_tensor(coordinates, metric)}")
print(f"Ricci tensor: {ricci_tensor(coordinates, metric)}")
print(f"Ricci scalar: {ricci_scalar(coordinates, metric)}")
print(f"Einstein tensor: {einstein_tensor(coordinates, metric)}")
print(f"Stress-energy-momentum tensor: {stress_energy_momentum_tensor(coordinates, metric)}")
print(f"Kretschmann invariant: {kretschmann_invariant(coordinates, metric)}")

Running this outputs

Christoffel symbols: [[[ 0.00000000e+00 -1.66666706e-04  0.00000000e+00  0.00000000e+00]
  [-1.66666706e-04  0.00000000e+00  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

 [[-2.67840757e+26  0.00000000e+00  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  1.66666706e-04  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  1.26857006e+10  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  9.51427546e+09]]

 [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  3.33333333e-04  0.00000000e+00]
  [ 0.00000000e+00  3.33333333e-04  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  0.00000000e+00 -4.33012702e-01]]

 [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
  [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  3.33333333e-04]
  [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  5.77350269e-01]
  [ 0.00000000e+00  3.33333333e-04  5.77350269e-01  0.00000000e+00]]]
Torsion tensor: [[[0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]]

 [[0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]]

 [[0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]]

 [[0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]]]
Riemann tensor: [[[[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  1.11111137e-07  0.00000000e+00  0.00000000e+00]
   [-1.11111137e-07  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  2.11428394e+06  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [-2.11428394e+06  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.58571295e+06]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [-1.58571295e+06  0.00000000e+00  0.00000000e+00  0.00000000e+00]]]


 [[[ 0.00000000e+00  1.78560505e+23  0.00000000e+00  0.00000000e+00]
   [-1.78560505e+23  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  2.11428394e+06  0.00000000e+00]
   [ 0.00000000e+00 -2.11428394e+06  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.58571295e+06]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00 -1.90734863e-06]
   [ 0.00000000e+00 -1.58571295e+06  1.90734863e-06  0.00000000e+00]]]


 [[[ 0.00000000e+00  0.00000000e+00 -8.92802525e+22  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 8.92802525e+22  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  5.55555687e-08  0.00000000e+00]
   [ 0.00000000e+00 -5.55555687e-08  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00 -3.17142590e+06]
   [ 0.00000000e+00  0.00000000e+00  3.17142590e+06  0.00000000e+00]]]


 [[[ 0.00000000e+00  0.00000000e+00  0.00000000e+00 -8.92802525e+22]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 8.92802525e+22  0.00000000e+00  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  5.55555687e-08]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00 -5.55555687e-08  0.00000000e+00  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  4.22856787e+06]
   [ 0.00000000e+00  0.00000000e+00 -4.22856787e+06  0.00000000e+00]]

  [[ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]
   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00]]]]
Ricci tensor: [[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]
Ricci scalar: 0.0
Einstein tensor: [[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]
Stress-energy-momentum tensor: [[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]
Kretschmann invariant: 2.649005370647907

Like before, we've obtained our Christoffel symbols for the Schwarzschild metric and the torsion tensor is zero as expected.

Furthermore, since the Schwarzschild metric is a vacuum solution to the Einstein Field Equations, the Ricci tensor, Ricci scalar, Einstein tensor, and Stress-energy-momentum tensor are all zero.

However, observe that the Riemann tensor is not zero. In fact, some of its components are quite large in magnitude. We also obtain a value for the Kretschmann invariant from the Riemann tensor's contraction. To verify this and thereby verify whether the components of our calculated Riemann tensor are correct or not, we can compute the Krietschmann invariant directly using a relatively trivial scalar formula:

RjklmRjklm=48G2M2c4r6R^{jklm} R_{jklm} = \frac{48 G^2 M^2}{c^4 r^6}

G = 6.67e-11
M = 4.297e+6 * 1.989e+30 # 4.3 million solar masses, mass of Sgr A*
c = 299792458.0

r = 3000

kr = (48 * G**2 * M**2) / (c**4 * r**6)

print("Kretschmann invariant is", kr)

Running this, we get the output

Kretschmann invariant is 2.649005370647906

Which matches the result we obtained by contracting all indices of the Riemann tensor up to 15 decimal places!

Conclusion

In this exploration, we've witnessed first hand the power of automatic differentiation in enabling derivative computation of all quantities to maximum machine precision in terms of floating point accuracy.

We saw the effectiveness and simplicity of JAX's jax.jacfwd to compute Jacobians/derivatives of tensors in forward-mode automatic differentiation.

We've also seen the benefits of JAX's jnp.einsum with its ability to do index manipulations and perform calculations at a high level, allowing one to do tensor calculus operations in a convenient and efficient manner. The language of tensors and tensor calculus is pervasive in other topics and fields of theoretical physics, such as covariant electromagnetism and quantum field theory. At the very least, it would be an interesting exercise to explore a similar approach leveraging modern AI/ML libraries and frameworks in order to perform tensor computations for calculations arising in these fields.

JAX's capabilities further allows us to parallelize all of this code and run it on accelerators such as GPUs and TPUs with no changes to the original code, in order to speed up calculations for heavy numerical computations and simulations requiring parallelism, speed, and efficiency. We have not explored that aspect specifically in this demonstration, and all computations have been done on the CPU since none of the computations or use cases required usage of parallelism and accelerators, but nonetheless the capabilities exist and can be called upon if a specific use case demands it.

The explanations, scripts, and results of this exploration are also conveniently documented in this Jupyter notebook hosted on Google Colab, which you can use to reproduce and build upon the ideas presented here.

Future work

Some ideas and extensions in this general direction that are worth exploring:

Perform the same calculations for other prominent metrics such as the Kerr and Kerr-Newmann metrics
Write functions to compute the Weyl tensor and the Weyl invariant, other important curvature-related quantities.
Provide alternative but equivalent PyTorch and Tensorflow implementations to the JAX implementation here.
Use the @jax.jit decorator on relevant subroutines for using JIT compilation in order to speed up computations.
Look into jax.jacrev for reverse-mode automatic differentiation and try to combine it with jax.jacfwd for optimal speed and accuracy.
Configure this code to run on GPUs and other accelerators.
Train a neural network to parameterize the metric tensor and solve an optimization problem in order to find the metric tensor components from data.
Compute the Christoffel symbols for a metric and solve the geodesic equation to find the EOM using automatic differentiation powered differential equation solvers, such as diffrax
Extend this methodology to other aspects and calculations of numerical relativity, specifically those tackled by popular general relativity Python libraries such as EinsteinPy

Reverse engineering IAN using IAN - Part 1: The TypeScript client

Baalateja Kataru — Tue, 17 Sep 2024 11:28:54 +0000

Introduction

I recently got access to IAN by Cyphae.

IAN, short for Intelligent Agent Network, is as the name implies a network of LLM agents with a meta-agentic analysis system on top that collates and provides a systematic analysis of the outputs of the agent network along with their individual responses. IAN is free to use and does not need one to be signed up to use it.

Ever since I started playing around with it, I've been thoroughly impressed by its ability to be more accurate than any one of the individual LLMs operating in isolation.

My theory is that this is due to the network effect of agents collaborating and correcting each other's shortcomings. If statistical mechanics and nuclear physics have taught us anything, it's that the whole is greater than the sum of the parts, and this may be true for AI systems as well.

IAN initializes with a default squad consisting of

GPT-4o
Perplexity
Claude 3.5

While these are quite good at most tasks, they aren't as effective as the deck I usually ball with:

LLaMa3.1-70b (or LLaMa3.1-405b when it works)
Perplexity
Claude 3.5

I've found that a lineup combining Claude 3.5's sophisticated creativity, Perplexity's retrieval augmented generation superpowers, and LLaMa's reasoning and text processing capabilities seems to work the best for my prompting style.

What can IAN do for me

As an example illustrating what IAN can do, a while back I was trying to improve my command over the concept of lifetimes in Rust, with the hopes of reducing how often I get my ass kicked by the borrow checker.

Being lazy and not wanting to spend time browsing resources and collating information myself when we are living in the age of language models, I prompted IAN to help me out with this.

I asked

Can you help me improve my understanding of how lifetimes work in Rust. I'd appreciate it if you could supplement your explanations with some examples and exercises for practice as well as important pointers to when it's necessary to use lifetimes.

IAN responded with

Understanding Lifetimes in Rust

Lifetimes in Rust are annotations that describe the scope of reference validity. They ensure memory safety by preventing dangling references and are a key part of Rust's ownership system.
Key Concepts

    Reference Validity: Ensures references point to valid data within the scope.
    Borrow Checking: Rust's compiler mechanism for safe references.
    Compile-Time Feature: Checks lifetimes during compilation to ensure memory safety.
    Elision: The compiler infers lifetimes in straightforward cases to simplify code.

Basic Syntax

fn example<'a>(x: &'a i32) -> &'a i32 {
    x
}


Examples

    Basic Lifetime Annotation:

    fn longest<'a>(s1: &'a str, s2: &'a str) -> &'a str {
        if s1.len() > s2.len() {
            s1
        } else {
            s2
        }
    }


    This function takes two string slices with the same lifetime and returns the longest one.
    Structs with Lifetimes:

    struct ImportantExcerpt<'a> {
        part: &'a str,
    }

    fn main() {
        let novel = String::from("Call me Ishmael. Some years ago...");
        let first_sentence = novel.split('.').next().expect("Could not find a '.'");
        let i = ImportantExcerpt { part: first_sentence };
    }


    The struct `ImportantExcerpt` holds a reference with a lifetime `'a`.

Exercises

    Exercise 1: Write a function `smallest` that returns the smallest of two integer references with the same lifetime annotation.

    fn smallest<'a>(x: &'a i32, y: &'a i32) -> &'a i32 {
        if x < y {
            x
        } else {
            y
        }
    }


    Exercise 2: Define a struct `Book` with a title and author as string slices. Write a function that takes a slice of books and returns the book with the longest title.

    struct Book<'a> {
        title: &'a str,
        author: &'a str,
    }

    fn book_with_longest_title<'a>(books: &'a [Book<'a>]) -> &'a Book<'a> {
        let mut longest_book = &books[0];
        for book in books.iter() {
            if book.title.len() > longest_book.title.len() {
                longest_book = book;
            }
        }
        longest_book
    }


When to Use Lifetimes

    When writing functions that take references as parameters and/or return references.
    When your structs hold references to prevent dangling references.
    When dealing with complex scope and reference relationships to satisfy the borrow checker.

Informational Differences and Inconsistencies

    Key Concept Emphasis:
        GPT-4o and Perplexity emphasize the role of lifetimes in preventing dangling references.
        Claude 3.5 focuses on the compiler's role and the implicit nature of lifetimes.
        Likely Correct: GPT-4o and Perplexity, as preventing dangling references is central to Rust's ownership system.

Agent Strengths and Weaknesses

    GPT-4o:
        Strength: Provides detailed explanations and examples.
        Weakness: Can be verbose and slightly complex for beginners.
    Perplexity:
        Strength: Provides a detailed and comprehensive explanation.
        Weakness: Lacks some practical exercises to reinforce concepts.
    Claude 3.5:
        Strength: Simplicity and clear examples.
        Weakness: Less detailed explanation on the importance of lifetimes.

Apart from answering the actual prompt itself and providing working code, IAN also provides a comparative analysis of "Informational Differences and Inconsistencies" and "Agent Strengths and Weaknesses", distilling each LLM agent's strengths and weaknesses in the context of the differences between their responses.

To me, this is what sets it apart as an LLM-powered service. You're not just prompting one LLM. You're prompting three. And you're not just going to be getting their responses. You're also going to be getting an analysis of how well they performed in comparison to each other. This is a big win for smarter and fairer AI systems.

What can I do for IAN?

Given IAN's increased precision and accuracy, which tends to lower the odds of hallucinations, I've been using it to help me understand codebases and generate code snippets. I craft prompts containing instructions and source files strategically interwoven together to maximize the context I'm able to give, and these days LLMs have context windows that are massive enough that they can process hundreds to thousands of lines with ease so I don't have to be particularly worried about prompt size.

I've also been wanting to get IAN in the CLI. Like a true turtle, I spend most of my time in a shell, and having the ability to prompt and access IAN from the CLI would be quite convenient. It makes it far easier for specific tasks such as when providing source files and code bases as context to IAN without needing to tediously paste them into the prompt by hand.

All of this led me to wonder about the possibility of building applications on top of IAN. I realized this would require such applications to be able to interact with IAN programmatically. However, I know IAN is still in beta and does not have a full-featured web UI yet, so an API client is a hard ask.

A quick search on GitHub and the interweb didn't give me any relevant results for any preexisting solutions, but we cannot do without clients if we are to build applications. IAN's web UI exists, and technically it is a client. What if we could potentially reverse engineer it to hack together an API interface specification that we can use to build our clients?

My mind couldn't stop thinking about the immense potential and possibilities, so it was only a matter of time before I arrived at the thought of "Hey, this is a really useful thing and I want to use it in more places. I think it's worth a little hack. What's to lose?"

Besides, I have IAN with me. With a general idea of what I want to do and how to proceed at every step, I don't need to spend more time than necessary figuring out implementation details when I can accelerate significantly by taking IAN's help with figuring those out.

This will be an odyssey in prompt crafting and LLM-powered reverse engineering in order to create ianOS (yes, I came up with it), a suite of clients and applications designed to be used with IAN.

Hacking the web UI

To begin, we must first ask the question of what IAN even is as an application.

IAN's interface at cyphae.com is a web UI, and any web UI, let it be IAN or ChatGPT or Perplexity, makes (usually RESTful HTTP) network requests to a backend cloud service with a data payload containing the user's prompt with the goal of invoking an LLM for completions and answering.

So I opened the Network tab in Firefox's developer tools interface to monitor what requests are being sent and what responses are being received.

I also opened the console tab to quickly execute JavaScript scripts and snippets to test things out along the way.

I made sure to log out because I want an API that works even if one is not logged in, just like IAN's web UI. I didn't want the state/behavior of the web UI to be unintentionally influenced by my user account's credentials in any way.

With the Network tab open, I first refreshed the page to bench what requests are made at the time of First Contentful Paint (FCP).

A couple of HTTP GET requests to fetch media and code files, nothing special.

Of course it wouldn't be anything special because IAN will only make a HTTP call for LLM inference when we submit a prompt in the web UI.

So let's do that.

I prompted IAN as follows

Can you write me a HTTP GET request in C++? If you're using any external libraries, please remember to include the relevant CMakeLists.txt and any other Make/CMake files that may be necessary.

As I submitted the prompt, two HTTP calls - a POST request and an OPTIONS request - were made to a domain called oraloom.com

The OPTIONS request completed near-instantly. The POST request, on the other hand, took a good 30 seconds to complete and transferred about 7 KB of text/plain data back and forth, which could be the prompt and its corresponding response strings.

The OPTIONS request's only purpose is to return the list of headers and methods that are allowed by the server, so it's not of much use to us.

Instead, let's inspect the POST request. Based on its HTTP verb (POST), response latency, and payload size, it seems to be the request responsible for sending the prompt payload to IAN's backend.

In Firefox, you can right-click any network request in the developer tools and select Copy as Fetch to copy a JS code snippet to your clipboard containing a fetch request call for the respective network request. Alternatively, you can also select Use as Fetch in Console to automatically paste the snippet into the console.

Selecting Copy as Fetch, let's copy this snippet to the clipboard.

await fetch("https://oraloom.com/prompt", {
    "credentials": "omit",
    "headers": {
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0",
        "Accept": "*/*",
        "Accept-Language": "en-US,en;q=0.5",
        "Content-Type": "application/json",
        "Sec-Fetch-Dest": "empty",
        "Sec-Fetch-Mode": "cors",
        "Sec-Fetch-Site": "cross-site",
        "Priority": "u=4"
    },
    "referrer": "https://cyphae.com/",
    "body": "{\"account\":\"guest\",\"wallet\":\"1\",\"user\":\"guest\",\"a1\":\"GPT-4o\",\"a2\":\"Perplexity\",\"a3\":\"Claude 3.5\",\"input\":\"Can you write me a HTTP GET request in C++? If you're using any external libraries, please remember to include the relevant CMakeLists.txt and any other Make/CMake files that may be necessary.\"}",
    "method": "POST",
    "mode": "cors"
});

Now let's paste this into the console and run it to see if it works out of the box without requiring any tweaks like changing the headers or providing any one-time hashes generated for this client-browser session.

On running this, I encountered a NetworkError pertaining to the kind of headers CORS allows via Access-Control-Allow-Headers when accessing remote resources.

Apparently, the priority header is disallowed. That's okay, let's remove it from the POST request and try again.

Damn, another disallowed header. Okay fine, let's remove user-agent and try once more.

Oh, that's better.

It seems fetch succeeded and obtained a HTTPResponse object with a body containing a ReadableStream object.

That is very promising. ReadableStreams are commonly used to stream responses from LLMs over the internet to a client so this means we're on the right track.

Let's copy this JS fetch snippet over to Neovim and run it locally

P.S.: The fetch snippet uses async/await but I'm running a node version of v20.15.1, which does not support top-level async/await as a stable feature, so I dumped the snippet into an enclosing async function main() and invoked it. I also called await response.text() to synchronously parse the ReadableStream object in the body into a string before passing it into console.log.

Upon running this, it takes a while but eventually we receive a response consisting of a HTML document containing the LLM's response along with what seems to be markup and styling to render the response with any browser or web UI.

<html lang="en>

<head>  
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Aggregated HTTP GET in C++ Guide</title>
    <style>
        body {  
            font-family: Arial, sans-serif;
            background-color: black;
            color: white;
            padding: 20px;
        }

        h1 {
            background: linear-gradient(to right, #60a5fa, #fdba74); 
            -webkit-background-clip: text;
            color: transparent;   
        }

        .section-title {
            text-decoration: underline; 
            font-size: 1.2em; 
        }

        .agent-box {
            border: 1px solid #60a5fa;  
            padding: 10px;   
            margin-bottom: 20px; 
        }

        code {                    
            display: block;                
            white-space: pre-wrap;
            background-color: #333; 
            padding: 10px; 
            border-radius: 5px;   
            margin-top: 10px;  
            margin-bottom: 10px;
        } 
    </style>
</head>

<body>
    <h1>Aggregated HTTP GET in C++ Guide</h1>

    <h2 class="section-title">Introduction</h2>             
    <p>This guide provides examples and build instructions on how to perform an HTTP GET request in C++ using different libraries. We'll cover libcurl, cpp-httplib, and HTTPRequest. Each exa
mple includes consistent information while acknowledging any informational differences and highlighting each approach's strengths and weaknesses.</p> 

    <h2 class="section-title">Consistent Information</h2>
    <p>The common approaches for making HTTP GET requests in C++ involve using external libraries such as <code>libcurl</code>, <code>cpp-httplib</code>, or <code>HTTPRequest</code>. The set
up involves writing the C++ code and configuring the build system using CMake.</p>

    <h2 class="section-title">C++ Code Examples and Build Instructions</h2>

    <div class="agent-box">
        <h3>Instructions Using libcurl</h3>
        <code>
            // main.cpp<br>
            #include &lt;iostream&gt;<br>
            #include &lt;curl/curl.h&gt;<br><br>
            size_t WriteCallback(void* contents, size_t size, size_t nmemb, std::string* output) {<br>
                size_t total_size = size * nmemb;<br>
                output->append((char*)contents, total_size);<br>
                return total_size;<br>
            }<br><br>
            int main() {<br>
                CURL* curl;<br>
                CURLcode res;<br>
                std::string response_string;<br>
                curl_global_init(CURL_GLOBAL_DEFAULT);<br>
                curl = curl_easy_init();<br>
                if (curl) {<br>
                    curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com");<br>
                    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);<br>
                    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response_string);<br>
                    res = curl_easy_perform(curl);<br>
                    if (res != CURLE_OK) {<br>
                        std::cerr << "curl_easy_perform() failed: " << curl_easy_strerror(res) << std::endl;<br>
                    } else {<br>
                        std::cout << "Response data: " << response_string << std::endl;<br>
                    }<br>
                    curl_easy_cleanup(curl);<br>
                }<br>
                curl_global_cleanup();<br>
                return 0;<br>
            }
        </code>

        <h4>Build Configuration (CMakeLists.txt)</h4>
        <code>
            cmake_minimum_required(VERSION 3.10)<br>
            project(HttpGetExample)<br><br>
            set(CMAKE_CXX_STANDARD 11)<br><br>
            find_package(CURL REQUIRED)<br><br> 
            add_executable(HttpGetExample main.cpp)<br>
            target_link_libraries(HttpGetExample ${CURL_LIBRARIES})<br>
            include_directories(${CURL_INCLUDE_DIRS})
        </code>
    </div>

    <div class="agent-box">
        <h3>Instructions Using cpp-httplib</h3> 
        <code>
            // main.cpp<br>
            #include "httplib.h"<br><br>
            int main() {<br>
                httplib::Client cli("http://example.com");<br>
                auto res = cli.Get("/path/to/resource");<br>
                if (res) {<br>
                    std::cout << "Status: " << res->status << std::endl;<br>
                    std::cout << "Body: " << res->body << std::endl;<br>
                } else {<br>
                    std::cerr << "Request failed: " << cli.get_error_message() << std::endl;<br>
                }<br>
                return 0;<br>
            }
        </code>
        <h4>Build Configuration (CMakeLists.txt)</h4>
        <code>
            cmake_minimum_required(VERSION 3.10)<br>
            project(HTTPLibExample)<br><br>
            set(CMAKE_CXX_STANDARD 14)<br><br>
            add_executable(${PROJECT_NAME} main.cpp)<br>
            target_include_directories(${PROJECT_NAME} PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
        </code>
    </div>

    <div class="agent-box"> 
        <h3>Instructions Using HTTPRequest</h3>  
        <code>    
            // main.cpp<br>
            #include "HTTPRequest.hpp"<br><br>
            int main() {<br>
                try {<br>
                    http::Request request{"http://example.com/path/to/resource"};<br>
                    const auto response = request.send("GET");<br>
                    std::cout << std::string{response.body.begin(), response.body.end()} << '\\n';<br>
                } catch (const std::exception& e) {<br>
                    std::cerr << "Request failed, error: " << e.what() << '\\n';<br>
                }<br>
                return 0;<br>
            }
        </code>
        <h4>Build Configuration (CMakeLists.txt)</h4>
        <code>
            cmake_minimum_required(VERSION 3.10)<br>
            project(HTTPRequestExample)<br><br> 
            set(CMAKE_CXX_STANDARD 17)<br><br>
            add_executable(${PROJECT_NAME} main.cpp)<br>
            target_include_directories(${PROJECT_NAME} PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
        </code>
    </div>

    <h2 class="section-title">Informational Differences and Inconsistencies</h2>
    <ul>
        <li><strong>Libraries Used:</strong>
            <ul>
                <li><strong>libcurl:</strong> Recommended widely across sources for its robustness.</li>
                <li><strong>cpp-httplib:</strong> Suggested by Perplexity, known for its simplicity and ease of use.</li>
                <li><strong>HTTPRequest:</strong> Also highlighted by Perplexity for its simplicity.</li>
            </ul>
        </li>
        <li><strong>URL Provided:</strong>
            <ul>
                <li><strong>General Usage:</strong> "http://example.com" (placeholder URL, most likely correct; commonly used for demonstration purposes).</li>
                <li><strong>Specific Example:</strong> "https://api.example.com/data" (used by Claude 3.5, replace with actual endpoint).</li>
            </ul>
        </li>
    </ul>
    <p><strong>Likely Correct:</strong> The placeholder URL "http://example.com" is commonly used for demonstration purposes and should be replaced with the target URL as needed.</p>

    <h2 class="section-title">Agent Strengths and Weaknesses</h2>
    <div class="agent-box">
        <h3>GPT-4o</h3>
        <ul>
            <li><strong>Strength:</strong> Provides a detailed example and CMake setup for libcurl.</li>
            <li><strong>Weakness:</strong> Does not offer alternative library options.</li>
        </ul>
    </div>
    <div class="agent-box">
        <h3>Perplexity</h3>
        <ul>
            <li><strong>Strength:</strong> Introduces simpler library options like `cpp-httplib` and `HTTPRequest`.</li>
            <li><strong>Weakness:</strong> Does not cover widely-used options like libcurl.</li>
        </ul>
    </div>
    <div class="agent-box">
        <h3>Claude 3.5</h3>
        <ul>
            <li><strong>Strength:</strong> Comprehensive explanation and setup for libcurl.</li>
            <li><strong>Weakness:</strong> Uses a specific URL and does not mention simpler library options.</li>
        </ul>
    </div>

    <h2 class="section-title">Building and Running the Project</h2>
    <p>Follow these steps to build and run your project:</p>
    <ol>
        <li>Create a directory for the build:
            <code>mkdir build && cd build</code>
        </li>
        <li>Run CMake to configure the build environment:
            <code>cmake ..</code>
        </li>
        <li>Build the project:
            <code>make</code>
        </li>
        <li>Run the executable:
            <code>./<em>executable_name</em></code>
        </li>
    </ol>
    <p><strong>Note:</strong> Replace <em>executable_name</em> with the name used in your CMake configuration.</p>
    <p>If you need further assistance or have any questions, feel free to ask!</p>
</body>
</html>

Curious to see how this looks like, I dumped this into an index.html file and opened it in Firefox to see the rendered output.

Impressive! Although the design choice to send back an HTML document may have been for internal reasons, I for one appreciate this approach. This way, building the web UI component of any application attempting to integrate with IAN becomes much easier.

Nice, so we have some working JS code that we can use as a starting point to build a more comprehensive client.

Building The Client

First, let's flesh out the JS snippet and tweak it to be more general by allowing it to accept external data as input to relevant parameters.

To do this, I just prompted IAN for the following

I have this js snippet sending a HTTP request and reading out the response to the console.

*js code pasted here*

I have a couple of modifications I want you to make.

Make this code more general such that it accepts external data passed into it as input for parameters such as account, wallet, user, a1, a2, a3, input

Refactor the request functionality into a separate function called postIANPrompt and call this inside main()

This was IAN's solution

async function postIANPrompt(url, account, wallet, user, a1, a2, a3, input) {
    const body = JSON.stringify({
        account,
        wallet,
        user,
        a1,
        a2,
        a3,
        input,
    });

    const response = await fetch(url, {
        credentials: "omit",
        headers: {
            Accept: "*/*",
            "Accept-Language": "en-US,en;q=0.5",
            "Content-Type": "application/json",
            "Sec-Fetch-Dest": "empty",
            "Sec-Fetch-Mode": "cors",
            "Sec-Fetch-Site": "cross-site",
        },
        referrer: "https://cyphae.com/",
        body,
        method: "POST",
        mode: "cors",
    });

    if (!response.ok) {
        throw new Error(`Response status: ${response.status}`);
    }

    return await response.text();
}

async function main(account, wallet, user, a1, a2, a3, input) {
    const url = "https://oraloom.com/prompt";

    try {
        const result = await postIANPrompt(
            url,
            account,
            wallet,
            user,
            a1,
            a2,
            a3,
            input,
        );

        console.log(result);
    } catch (error) {
        console.error("Error:", error);
    }
}

// Example usage:
main(
    "guest",
    "1",
    "guest",
    "GPT-4o",
    "Perplexity",
    "Claude 3.5",
    "Can you write me a HTTP GET request in C++? If you're using any external libraries, please remember to include the relevant CMakeLists.txt and any other Make/CMake files that may be necessary.",
);

Quite comprehensive, and IAN also decided to include error handling when I didn't initially prompt it to do so, a welcome addition. This is also acknowledged in its aggregated analysis of the LLM outputs.

I implemented the changes and executed the latest version, which ran without any errors.

Alright, so far so good.

Upgrading to TypeScript

Now, JavaScript is nice, but TypeScript is a gift from the type theory gods for a better web development experience for everyone. So let's bring in the TypeScript compiler.

I also want to add some features to make this client more user-friendly. For example, each request takes at least 5-50 seconds to complete and we can make this waiting process less painful for the user with some kind of indicator in the UI experience.

So first, I used bun to init a new project, and install the TypeScript compiler and bun's types.

$ bun init
$ bun add -d typescript @types/bun

bun scaffolds a couple of files for us and we end up with this file structure.

Next, I asked IAN to write me a full fledged TypeScript client keeping in mind to include the following features:

Typings, interfaces, error handling, and schemas for DTOs.
Abstract out the request into a class that can be instantiated to create multiple request objects.
Typings and options for a1, a2, a3, i.e., parameters to specify which models to be used for the agent network when authenticated, with options being string ids from the list ["GPT-4o", "GPT-4o-mini", "GPT-3.5", "Perplexity", "Gemini 1.5 Pro", "Gemini 1.5 Flash", "Gemini Pro", "Llama3.1-405b", "Llama3.1-70b", "Llama3-70b", "Qwen2-72B", "Mistral-8x22b-instruct", "Claude 3 Haiku", "Claude 3 Opus", "Claude 3 Sonnet", "Claude 3.5", "none"] obtained from the IAN web UI.
Typings and options to specify IAN credentials such as account, wallet, user if the user has an IAN account.
Adding a loading indicator and logging the amount of time elapsed since the request was sent to indicate some kind of progress state to the user.
zod and its schemas for data validation.

This required a decently comprehensive prompt.

You're an expert TypeScript developer who knows how to translate JavaScript code to TypeScript with appropriate improvements and modifications.

I have this JS code that sends a HTTP request to an LLM API and reads out the response to the console.

*test.js code pasted here*

I need your expertise and help with this.

Write me an analogous, full-fledged client for TypeScript keeping in mind to include the following features

Typings, interfaces, error handling, and schemas for DTOs.

Abstract out the request into a class that can be instantiated to create multiple request objects.

Typings and options for a1, a2, a3, i.e., parameters to specify which models to be used for the agent network when authenticated, with options being string ids from the list ["GPT-4o", "GPT-4o-mini", "GPT-3.5", "Perplexity", "Gemini 1.5 Pro", "Gemini 1.5 Flash", "Gemini Pro", "Llama3.1-405b", "Llama3.1-70b", "Llama3-70b", "Qwen2-72B", "Mistral-8x22b-instruct", "Claude 3 Haiku", "Claude 3 Opus", "Claude 3 Sonnet", "Claude 3.5", "none"] obtained from a website.

Typings and options to specify credentials such as account, wallet, user if the user has an account.

Adding a loading indicator and logging the amount of time elapsed since the request was sent to indicate some kind of progress state to the user.

zod and its schemas for data validation.

You can split your code across multiple files if necessary, and feel free to import any libraries you think may be required.

Remember to write a comprehensive solution and give me each individual source file so that I can use it in my project.

IAN responded with an equally comprehensive response.

Aggregated TypeScript Client Implementation

Consistent Information

All sources consistently mention the creation of a TypeScript client for making HTTP requests, using zod for data validation, and handling errors appropriately. The structure of the implementation involves separate files for types and interfaces, the HTTP client class, and the main execution script.

Implementation Steps

1. Install Dependencies

npm install typescript zod node-fetch @types/node-fetch

2. Create TypeScript Configuration

{
  "compilerOptions": {
    "target": "ES6",
    "module": "commonjs",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  }
}

3. Define Types, Interfaces, and Schemas

// models.ts (aliases: dtos.ts, types.ts)
import { z } from 'zod';

export const modelOptions = [
  "GPT-4o",
  "GPT-4o-mini",
  "GPT-3.5",
  "Perplexity",
  "Gemini 1.5 Pro",
  "Gemini 1.5 Flash",
  "Gemini Pro",
  "Llama3.1-405b",
  "Llama3.1-70b",
  "Llama3-70b",
  "Qwen2-72B",
  "Mistral-8x22b-instruct",
  "Claude 3 Haiku",
  "Claude 3 Opus",
  "Claude 3 Sonnet",
  "Claude 3.5",
  "none"
] as const;

export type ModelOption = typeof modelOptions[number];

export interface PostIANPromptRequest {
  account: string;
  wallet: string;
  user: string;
  a1: ModelOption;
  a2: ModelOption;
  a3: ModelOption;
  input: string;
}

export const postIANPromptRequestSchema = z.object({
  account: z.string(),
  wallet: z.string(),
  user: z.string(),
  a1: z.enum(modelOptions),
  a2: z.enum(modelOptions),
  a3: z.enum(modelOptions),
  input: z.string()
});

export const IANResponseSchema = z.object({
  response: z.string(),
  elapsedTime: z.number(),
});

export type IANResponse = z.infer;

4. Create HTTP Client for Requests

// HTTPClient.ts (aliases: request.ts, IANClient.ts)
import fetch from 'node-fetch';
import { PostIANPromptRequest, postIANPromptRequestSchema, IANResponse, IANResponseSchema } from './models';
import { z } from 'zod';

export class HTTPClient {
  private url: string;

  constructor(url: string) {
    this.url = url;
  }

  async postIANPrompt(request: PostIANPromptRequest): Promise {
    const validation = postIANPromptRequestSchema.safeParse(request);
    if (!validation.success) {
      throw new Error('Invalid input data');
    }

    const startTime = Date.now();
    console.log('Sending request to IAN...');

    try {
      const response = await fetch(this.url, {
        method: 'POST',
        headers: {
          'Accept': '*/*',
          'Accept-Language': 'en-US,en;q=0.5',
          'Content-Type': 'application/json',
          'Sec-Fetch-Dest': 'empty',
          'Sec-Fetch-Mode': 'cors',
          'Sec-Fetch-Site': 'cross-site',
        },
        body: JSON.stringify(request),
        credentials: 'omit',
        mode: 'cors',
      });

      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`);
      }

      const rawResponse = await response.text();
      const elapsedTime = Date.now() - startTime;

      const validatedResponse = IANResponseSchema.parse({
        response: rawResponse,
        elapsedTime,
      });

      console.log(`Request completed in ${elapsedTime}ms`);
      return validatedResponse;
    } catch (error) {
      console.error('Error in IAN request:', error);
      throw error;
    }
  }
}

5. Main Execution Script

// index.ts (aliases: main.ts)
import { HTTPClient } from './HTTPClient';
import { PostIANPromptRequest } from './models';

async function main() {
  const url = 'https://oraloom.com/prompt';
  const client = new HTTPClient(url);

  const request: PostIANPromptRequest = {
    account: 'guest',
    wallet: '1',
    user: 'guest',
    a1: 'GPT-4o',
    a2: 'Perplexity',
    a3: 'Claude 3.5',
    input: 'Can you write me a HTTP GET request in C++?'
  };

  try {
    const result = await client.postIANPrompt(request);
    console.log(result);
  } catch (error) {
    console.error('Error:', error);
  }
}

main();

Run the Application

npm run build
npm start

Inconsistencies and Informational Differences

Inconsistency in HTTP Libraries: GPT-4o and Claude 3.5 use node-fetch while Perplexity uses axios.

Inconsistency in Typings: Perplexity and Claude 3.5 define the types file as dtos.ts and types.ts respectively, whereas GPT-4o names it models.ts.
Agent Strengths and Weaknesses
GPT-4o

    Strength: Provides a well-structured modular approach with clear definitions and explanations.
    Weakness: Uses node-fetch which might require additional polyfills in Node.js environments.

Perplexity

    Strength: Uses axios which is more feature-rich and often preferred for its simplicity in handling HTTP requests.
    Weakness: Does not align with node-fetch used by other sources, leading to inconsistency.

Claude 3.5

    Strength: Well-defined schemas and interfaces with clear separation of concerns.
    Weakness: Similar to GPT-4o, uses node-fetch which might require polyfills and additional configurations.

Well, this looks good. IAN seems to have included something for all the features we specified. Let's install the dependencies, create the necessary files, and test this code out to see if it works out of the box again.

I created src/types.ts, src/client.ts, and src/main.ts, pasted the respective code snippets into them, and stitched up any imports that were missing between the three files.

As soon as I pasted the code into the files, ESLint pointed out a couple of bugs IAN made that I had to fix by hand. Specifically,

IAN didn't specify the z.infer generic's concrete type which I deducted from the context of the rest of the code to be typeof IANResponseSchema.
```
// BUG
export type IANResponse = z.infer;
// FIX
export type IANResponse = z.infer<typeof IANResponseSchema>;
```

IAN didn't specify the Promise generic's concrete type which I deducted from the context of the rest of the code to be IANResponse

// BUG
async postIANPrompt(request: PostIANPromptRequest): Promise {
// FIX
async postIANPrompt(request: PostIANPromptRequest): Promise<IANResponse> {

So I guess no zero-shot luck this time, but that's okay. It got most of it right and the bugs only required minor fixes. I guess even LLMs struggle with type systems lol.

Let's run this TypeScript client implementation with bun.

$ bun run src/main.ts

We initially get a Sending request to IAN... message.

Then, the response is displayed in the console, which now also contains an elapsedTime property representing the number of milliseconds it took to complete the request.

Wrapping up and what's next

In my opinion, everything came together much faster than expected. Without IAN's assistance, it would have taken much longer to get to this point with all the time spent looking up documentation, stiching together code snippets, and what not.

I tested this client a bit more by specifying different combinations of agents and even adding my own user credentials to see if that works.

Finally, I refactored the project to store the progress inside an ianos/clients/js-ts folder, did my source control due dilligence, and pushed the repository to GitHub, which you can find at bkataru/ianos.

Next up in the series, let's continue to reverse engineer IAN with IAN and build ianOS along the way by creating:

Python, Dart, Go, and Rust clients.
a React-based responsive and performant web UI.
a Python-based LLM agent framework with capabilities such as tool-calling and chain of thought.
a Flutter mobile app to interact with IAN on Android/iOS.
a Go CLI binary that works with the local filesystem in order to read in files as context before prompting IAN.
a Rust desktop application using Dioxus.

It's 2024 so why not setup Neovim (w/ NvChad) for Haskell

Baalateja Kataru — Sat, 13 Jul 2024 03:07:51 +0000

Introduction

Recently, I've been spending time on a persistent penboot installation of Fedora Workstation on a 32 GB SanDisk drive.

I've avoided installing any GUI text/code editors such as VSCode (my defacto choice) or Sublime Text, and decided to stick to a basic terminal-and-vim based workflow keeping in mind the limited storage space I have to operate with.

I wanted to divide my usage between the two most popular vim editors out there by

Using Vim for learning from tutorials, blogs, and basic books
Using Neovim for projects and advanced books

Why? Vim's lack of features means it's more or less just a text editor with syntax highlighting. This makes writing code tedious, but tends to sharpen my command of what I'm learning. Whenever I find myself going back to basics (learning a language/framework/toolchain, following a book on some fundamentals), the minimalism of Vim has me paying more attention to what I'm typing onto the screen and be more careful of what I'm doing since I don't have a linter/LSP to catch me from falling if I make a mistake. Needless to say, I didn't configure my vim installation even a little as a result.

But when I'm working on projects or going through technically heavy books, I prefer to bring along my entire belt of gadgets - LSPs, linters, formatters, debuggers, what have you, so that I can maximize my developer velocity and streamline the DX as much as possible. This allows me to focus more on what I'm learning conceptually and offload the menial and tedious to as much tooling as I can. Neovim's rich and modern featureset and extensibility with its Lua-based plugin system makes it perfect for this task, while also allowing me to use my familiar vim motions to move about.

I've been playing with Neovim and trying to configure it for Haskell. I just started going through Learn Physics with Functional Programming - A Hands-on Guide to Exploring Physics with Haskell
by Scott N. Walck so I figured this was the perfect opportunity to figure out how well Neovim and Haskell play together. After much surfing of the world wide web, I couldn't find a straightforward, up-to-date guide to setup Neovim for Haskell in 2024, so I decided I'd make one myself and document the process along the way.

Install GHCup

If you're new to Haskell, the de-facto toolchain is based around:

GHC - Glasgow Haskell Compiler, the go-to standard compiler
Cabal - Haskell build system
Stack - Similar to and sometimes an alternative of Cabal
HLS - The Haskell Language Server, Haskell's implementation of the Language Server Protocol for IDEs/code editors that speak the Language Server Protocol (such as Neovim).

This stack is managed coherently by GHCup, the de-facto toolchain manager for Haskell.

If you don't have GHCup installed on your system, first follow their well elucidated installation guide since you'll need to ensure some system dependencies are available for GHCup to operate correctly.

Since I'm on Fedora Workstation, I'll use dnf to install the system dependencies listed in System Requirements that are specific to my OS and architecture.

$ sudo dnf update -y
$ sudo dnf install -y gcc gcc-c++ gmp gmp-devel make ncurses ncurses-compat-libs xz perl

Next, run GHCup's installation script for your platform as given in How to install. I'll run the Linux one in bash using curl

$ curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh

Next, make sure to choose all defaults, which will

Install all recommended versions of the toolchain components and set them as defaults. Read Which versions get installed? to understand the options and differences.
NOT install HLS (Haskell Language Server), since we will be installing it separately using Mason via NvChad, which uses GHCup underneath the hood to install HLS anyway.
Setup Stack to play well with GHCup's GHC version.

After installation, GHCup and the defaults will be placed in PATH and available to further shell sessions. Either restart your current shell session or type the following in Bash to source the changes into your current shell

$ . $HOME/.ghcup/env

You can open ghcup tui to make sure that all the recommended versions are installed and set as defaults

Note that,

✅ means installed

✅ ✅ means installed and set as default

Install Neovim

If you don't have Neovim, the quickest way to get it is via your system package manager

Otherwise, alternate methods are listed on the same page, which contains the official installation instructions

Install NvChad

NvChad is a very opinionated but very convenient Neovim configuration to have you hit the road running with Neovim.

The quickest way to install NvChad is to clone a starter NvChad config and use :MasonInstallAll to install all necessary plugins right after, as given in quickstart/install. Visit this guide if you have issues installing NvChad, as it does require some system dependencies to be installed and for you to have a Nerd Font installed and selected as your terminal's font.

Remember to backup any preexisting Neovim config before you download NvChad as your Neovim config by running
$ mv ~/.config/nvim ~/.config/nvim-old first

Overrides and Configs

Next, we need to use Mason to install HLS for language support and Haskell's Tree Sitter configuration for syntax highlighting support

We specify this by overriding their respective Neovim plugins

~/.config/nvim/lua/plugins/init.lua

return {
  -- ...
  -- ...
  {
    "williamboman/mason.nvim",
    opts = {
      ensure_installed = {
        "haskell-language-server"
      },
    },
  },

  {
    "neovim/nvim-lspconfig",
    config = function()
      require("nvchad.configs.lspconfig").defaults()
      require "configs.lspconfig"
    end,
  },

  {
    "nvim-treesitter/nvim-treesitter",
    opts = {
      ensure_installed = {
        "haskell"
      },
    },
  },
}

and configure Neovim to talk LSP with our Haskell language server.

~/.config/nvim/lua/configs/lspconfig.lua

local on_attach = require("nvchad.configs.lspconfig").on_attach
local on_init = require("nvchad.configs.lspconfig").on_init
local capabilities = require("nvchad.configs.lspconfig").capabilities

local lspconfig = require "lspconfig"

-- ...
-- ...

lspconfig.hls.setup {
  on_attach = on_attach,
  on_init = on_init,
  capabilities = capabilities,
  filetypes = { 'haskell', 'lhaskell', 'cabal'},
}

That's it! We're done.

Now, open a Haskell file using nvim

$ nvim main.hs

Behold, your Neovim is now a powerful wielder of the wizardry that is Haskell and Functional Programming

Full disclaimer: I wrote this mainly for my own reference because I hate forgetting build/config steps. Feel free to leave any feedback and/or suggestions in the comments below!

Exploring Numeric Data Types in Rust and Go

Baalateja Kataru — Tue, 23 Apr 2024 21:20:32 +0000

Introduction

The comprehensive yet strict static type systems provided by Rust and Go, modern compiled systems programming languages that are closer to the metal than other dynamically typed scripting languages such as Python or JavaScript, allow one to deterministically declare sized variables as well as unsized, architecture dependent variables.

The privilege of a garbage collector (GC) in Go means that there is more flexibility when it comes to manipulating types and values. Rust, being a statically-typed language that has no GC but instead relies on its Ownership model to ensure memory safety, is comparatively more strict at enforcing variable assignments and type conversions when multiple types are involved. The consequence of this difference is that storing the maximum/minimum values of certain numeric type in Rust sometimes requires a variable with a type having more storage capacity (eg. storing the minimum value of the i32 type requires an i64-typed variable)

In this post, we will explore the primitive numeric data types present in both languages while simultaneously drawing a comparison of the similarities and the differences.

Integers

Integers are numeric values without a fractional part. Examples include 24, -2423, 1250682, etc...

Sized Integers

Both Rust and Go have sized integer types: integer data types that are stored with a fixed memory value which in turn determines their ranges when it comes to the possible integer values that can be stored in them.

Unsigned, Sized Integers

Unsigned integers hold non-negative integer values, i.e., values that can only be 0 or positive integers.

Given an unsigned integer type that stores n bits in memory, the minimum and maximum values that can be stored are 0 and (2^n) - 1 respectively.

The various unsigned data types are:

Unsigned, 8-bit integer 0 to 255
- Rust: u8
- Go: uint8
- Minimum: 0
- Maximum: 2^8 - 1 = 255
Unsigned, 16-bit integer 0 to 65535
- Rust: u16
- Go: uint16
- Minimum: 0
- Maximum: 2^16 - 1 = 65535
Unsigned, 32-bit integer 0 to 4294967295
- Rust: u32
- Go: uint32
- Minimum: 0
- Maximum: 2^32 - 1 = 4294967295
Unsigned, 64-bit integer 0 to 18446744073709551615
- Rust: u64
- Go: uint64
- Minimum: 0
- Maximum: 2^64 - 1 = 18446744073709551615
Unsigned, 128-bit integer 0 to 340282366920938463463374607431768211455
- Rust: u128
- Go: no equivalent primitive type
- Minimum: 0
- Maximum: 2^128 - 1 = 340282366920938463463374607431768211455

Signed, Sized Integers

Signed integers can hold both positive and negative integer values, but effectively only have half the storage capacity of their unsigned counterparts as the allocated memory is split between the positive and negative halfs. In terms of bits of memory, this can be seen as the signed integer having one less bit of storage capacity, since the sign bit takes up one extra bit.

Given a signed integer type that stores n bits in memory, the minimum and maximum values that can be stored are -2^(n - 1) and 2^(n - 1) - 1 respectively.

The various signed data types are:

Signed, 8-bit integer -128 to 127
- Rust: i8
- Go: int8
- Minimum: -2^7 = -128
- Maximum: 2^7 - 1 = 127
Signed, 16-bit integer -32768 to 32767
- Rust: i16
- Go: int16
- Minimum: -2^15 = -32768
- Maximum: 2^15 - 1 = 32767
Signed, 32-bit integer -2147483648 to 2147483647
- Rust: i32
- Go: int32
- Minimum: -2^31 = -2147483648
- Maximum: 2^31 - 1 = 2147483647
Signed, 64-bit integer -9223372036854775808 to 9223372036854775807
- Rust: i64
- Go: int64
- Minimum: -2^63 = -9223372036854775808
- Maximum: 2^63 - 1 = 9223372036854775807
Signed, 128-bit integer -170141183460469231731687303715884105728 to 170141183460469231731687303715884105727
- Rust: i128
- Go: no equivalent primitive type
- Minimum: -2^127 = -170141183460469231731687303715884105728
- Maximum: 2^127 - 1 = 170141183460469231731687303715884105727

Unsized Integers

The unsized integer type is used when the size of the integer is not known or not important. The actual size is determined based on the architecture of the underlying machine running the program, which is true for both Rust and Go programs.

If the system is a 32-bit architecture, the appropriate 32-bit integer type (u32/i32 in Rust, uint32/int32 in Go) is used.

If the system is a 64-bit architecture, however, the appropriate 64-bit integer type (u64/i64 in Rust, uint64/int64 in Go) is used.

Unsigned, unsized integer 0 to 4294967295 or 0 to 18446744073709551615
- Rust: usize
- Go: uint
- Minimum: 0
- Maximum: 2^32 - 1 = 4294967295 or 2^64 - 1 = 18446744073709551615
Signed, unsized integer -2147483648 to 2147483647 or -9223372036854775808 to 9223372036854775807
- Rust: isize
- Go: int
- Minimum: -2^31 = -2147483648 or -2^63 = -9223372036854775808
- Maximum: 2^31 - 1 = 2147483647 or 2^63 - 1 = 9223372036854775807

Floating Points

Floating point numbers are used to represent real numbers, i.e., numbers with a fractional component. Examples include 3.1415926, 500.672492, and -28402.75927. All floating point types are signed by default, and there is no architecture-dependent unsized variant either.

32-bit floating point -3.4028235e+38 to 3.4028235e+38
- Rust: f32
- Go: float32
- Minimum: -3.4028235e+38
- Smallest, Positive value: 1.1754944e-38 in Rust, 1e-45 in Go
- Maximum: 3.4028235e+38
64-bit floating point -1.7976931348623157e+308 to 1.7976931348623157e+308
- Rust: f64
- Go: float64
- Minimum: -1.7976931348623157e+308
- Smallest, Positive value: 2.2250738585072014e-308 in Rust, 5e-324 in Go
- Maximum: 1.7976931348623157e+308

Implementations

Here are Rust and Go programs that try to compute the minimum and maximum values of each numeric type described above and test them against the actual values which are declared as constants in each language's respective standard libraries.

Rust

use ibig::{ibig, ubig, IBig, UBig};

fn main() {
    {
        println!("- unsigned, unsized integer -");
        let min: usize = 0;
        let max: usize = usize::MAX;
        println!("usize min: {min}");
        println!("usize max: {max}");
    }

    {
        println!("- unsigned, 8-bit integer -");
        // computations
        let min: u8 = 0;
        let max: u16 = 2_u16.pow(8) - 1;
        // tests
        assert_eq!(min, u8::MIN);
        assert_eq!(max, u8::MAX.into());
        // values
        println!("u8 min: {min}");
        println!("u8 max: {max}");
    }

    {
        println!("- unsigned, 16-bit integer -");
        // computations
        let min: u16 = 0;
        let max: u32 = 2_u32.pow(16) - 1;
        // tests
        assert_eq!(min, u16::MIN);
        assert_eq!(max, u16::MAX.into());
        // values
        println!("u16 min: {min}");
        println!("u16 max: {max}");
    }

    {
        println!("- unsigned, 32-bit integer -");
        // computations
        let min: u32 = 0;
        let max: u64 = 2_u64.pow(32) - 1;
        // tests
        assert_eq!(min, u32::MIN);
        assert_eq!(max, u32::MAX.into());
        // values
        println!("u32 min: {min}");
        println!("u32 max: {max}");
    }

    {
        println!("- unsigned, 64-bit integer -");
        // computations
        let min: u64 = 0;
        let max: u128 = 2_u128.pow(64) - 1;
        // tests
        assert_eq!(min, u64::MIN);
        assert_eq!(max, u64::MAX.into());
        // values
        println!("u64 min: {min}");
        println!("u64 max: {max}");
    }

    {
        println!("- unsigned, 128-bit integer -");
        // computations
        let min: u128 = 0;
        let max: UBig = ubig!(2).pow(128) - 1;
        // tests
        assert_eq!(min, u128::MIN);
        assert_eq!(max, u128::MAX.into());
        // values
        println!("u128 min: {min}");
        println!("u128 max: {max}");
    }

    println!();

    {
        println!("- signed, unsized integer -");
        let min: isize = isize::MIN;
        let max: isize = isize::MAX;
        println!("isize min: {min}");
        println!("isize max: {max}");
    }

    {
        println!("- signed, 8-bit integer -");
        // computations
        let min: i16 = -(2_i16.pow(7));
        let max: u8 = 2_u8.pow(7) - 1;
        // tests
        assert_eq!(min, i8::MIN.into());
        assert_eq!(max, i8::MAX.try_into().unwrap());
        // values
        println!("i8 min: {min}");
        println!("i8 max: {max}");
    }

    {
        println!("- signed, 16-bit integer -");
        // computations
        let min: i32 = -(2_i32.pow(15));
        let max: u16 = 2_u16.pow(15) - 1;
        // tests
        assert_eq!(min, i16::MIN.into());
        assert_eq!(max, i16::MAX.try_into().unwrap());
        // values
        println!("i16 min: {min}");
        println!("i16 max: {max}");
    }

    {
        println!("- signed, 32-bit integer -");
        // computations
        let min: i64 = -(2_i64.pow(31));
        let max: u32 = 2_u32.pow(31) - 1;
        // tests
        assert_eq!(min, i32::MIN.into());
        assert_eq!(max, i32::MAX.try_into().unwrap());
        // values
        println!("i32 min: {min}");
        println!("i32 max: {max}");
    }

    {
        println!("- signed, 64-bit integer -");
        // computations
        let min: i128 = -(2_i128.pow(63));
        let max: u128 = 2_u128.pow(63) - 1;
        // tests
        assert_eq!(min, i64::MIN.into());
        assert_eq!(max, i64::MAX.try_into().unwrap());
        // values
        println!("i64 min: {min}");
        println!("i64 max: {max}");
    }

    {
        println!("- signed, 128-bit integer -");
        // computations
        let min: IBig = -ibig!(2).pow(127);
        let max: u128 = 2_u128.pow(127) - 1;
        // tests
        assert_eq!(min, i128::MIN.into());
        assert_eq!(max, i128::MAX.try_into().unwrap());
        // values
        println!("i128 min: {min}");
        println!("i128 max: {max}");
    }

    println!();

    {
        println!("- 32-bit floating point -");
        let min: f32 = f32::MIN;
        let smallest: f32 = f32::MIN_POSITIVE;
        let max: f32 = f32::MAX;
        println!("f32 min: {min}");
        println!("f32 smallest: {smallest}");
        println!("f32 max: {max}");
    }

    {
        println!("- 64-bit floating point -");
        let min: f64 = f64::MIN;
        let smallest: f64 = f64::MIN_POSITIVE;
        let max: f64 = f64::MAX;
        println!("f64 min: {min}");
        println!("f64 smallest: {smallest}");
        println!("f64 max: {max}");
    }
}

Running this via cargo run produces:

- unsigned, unsized integer -
usize min: 0
usize max: 18446744073709551615
- unsigned, 8-bit integer -
u8 min: 0
u8 max: 255
- unsigned, 16-bit integer -
u16 min: 0
u16 max: 65535
- unsigned, 32-bit integer -
u32 min: 0
u32 max: 4294967295
- unsigned, 64-bit integer -
u64 min: 0
u64 max: 18446744073709551615
- unsigned, 128-bit integer -
u128 min: 0
u128 max: 340282366920938463463374607431768211455

- signed, unsized integer -
isize min: -9223372036854775808
isize max: 9223372036854775807
- signed, 8-bit integer -
i8 min: -128
i8 max: 127
- signed, 16-bit integer -
i16 min: -32768
i16 max: 32767
- signed, 32-bit integer -
i32 min: -2147483648
i32 max: 2147483647
- signed, 64-bit integer -
i64 min: -9223372036854775808
i64 max: 9223372036854775807
- signed, 128-bit integer -
i128 min: -170141183460469231731687303715884105728
i128 max: 170141183460469231731687303715884105727

- 32-bit floating point -
f32 min: -340282350000000000000000000000000000000
f32 smallest: 0.000000000000000000000000000000000000011754944
f32 max: 340282350000000000000000000000000000000
- 64-bit floating point -
f64 min: -179769313486231570000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
f64 smallest: 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072014
f64 max: 179769313486231570000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Note that we make use of an external crate ibig to help store and manage big integer computations, especially when trying to compute the minimum and maximum values of the largest data types of each kind of signed and unsigned integer. Make sure to add this dependency to your Cargo.toml so that Rust can pull the crate and its dependencies during cargo run

[dependencies]
ibig = "0.3.6"

Go

package main

import (
    "fmt"
    "math"
    "testing"

    "gotest.tools/v3/assert"
)

func main() {
    var t = &testing.T{}

    func() {
        fmt.Println("- unsigned, unsized integer -")
        var min uint = 0
        var max uint = math.MaxUint
        fmt.Printf("uint min: %d\n", min)
        fmt.Printf("uint max: %d\n", max)
    }()

    func() {
        fmt.Println("- unsigned, 8-bit integer -")
        // computations
        var min uint8 = 0
        var max uint8 = 1<<8 - 1
        // tests
        assert.Assert(t, min == 0)
        assert.Assert(t, max == math.MaxUint8)
        // values
        fmt.Printf("uint8 min: %d\n", min)
        fmt.Printf("uint8 max: %d\n", max)
    }()

    func() {
        fmt.Println("- unsigned, 16-bit integer -")
        // computations
        var min uint16 = 0
        var max uint16 = 1<<16 - 1
        // tests
        assert.Assert(t, min == 0)
        assert.Assert(t, max == math.MaxUint16)
        // values
        fmt.Printf("uint16 min: %d\n", min)
        fmt.Printf("uint16 max: %d\n", max)
    }()

    func() {
        fmt.Println("- unsigned, 32-bit integer -")
        // computations
        var min uint32 = 0
        var max uint32 = 1<<32 - 1
        // tests
        assert.Assert(t, min == 0)
        assert.Assert(t, max == math.MaxUint32)
        // values
        fmt.Printf("uint32 min: %d\n", min)
        fmt.Printf("uint32 max: %d\n", max)
    }()

    func() {
        fmt.Println("- unsigned, 64-bit integer -")
        // computations
        var min uint64 = 0
        var max uint64 = 1<<64 - 1
        // tests
        assert.Assert(t, min == 0)
        assert.Assert(t, max == math.MaxUint64)
        // values
        fmt.Printf("uint64 min: %d\n", min)
        fmt.Printf("uint64 max: %d\n", max)
    }()

    fmt.Println();

    func() {
        fmt.Println("- signed, unsized integer -");
        var min int = math.MinInt
        var max int = math.MaxInt
        fmt.Printf("int min: %d\n", min)
        fmt.Printf("int max: %d\n", max)
    }()

    func() {
        fmt.Println("- signed, 8-bit integer -")
        // computations
        var min int8 = -(1 << 7)
        var max int8 = (1 << 7) - 1
        // tests
        assert.Assert(t, min == math.MinInt8)
        assert.Assert(t, max == math.MaxInt8)
        // values
        fmt.Printf("int8 min: %d\n", min)
        fmt.Printf("int8 max: %d\n", max)
    }()

    func() {
        fmt.Println("- signed, 16-bit integer -")
        // computations
        var min int16 = -(1 << 15)
        var max int16 = (1 << 15) - 1
        // tests
        assert.Assert(t, min == math.MinInt16)
        assert.Assert(t, max == math.MaxInt16)
        // values
        fmt.Printf("int16 min: %d\n", min)
        fmt.Printf("int16 max: %d\n", max)
    }()

    func() {
        fmt.Println("- signed, 32-bit integer -")
        // computations
        var min int32 = -(1 << 31)
        var max int32 = (1 << 31) - 1
        // tests
        assert.Assert(t, min == math.MinInt32)
        assert.Assert(t, max == math.MaxInt32)
        // values
        fmt.Printf("int32 min: %d\n", min)
        fmt.Printf("int32 max: %d\n", max)
    }()

    func() {
        fmt.Println("- signed, 64-bit integer -")
        // computations
        var min int64 = -(1 << 63)
        var max int64 = (1 << 63) - 1
        // tests
        assert.Assert(t, min == math.MinInt64)
        assert.Assert(t, max == math.MaxInt64)
        // values
        fmt.Printf("int64 min: %d\n", min)
        fmt.Printf("int64 max: %d\n", max)
    }()

    fmt.Println();

    func() {
        fmt.Println("- 32-bit floating point -")
        var smallest float32 = math.SmallestNonzeroFloat32
        var max float32 = math.MaxFloat32
        fmt.Printf("float32 min: %g\n", -max)
        fmt.Printf("float32 smallest: %g\n", smallest)
        fmt.Printf("float32 max: %g\n", max)
    }()

    func() {
        fmt.Println("- 64-bit floating point -")
        var smallest float64 = math.SmallestNonzeroFloat64
        var max float64 = math.MaxFloat64
        fmt.Printf("float64 min: %g\n", -max)
        fmt.Printf("float64 smallest: %g\n", smallest)
        fmt.Printf("float64 max: %g\n", max)
    }()

}

Running this via go run outputs

- unsigned, unsized integer -
uint min: 0
uint max: 18446744073709551615
- unsigned, 8-bit integer -
uint8 min: 0
uint8 max: 255
- unsigned, 16-bit integer -
uint16 min: 0
uint16 max: 65535
- unsigned, 32-bit integer -
uint32 min: 0
uint32 max: 4294967295
- unsigned, 64-bit integer -
uint64 min: 0
uint64 max: 18446744073709551615

- signed, unsized integer -
int min: -9223372036854775808
int max: 9223372036854775807
- signed, 8-bit integer -
int8 min: -128
int8 max: 127
- signed, 16-bit integer -
int16 min: -32768
int16 max: 32767
- signed, 32-bit integer -
int32 min: -2147483648
int32 max: 2147483647
- signed, 64-bit integer -
int64 min: -9223372036854775808
int64 max: 9223372036854775807

- 32-bit floating point -
float32 min: -3.4028235e+38
float32 smallest: 1e-45
float32 max: 3.4028235e+38
- 64-bit floating point -
float64 min: -1.7976931348623157e+308
float64 smallest: 5e-324
float64 max: 1.7976931348623157e+308

Go does not have a primitive assert statement for asserting expressions like in Rust, but the third party gotest.tools/v3/assert module implements similar behaviour by augmenting the default testing module provided in Go's standard library. Remember to hit go mod tidy to pull this dependency before running this program.

Go also does not have a block construct like in Rust to declare arbitrary scopes in a program, but similar functionality can be achieved in Go by using Immediately Invoked Function Expressions (IIFEs) as seen above.

A Useful Note About Type Inference

Both Go and Rust have type inference capabilities, which allow them to infer the type of a variable when it is not explicitly annotated. Go infers all integer valued variables to be of the unsized type int, which can correspond to int32 or int64 depending on architecture, while Rust infers all such variables to be of type i32. On the contrary, all floating point valued variables in Go as well as Rust are inferred to be of type float64/f64 respectively. The Rust Book justifies this choice over the lower precision but lesser memory f32 type as follows

The default type is f64 because on modern CPUs, it’s roughly the same speed as f32 but is capable of more precision.

-- Data Types (Chapter 3, Section 3.2)

Afterthoughts

This post was mainly for personal reference and exploration because I tend to keep forgetting the intricacies of numeric data types in Rust and Go, especially the maximum and minimum values of each data type. A lack of awareness of these ranges can cause overflow errors in both languages, something the Rust compiler helpfully points out when running in debug mode.

I hope this helps others as well who want a quick reference to numeric data types in Rust and Go and how to compute their maximum and minimum values in both languages. Feel free to leave comments suggesting additions or improvements that you feel are relevant to this discussion.