How I made Go feel interpreted with Go plugins

Interactive programming environments have become an important part of modern development. REPLs, notebooks, and live coding tools all share the same idea: code is written and executed incrementally, while the running program keeps its state. Jupyter Notebook has become the reference environment for this workflow in data science. It allows developers to experiment, prototype, and gradually shape ideas into working code, one cell at a time.

Building a notebook kernel for a compiled language is challenging because compilation and execution are traditionally separated. A program is compiled, started, executed, and eventually terminated. A notebook breaks this model: it expects a process that stays alive while new pieces of code are continuously added.

When building kernels for a compiled language like Go, there are two main approaches, each representing a different trade-off.

The first is interpretation (gopherdata/gophernotes): by evaluating the language dynamically, it provides a natural notebook experience, but at the cost of moving away from the language's execution model and sacrificing much of what makes a compiled language attractive.

The second is compilation (janpfeifer/gonb): by keeping the language fully compiled, it preserves native execution, but each new cell effectively becomes a new program. Incremental development is therefore constrained, since state must either be rebuilt or reconstructed between executions.

I wanted a different trade-off.

---

What if Go had already provided the building block?

What if Go had already (without really intending to) provided exactly the primitive needed to build an interactive notebook? The plugin package is Go's way of loading shared objects (.so) at runtime—essentially the same class of mechanism that C provides through dlopen.

The original motivation behind Go plugins was not interactive programming. The goal was mainly to produce Go-based shared libraries that could be consumed by other environments. The original proposal discussed use cases such as CPython extensions, and later projects like gomobile naturally built on the same idea. I suspect that the ability to consume these plugins from Go itself was, at least partly, a natural completion of the model rather than the result of a specific killer use case.

Yet, ten years later, this capability turned out to be exactly what I needed. My original goal was to build a notebook kernel for Go: a notebook needs a running process that can accept new code while preserving everything already in memory. Go plugins provide precisely that. The same execution model later naturally extended to a REPL.

---

How does it work?

There are really two separate problems to solve:

• How to execute notebook cells.
• How to share state between them.

Executing cells

The first problem is solved by Go plugins. Each notebook cell is compiled into a shared object (.so) and loaded into the host process using plugin.Open. The host then looks up a well-known Execute entry point (despite its deceptively simple name) using plugin.Lookup and invokes it.

One important limitation is that Go plugins cannot be unloaded. Every distinct cell therefore remains mapped for the lifetime of the process. In practice, this represents roughly 1–1.4 MB of RSS per unique cell, and unchanged cells are never rebuilt thanks to a content-based cache. This unavoidable memory leak (let's call it what it is) is the main drawback of the approach.

Sharing state

Sharing state is more subtle than it first appears. Although every plugin executes inside the same process and therefore shares the same Go heap, plugins are still independently compiled units. Nothing allows one plugin to directly recover a variable declared by another.

Several mechanisms bridge this gap:

• Every plugin receives a shared runtime.Registry through the ctx argument passed to Execute.
• Types and functions declared by previous cells are preserved as source code and injected before compiling every subsequent plugin.
• To determine whether an identifier introduces a new declaration or references an existing symbol, gocell relies on go/types, more specifically its Defs and Uses maps.

Variables are handled differently. Every referenced symbol is hydrated exactly once at the beginning of Execute:

namePtr := (*T)(ctx.GetPointer("name"))

The generated source is then rewritten so that every occurrence of name becomes (*namePtr). The lookup therefore happens once—not every time the variable is accessed. Since identifier resolution comes from go/types, shadowed variables (inside an if, for, or any nested scope) are handled correctly.

Imports are resolved in two stages. The session tracks every explicit import declared across previous cells. The generated source is then passed through the real goimports, which automatically adds missing imports and removes unused ones. As a result, a cell can simply write math.Sqrt(x) without ever declaring import "math".

Finally, panics require special handling. A panic reports a line number in the generated Go source—not in the notebook cell the user actually wrote. Between the two, gocell has injected variable hydration, reconstructed declarations, interrupt checks and other runtime scaffolding. The runtime therefore maps generated line numbers back to the user's original cell before reporting the error.

Here is what this looks like in practice, across two cells:

• Cell 1: message := "Hello, world!"
• Cell 2: fmt.Println(message)

Cell 1 as implemented by gocell:

package main

import (
    "fmt"
    "reflect"
    "strings"
    "unsafe"

    "github.com/alexispires/gocell/pkg/runtime"
)

// Anti-unused import guards
var (
    _ = fmt.Sprintf
    _ = unsafe.Pointer(nil)
    _ = strings.HasPrefix
    _ = reflect.ValueOf
)

var __gocell_ctx *runtime.Context

// --- Plugin Execute entry point ---
func Execute(ctx *runtime.Context) error {
    __gocell_ctx = ctx
    // Cell statements (used symbols already rewritten to go through their _ptr)
    message := "Hello, world!"

    // Export new symbols to the Registry (direct pointer vs heap copy)
    ptr_message := unsafe.Pointer(&message)
    keepAlive_message := &message
    ctx.SetPointer("message", fmt.Sprintf("%T", &message), ptr_message, keepAlive_message)

    return nil
}

Cell 2 as implemented by gocell:

package main

import (
    "fmt"
    "reflect"
    "strings"
    "unsafe"

    "github.com/alexispires/gocell/pkg/runtime"
)

// Anti-unused import guards
var (
    _ = fmt.Sprintf
    _ = unsafe.Pointer(nil)
    _ = strings.HasPrefix
    _ = reflect.ValueOf
)

var __gocell_ctx *runtime.Context

// --- Plugin Execute entry point ---
func Execute(ctx *runtime.Context) error {
    __gocell_ctx = ctx
    // Point directly at existing symbols -- no copy, no write-back
    message_ptr := (*string)(ctx.GetPointer("message"))

    // Cell statements (used symbols already rewritten to go through their _ptr)
    fmt.Println(*message_ptr)

    return nil
}

---

So what does gocell actually enable?

gocell, an open-source project available on GitHub, was built around three core promises.

1. It is complete

A notebook should feel like one long-running Go program. With gocell:

• goroutines survive across cells;
• variables preserve their identity;
• user-defined types and functions remain available;
• generics work exactly as they do in Go.

This makes a significant difference in real data-science workflows: a model trained in one cell can immediately be evaluated, refined or reused in another without ever rebuilding its execution state.

2. It is fast

Interactive notebooks should not come at the cost of performance. Because every cell is compiled to native Go code, gocell avoids interpreter overhead while preserving a persistent execution state.

On the bundled machine-learning example (heavy-model.ipynb, 8 million samples, 140 epochs), gocell is roughly 1.5× faster than GoNB without its cache, and about 20× faster than gophernotes. When compared to GoNB using gonb/cache, both are essentially tied.

Heavy-model (8M samples, 140 epochs)	Step 1 (fit)	Step 2	Step 3	Total
gocell	2816 ms	647 ms	679 ms	4142 ms
GoNB (no cache)	2620 ms	1968 ms (refit)	1597 ms (refit)	6185 ms
GoNB (gonb/cache)	3129 ms	584 ms	563 ms	4276 ms
gophernotes	81 991 ms	1.6 ms	1.5 ms	81 994 ms

The difference with GoNB is architectural rather than computational. Both execute native Go code. GoNB reconstructs state between cells through serialization, whereas gocell simply keeps a single live Go heap for the lifetime of the notebook.

The trade-off appears during startup. Since every new cell must be compiled into a plugin, gocell pays a higher cold-start cost.

Cold start (fresh process → ready → first fmt.Println("hello world"))	Run 1	Run 2
gocell	753 + 1693 = 2446 ms	228 + 803 = 1031 ms
GoNB	412 + 974 = 1386 ms	264 + 548 = 811 ms
gophernotes	698 + 110 = 807 ms	456 + 107 = 563 ms

Once execution begins, however, gocell delivers native performance while preserving a persistent runtime.

3. It is idiomatic

Perhaps most importantly, gocell does not reinterpret Go. There is no new syntax, no hidden semantics, and no custom language extensions. It is built almost entirely on top of Go's own standard packages:

• go/ast
• go/types
• plugin

The goal was never to change Go—it was simply to make Go feel interactive.

Finally, gocell is actually two applications sharing the exact same execution engine: one is a standalone REPL, and the other is a Jupyter kernel. The notebook was the original motivation behind the project, but both frontends are powered by the same core implementation.

---

Conclusion

I'm not claiming that gocell is perfect (even if, of course, I'd like to think it is). Compiling plugins introduces a noticeable cold-start cost, which can affect the developer experience. Memory usage also grows gradually as new cells are executed since plugins cannot be unloaded. Finally, the plugin package is not available on Windows—although, in practice, many Go developers on Windows already work through WSL.

So what's next? First and foremost is testing: the architecture is in place, but making it as reliable and robust as possible is the top priority. The second goal is reducing the impact of cold starts. Finally, although the Jupyter kernel was the original motivation behind the project, I'd also like to continue improving the standalone REPL—both frontends share the same execution engine, and making one better almost always benefits the other.