DEV Community

Cover image for I Built a Dataflow Engine Where the Graph Is the Program
mac172
mac172

Posted on

I Built a Dataflow Engine Where the Graph Is the Program

Header Image

I’m Building a Low-Level Dataflow Environment for Experimenting With Computation

What if you could construct a computational system, change one of its parameters, and immediately see how that change propagates through the system?

That's the idea I'm exploring with Computational World (CW) also called Simwire.

CW is an experimental browser-based environment built around a low-level, topological dataflow substrate.

It isn't intended to be another block-based programming tool or a collection of visual effects.

The goal is much simpler:

Make computation visible, composable, and directly manipulable.


The problem I'm exploring

A lot of visual programming and simulation environments make experimentation easier by hiding the underlying computation.

You might get a node called:

Particle System
Enter fullscreen mode Exit fullscreen mode

or:

Sine Wave
Enter fullscreen mode Exit fullscreen mode

or:

Physics Simulation
Enter fullscreen mode Exit fullscreen mode

That's convenient, but the interesting computational structure is now inside the box.

Traditional programming gives you the opposite tradeoff.

You can see and control the computation, but the interaction loop is usually:

write code
   ↓
run
   ↓
inspect output
   ↓
change code
   ↓
run again
Enter fullscreen mode Exit fullscreen mode

I wanted to explore something in between.

What happens if the computational structure itself becomes the interface?


Computation as a graph

The basic model in CW is a directed graph.

Nodes represent operations.

Edges represent dependencies and data flow.

For example:

Flow Showcase

Input
  │
  ▼
Multiply
  │
  ▼
Add
  │
  ▼
Output
Enter fullscreen mode Exit fullscreen mode

This isn't just a diagram representing a program.

The graph is the program.

If the output depends on Add, and Add depends on Multiply, those dependencies exist in the actual execution model.

This makes the structure something that can be inspected and manipulated directly.


Topological execution

One of the first problems I had to solve was execution order.

Consider:

A ──→ C
B ──→ C
Enter fullscreen mode Exit fullscreen mode

C can't execute until both A and B have produced their outputs.

With a larger graph, manually deciding execution order obviously doesn't scale.

CW therefore treats the graph as a dependency graph and resolves a valid execution order using topological sorting.

The current implementation uses Kahn's algorithm.

Conceptually:

Find nodes with no unresolved dependencies
            ↓
Execute them
            ↓
Remove their outgoing dependencies
            ↓
Find newly available nodes
            ↓
Repeat
Enter fullscreen mode Exit fullscreen mode

This gives the runtime a deterministic way to propagate computation through the graph.

It also gives me a useful property:

the visual structure and the execution structure remain closely related.


Why primitives?

Another design decision I'm experimenting with is keeping operations relatively primitive.

Instead of immediately creating high-level nodes for every interesting behavior, I want complex systems to be constructed from smaller operations.

For example, instead of treating:

Sine Wave
Enter fullscreen mode Exit fullscreen mode

as a magical black box, I'd rather explore whether useful mathematical behavior can emerge from smaller primitives.

The same idea could eventually apply to:

  • mathematical models
  • physics systems
  • signal processing
  • algorithms
  • data transformations
  • computer architecture

The important question isn't whether this is always more convenient.

It isn't.

The question is whether composition itself can become a useful way of understanding a system.


Live manipulation

This is probably the part of CW that matters most to me.

A computational system shouldn't necessarily require a full edit-run-inspect cycle for every small experiment.

If I change a parameter on the canvas, I want the change to propagate through the graph immediately.

For example:

input = 5

      ↓

multiply × 2

      ↓

output = 10
Enter fullscreen mode Exit fullscreen mode

Change the input:

input = 20

      ↓

multiply × 2

      ↓

output = 40
Enter fullscreen mode Exit fullscreen mode

The important thing isn't the arithmetic.

It's the interaction.

You can change something and immediately observe what the rest of the system does.

That turns the computational graph into an experimental surface.


From computation to visualization

I'm also exploring the relationship between computation and visual output.

The values flowing through the graph don't have to terminate at a text output.

They can drive geometry and interactive visual streams.

Visual Showcase

Conceptually:

Computation
     ↓
Data
     ↓
Geometry / Visual Output
Enter fullscreen mode Exit fullscreen mode

This makes it possible to construct systems where changing the computational model directly changes what you see.

That's where simulations become particularly interesting.

A mathematical relationship can become motion.

A signal can become geometry.

A parameter can become an interactive control.

The computation becomes something you can observe rather than something that only exists inside source code.


Why build this in the browser?

There are practical reasons.

The browser gives me a relatively accessible environment for experimentation:

  • no installation for users
  • immediate visual feedback
  • interactive canvas
  • easy sharing
  • computational output can be rendered directly

More importantly, it lets the computational system and the interface exist in the same environment.

I can change the graph and immediately see the consequences.


What I'm trying to find out

I don't actually know yet whether this interaction model is better.

That's the experiment.

There are several questions I'm currently trying to answer.

How low-level should the primitives be?

Too high-level and the system becomes another collection of black boxes.

Too low-level and constructing anything becomes tedious.

There has to be a useful middle ground.

When does composition become confusing?

A graph is easy to understand when it has ten nodes.

What happens when it has hundreds?

At some point, visibility becomes complexity.

That's a problem I haven't solved.

Can this actually help people understand computation?

This is perhaps the biggest question.

Making something visual doesn't automatically make it understandable.

The visualization has to expose meaningful structure rather than simply decorate it.


What I want to explore next

I'm interested in pushing the system into increasingly different computational domains.

Mathematics

Construct mathematical relationships and manipulate their parameters interactively.

Physics

Build simple physical systems and observe how their behavior changes.

Signal processing

Represent transformations as dataflow pipelines and visualize signals moving through them.

Algorithms

Explore whether algorithmic processes can become easier to reason about when their execution is directly observable.

Computer architecture

Eventually, I'd like to experiment with representing lower-level concepts such as logic, memory, and computation itself.

That last one is still far away, but it's one of the reasons I'm interested in keeping the underlying system primitive and compositional.


This is still an experiment

CW is nowhere near finished.

I'm not trying to present it as a finished product.

I'm trying to find out whether this way of interacting with computation is actually useful.

The direction I'm exploring is:

construct
    ↓
connect
    ↓
change
    ↓
observe
    ↓
understand
Enter fullscreen mode Exit fullscreen mode

rather than:

write
    ↓
compile
    ↓
run
    ↓
inspect
Enter fullscreen mode Exit fullscreen mode

Maybe this interaction model works.

Maybe it breaks down as systems become more complex.

Maybe the right answer is somewhere between visual composition and traditional code.

That's what I'm trying to discover by building it.

If you're interested in dataflow systems, programming languages, simulation, creative coding, mathematics, or computational education, I'd be interested in hearing where you think this approach breaks — or where you think it could become useful.

Computational World is being built in public.

The current prototype is available here:

Simwire Sandbox

Top comments (2)

Collapse
 
raknaos profile image
Raknaos

The question "when does composition become confusing" is the one every node-graph runtime eventually hits, and I think Kahn's algorithm quietly gives you a gift here before the scaling problem arrives: when the ready-queue drains with nodes still unprocessed, those leftovers are exactly your cycle. That matters the moment you want state — a counter, a feedback loop, anything where a node's output feeds its own next tick. Pure topological order forbids it, so the honest options are breaking the cycle with an explicit delay node or falling back to full re-evaluation.

Collapse
 
mac172 profile image
mac172

This is a really good observation and thank you for the feedback. Looking at the current runtime, I’ve already taken the explicit-delay approach for temporal feedback: delay, counter, and accumulator act as stateful boundaries, so intentional feedback can cross simulation ticks without becoming an instantaneous cycle.

The interesting part is that my top-level Kahn implementation currently detects the situation implicitly but doesn't expose the leftover nodes as a cycle diagnostic yet. I currently append unresolved nodes after the queue drains, which means that part needs tightening.

So I think you're pointing at an important distinction for the runtime: intentional feedback should cross an explicit state/time boundary, while accidental instantaneous cycles should be rejected and diagnosed.

That's probably something I'll make more explicit in the execution model.