Inside TypeScript-go: Concrete AST Nodes, Shared State, and Tree Traversal

TypeScript 7 is a version of TypeScript whose compiler and editor tools were reimplemented in Go. This article examines one part of that implementation: its AST node model.

The intended reader works with Go or TypeScript but does not need compiler-internals experience. The topic is a general modeling problem: how to represent several related data shapes without putting every possible field into one oversized struct full of optional fields.

Source and license note

This article discusses the publicly available microsoft/typescript-go source repository, which is licensed under the Apache License 2.0. The implementation section includes short source excerpts; the remaining Go and TypeScript snippets are small, independently written examples. Links to the relevant upstream files are included where implementation details are discussed.

Source excerpts are from Microsoft’s TypeScript-go project:
https://github.com/microsoft/typescript-go

Licensed under the Apache License, Version 2.0.
A copy of the License is available at:
https://github.com/microsoft/typescript-go/blob/main/LICENSE

Step 0: compiler, transpiler, parser, and type checker

A compiler pipeline normally starts with a parser, which turns source text into an abstract syntax tree (AST). A type checker uses that tree to analyze types, and an emitter writes JavaScript or another output format. A compiler that emits another programming language is often called a transpiler; TypeScript-to-JavaScript is one example. This article focuses only on the AST as a data model, not on parsing or type-checking algorithms.

Step 1: a parser needs more than the original text

Suppose a compiler receives this code:

total(1, 2)

The parser represents this expression as an AST: a tree of nodes that capture source structure.

Call
├── Identifier: total
├── Number: 1
└── Number: 2

Here, Call, Identifier, and Number are node kinds: labels for distinct syntax roles. Later stages read this tree instead of re-interpreting source text.

Each kind needs different information.

• An identifier needs text, such as total.
• A number needs its value or original text.
• A call needs the expression being called and its arguments.

Some information is useful for every kind. For example, the compiler often needs the location of a node in the original file so it can show an error in the right place. It may also need a link to the node's parent.

This gives us the real design problem: how should one Go type represent what every node has, while still keeping call-only fields away from identifier-only fields?

Step 2: the simple approach is one large struct

A normal first implementation in Go is one struct. For a TypeScript reader, it is similar to defining one object shape with several optional properties.

type Node struct {
    Kind      Kind      // Says whether this is a call, identifier, or number.
    Start     int       // Byte position where this node begins in the source text.
    End       int       // Byte position immediately after the node ends.
    Parent    *Node     // The containing node, if there is one.
    Text      string    // Used by an identifier or number; unused by a call.
    Callee    *Node     // Used by a call; unused by an identifier or number.
    Arguments []*Node   // Used by a call; unused by an identifier or number.
}

This is not wrong. It is easy to create, easy to read, and often perfect for a small format.

The difficulty appears when the model grows. With this type, Go allows combinations that have no meaning. For example, an identifier can have both Text and Arguments, even though only a call should have arguments. The struct cannot express the rule itself; every part of the program must remember it.

if node.Kind == IdentifierKind {
    // `Text` is meaningful only after checking the kind.
    fmt.Println(node.Text)
}

One check is harmless. A compiler has many node kinds and many places that read them. Repeating the same checks makes it easier for one reader to forget a rule. Adding a new kind also tends to add more fields that are empty for nearly every existing node.

Step 3: TypeScript can describe valid shapes with a union

TypeScript has a discriminated union for this case. A shared tag, usually kind, connects a runtime value with the fields allowed for that shape. For a Go reader, it is a compile-time relationship between a tag value and the fields valid with it.

type IdentifierNode = {
  kind: "identifier"; // This fixed value identifies an identifier node.
  start: number; // Source position shared by every node.
  end: number;
  text: string; // Only an identifier has a name.
};

type CallNode = {
  kind: "call"; // This fixed value identifies a call node.
  start: number;
  end: number;
  callee: Node; // The expression to invoke.
  arguments: Node[]; // The values passed to that expression.
};

// A Node is one valid shape or the other, not an arbitrary mixture of both.
type Node = IdentifierNode | CallNode;

After checking kind, TypeScript narrows the union: it knows which fields are safe to use in that branch. This is checked when TypeScript is compiled, before the JavaScript runs.

function describe(node: Node): string {
  if (node.kind === "identifier") {
    // In this branch, TypeScript knows that `text` exists.
    return node.text;
  }

  // The only remaining shape is CallNode, so `arguments` exists here.
  return `call with ${node.arguments.length} arguments`;
}

The important benefit is not the syntax. It is that the type model says which combinations are valid.

Step 4: Go needs a different way to express the same separation

Go does not have built-in discriminated unions. TypeScript-go combines a shared struct header with a private interface. A Go interface can hold a concrete value that provides its methods, and a type assertion checks which concrete value it currently holds.

The TypeScript-go project uses those features to split its AST in two parts. The small example below is deliberately simpler than the real layout: its callData is not a TypeScript-go type.

• A central Node holds data shared by all nodes.
• A private nodeData value holds data that belongs to one specific node kind.

In the actual source, the shared Node has a kind, flags, a source range, an internal ID, a parent pointer, and private nodeData. The concrete nodeData values hold details such as identifier text, call arguments, and function bodies.

Here is a smaller example of the same idea. It is written for explanation, not copied from TypeScript-go.

type Node struct {
    Kind   Kind     // Shared: tells generic code which syntax form this is.
    Start  int      // Shared: start position in the input.
    End    int      // Shared: end position in the input.
    Parent *Node    // Shared: link to the containing node.
    // data is not a child node itself. It stores fields specific to this kind.
    // For a call expression, its concrete data can conceptually contain:
    //   Callee    *Node   // the expression being called
    //   Arguments []*Node // the expressions passed to the call
    data nodeData
}

// Visitor is the callback used while walking direct children.
// Returning true asks the traversal to stop early.
type Visitor func(*Node) bool

type nodeData interface {
    // Every kind-specific data type exposes its direct children.
    ForEachChild(Visitor) bool
}

type identifierData struct {
    Text string // Specific: only identifiers have identifier text.
}

func (identifierData) ForEachChild(visitor Visitor) bool {
    return false // An identifier is a leaf, so there is nothing to visit.
}

type callData struct {
    Callee    *Node   // Specific: the expression to call.
    Arguments []*Node // Specific: values supplied to the call.
}

func visit(visitor Visitor, node *Node) bool {
    if node != nil {
        return visitor(node) // Call the visitor only for a real child node.
    }
    return false // A missing optional child cannot stop the traversal.
}

func visitNodes(visitor Visitor, nodes []*Node) bool {
    for _, node := range nodes {
        if visitor(node) {
            return true // Stop as soon as the visitor asks to stop.
        }
    }
    return false
}

func (d callData) ForEachChild(visitor Visitor) bool {
    if visit(visitor, d.Callee) {
        return true // Do not inspect arguments after an early-stop request.
    }
    return visitNodes(visitor, d.Arguments) // Visit arguments in source order.
}

The real code starts from a concrete node object. CallExpression contains its child fields, and its factory obtains one from an arena before filling those fields. newNode obtains the embedded shared Node with data.AsNode(), sets its kind, and stores the same concrete object back in n.data. Therefore, for a call expression, n.data holds *CallExpression, not a separate callData value. See the actual CallExpression factory and the shared newNode function.

The traversal contract in the example is still useful, but the actual top-level dispatch has one important difference. Node.ForEachChild uses a generated switch on Kind, then type-asserts n.data and calls the concrete method:

func (n *Node) ForEachChild(v Visitor) bool {
    switch n.Kind {
    // ... other node kinds ...
    case KindCallExpression:
        return n.data.(*CallExpression).ForEachChild(v)
    }
    return false
}

CallExpression.ForEachChild then passes its direct Expression, optional QuestionDotToken, optional type arguments, and arguments to the visitor. In the upstream source, Visitor is func(*Node) bool; visit ignores a nil child and otherwise calls the supplied visitor; visitNodes applies that visitor to a list until it returns true. The true return is an early-stop signal, not an error. See the helpers and the concrete CallExpression traversal.

ForEachChild itself does not recursively walk the whole tree. It enumerates only direct children. The visitor supplied by its caller decides what to do with each child, including whether to recurse. A search visitor can use the early-stop result like this:

func Find(node *Node, matches func(*Node) bool) *Node {
    var found *Node
    var walk Visitor

    walk = func(current *Node) bool {
        if matches(current) {
            found = current
            return true // Tell every caller in the chain to stop.
        }
        return current.ForEachChild(walk) // Recurse only after testing this node.
    }

    if node != nil {
        walk(node) // Begin the depth-first traversal at the root.
    }
    return found
}

Here, walk is the actual visitor implementation. It checks the current node, recursively asks that node for its direct children, and returns true once it finds a match. Language-service features use the same separation: their visitor performs feature-specific work, while each AST node knows which fields are its children.

The word private matters here. In Go, a lower-case name such as data or nodeData cannot be used by code in another package. This serves the same purpose as keeping an implementation detail unexported from a TypeScript module.

Generic code can work with *Node and use the common fields. For example, a tree walker can report source locations for every node. When it needs to move down the tree, generated dispatch selects the concrete node and that node enumerates its children.

Step 5: the small example keeps Kind and data together

A constructor is a function that creates a value in a valid state. In Go, New... factory functions commonly play this role.

type Kind uint8

const (
    IdentifierKind Kind = iota // `iota` assigns the first numeric Kind value; it must use identifierData.
    CallKind                   // This kind must use callData.
)

The concrete data types used by the constructors are repeated here so their contents remain visible nearby.

type identifierData struct {
    Text string // The identifier's spelling, such as "total".
}

func (identifierData) ForEachChild(visitor Visitor) bool {
    return false // Identifiers are leaf nodes.
}

type callData struct {
    Callee    *Node   // The expression to call.
    Arguments []*Node // The expressions passed to the call.
}

func (d callData) ForEachChild(visitor Visitor) bool {
    if visit(visitor, d.Callee) {
        return true // Stop before visiting arguments when requested.
    }
    return visitNodes(visitor, d.Arguments)
}

func NewIdentifier(start, end int, text string) *Node {
    // Create an identifier kind with identifier-specific data.
    return &Node{
        Kind:  IdentifierKind,
        Start: start,
        End:   end,
        data:  identifierData{Text: text},
    }
}

func NewCall(start, end int, callee *Node, arguments []*Node) *Node {
    // Create a call kind with call-specific data.
    return &Node{
        Kind:  CallKind,
        Start: start,
        End:   end,
        data:  callData{Callee: callee, Arguments: arguments},
    }
}

The constructor is where the rule lives. Code elsewhere does not need to remember which fields a call requires. It calls NewCall, and that function pairs CallKind with callData.

When code needs information that belongs only to an identifier, it can use a method that checks the concrete data type.

func (n *Node) IdentifierText() (string, bool) {
    data, ok := n.data.(identifierData) // This succeeds only for an identifier.
    if !ok {
        return "", false // A call does not have identifier text.
    }
    return data.Text, true
}

n.data.(identifierData) is Go's type-assertion syntax. In the two-result form, data receives the concrete identifierData value only when the assertion succeeds, and ok reports whether it succeeded. When n.data holds callData, ok is false, so the function can safely return without reading Text. The Go language specification's type-assertion section documents both this safe two-result form and the one-result form that panics on a mismatch. Returning false here is safer than silently returning an empty string for the wrong kind of node.

What TypeScript-go adds beyond this small example

The real compiler needs more than a tiny expression tree. It needs to clone nodes, visit many different child arrangements, store information discovered during type checking, and handle a large number of language features.

TypeScript-go's private nodeData interface therefore has more methods than this example. The project also provides default base types, so an individual node implementation does not have to reimplement every method from scratch. The source comments explain that the team uses both interface methods and type switches: interface calls can help some paths, but hundreds of interface implementations also increase code size. The common Node definition and the private nodeData interface and default base type show those details.

The project is not offering nodeData as a public plugin API. It is an internal representation for a closed set of node kinds that the compiler itself owns. That is useful because the compiler can change traversal or cloning rules and update every known node type at once.

Does this AST design explain TypeScript 7's speed?

No. This design can make the model clearer and reduce opportunities to misuse fields, but it is not a general performance recipe.

The TypeScript team describes TypeScript 7's improvement as a combination of native code speed, shared-memory multithreading, and other optimizations. Parsing, type checking, and JavaScript output generation can run in parallel where that is safe. Type checking is more complicated because files share type information. TypeScript 7 uses a fixed number of type-checker workers so the work stays deterministic; using more workers can help large projects, but it can also use more memory. The official TypeScript 7 announcement explains that trade-off.

When is this pattern useful in an ordinary Go application?

Start with a normal struct when the data is small. A configuration value with two optional fields does not need an interface and several extra types.

This pattern becomes useful when all of the following are true:

• Your program has a fixed set of data shapes with clearly different fields.
• Many parts of the program need a small set of fields shared by every shape.
• Invalid combinations of fields are causing repeated checks or bugs.
• Central constructors and one traversal rule would make the model easier to understand.

The lesson is not “always use interfaces for trees.” It is simpler: keep shared data together, keep kind-specific data with its own kind, and make invalid combinations difficult to create.

Kinmokusei

Kinmokusei is a programming language with TypeScript-inspired syntax that compiles to readable Go. It is intended for writing web backends and Go libraries while using the normal Go toolchain and package ecosystem directly.