Publishing the core

After many weeks of hemming and hawing, going to and fro, I decided it's time to put some actual code out into the world.

A lot of code has been written, lots of experiments and ideas, notes, scribbles, thoughts, questions, and hundreds of millions of tokens (226M apparently!) worth of conversation with Claude. It's been fun and educational to explore topics in depth, reflect on all I've learned over the last many years, all the struggles, and be reminded of how little I know still. Claude has been a great partner on this journey, even though I rewrote almost all of the code it produced (I'll write more about that experience at the end of this post).

After all that, I needed to climb out of the rabbit hole, create an empty directory, and pull out the pieces that seemed reasonable.

Here's what's published:

• a core Go module at atlas9.dev/c/core

• non-core modules at:

Browse the source at atlas9.dev/src and clone the source with git clone https://git.atlas9.dev/atlas9.git.

There's also a demo app (src) that pulls the pieces together to provide a simple app with login, registration, passwords, user profiles, and OIDC login with Google and Apple. I'll be expanding this demo app as I add more features to the atlas9 modules.

Core

Let's look at some parts of the core.

There's an ID type:

id := core.NewID()
println(id.String()) // 04JE0MSNG09VXY4ZRMCMZF244G
core.ID{}.IsZero() == true

The ID type implements various interfaces that make it easy to (un)marshal from json, text, sql databases, etc. I wrote about designing this ID previously here.

There's a Clock interface, which exists to remind us to make time.Now() testable:

type Clock interface {
    Now() time.Time
}

There are a couple small types related to pagination, which I hope helps keep pagination consistent, and reminds us to implement pagination from the start. This might move into a generic "crud" helper in the future.

type Page[T any] struct {
    Items []T
    Cursor string
}

type PageReq struct {
    Limit int
    Cursor string
}

Path, Guard, and access control

The Path type represents the path to a resource. Most applications have some hierarchy of resources – you might have an "organization", and org can have multiple "team" containers. So if the "dev" team in the "acme" org stores a document, that might be represented as a (Path, ID) tuple like ("acme.dev", "04JE0MSNG09VXY4ZRMCMZF244G"). Or maybe you have a deeper hierarchy like "acme.eng.dev.contractors". A path is a string in a dot-separated format that describes the location of a resource in the application's hierarchy. Paths are used for determining access – you might grant Bob access to all resources in the "acme.eng.dev" path, for example.

Path isn't used in the demo app yet and needs more development.

core/iam.Guard is an interface that represents an access check.

type Guard interface {
    Check(context.Context, Action, core.Path) error
}

// the context would hold information about the current principal (user)
ctx := context.Background() 

// a real application would implement the guard interface for its access control model,
// or use an implementation provided by atlas9.
g := iam.AllowNone{}

// the Action type denotes that this string is an action used in an access control check.
act := iam.Action("example action")

// guard checks operate on paths – access is granted based on paths.
p := core.Path("acme.eng.dev")

g.Check(ctx, act, p) == iam.ErrForbidden

The ideal convention would be to prefer doing guard checks at lower levels, in the Store implementations, close to where the database is being accessed for example, to help ensure that requests don't accidentally gain access to resources they shouldn't have access to via references.

type DocStore struct {
  db dbi.DBI
  guard iam.Guard
}

var ActionGetDoc iam.Action = "get doc"

func (ds *DocStore) GetDoc(ctx context.Context, id core.ID) (*Doc, error) {
  var doc Doc
  err := dbi.Get(ctx, ds.db, &doc, `SELECT ... FROM documents ...`, id)
  if err != nil {
    return nil, err
  }
  if err := ds.guard.Check(ctx, ActionGetDoc, doc.Path); err != nil {
    return nil, err
  }
  return &doc, nil
}

Path, Guard, and friends need more thought and work. Access control is such a broad, complex topic, and applications have a wide variety of needs, that I've had a very hard time landing on something I feel good about. I don't love the design yet.

Guard, at the very least, will need to support list/query filtering, so it might get a Filter(ctx, action, path) Filters method in the future.

I'm certain that whatever I come up with won't work for some use cases, and that's ok, it should be easy to ignore and leave unused. I hope to find something that works nicely for common application designs though.

Users, Principals, Sessions, Passwords, OIDC

The core/iam package has types and interfaces for common identity concepts like users (represents a human user with a login, email, etc), principal (the identity of a caller, could be a user, a machine, etc), and sessions. It has some helpers for creating and verifying passwords.

Many applications have a "sign in with Google/Apple/Github/etc" feature, so core/iam provides an interface for storing identity providers for users, and the iam/oidc_provider module provides implementations for common identity providers.

In general, I've tried to keep the core light – core defines the concepts, the types and interfaces, and mostly leaves concrete implementations and their dependencies to other modules.

Routes

core/routes contains my preferred approach to building APIs – RPC instead of REST. My RPC design uses POST requests for everything, with no information encoded into URLs. I like the consistency and constraint of RPC compared to REST, so that I don't have to think about which request method to use, what information to put into the URL or query parameters, etc.

type Route struct {
    Pattern string
    Handler http.Handler
    ReqType any
    ResType any
}

// RPC endpoint
func RPC[T, U any](path string, handler RpcHandler[T, U]) Route

// Standard HTTP handler
func HTTP(pattern string, handler http.HandlerFunc) Route

// Static files
func Static(mux *http.ServeMux, filesys fs.FS, pattern, filepath string)

Go generics and routes.RPC allow me to write an API endpoint more like a standard function, with the request and response types clearly visible in the function signature. routes.Route provides routes as data – I have some unpublished work that can turn these routes into OpenAPI specs, generate clients, terraform, etc.

Unfortunately, the demo app doesn't demonstrate routes yet, because the app uses forms and url-encoded bodies instead of javascript and RPCs. So lots more work to do in this area. I'll write more about this in the future.

Database interface (DBI)

core/dbi grew out of the need to start read-write and read-only transactions.

userID, err := dbi.ReadOnly(ctx, p.DB, func(tx dbi.DBI) (core.ID, error) {
    users := p.Users(tx)
    passwords := p.Passwords(tx)

    user, err := users.GetByEmail(ctx, email)
    ...
})

And it will likely include a few helpers for common patterns, for example:

func (s *SqlitePasswordHashStore) GetHash(ctx context.Context, userID core.ID) (iam.PasswordHash, error) {
    var hash iam.PasswordHash
    err := dbi.Get(ctx, s.db, &hash, `SELECT hash FROM password_hashes WHERE user_id = $1`, userID)
    return hash, err
}

dbi.Get handles the common dance of running a query, scanning the result into a struct, handling errors, etc.

One design decision here worth pointing out is that "Stores" (the code that contains the database access) are created per-transaction by convention:

dbi.ReadOnly(ctx, p.DB, func(tx dbi.DBI) (core.ID, error) {
    users := p.Users(tx)
    passwords := p.Passwords(tx)
})

This allows multiple orthogonal application domains to be used within one transaction, which helps keep the store interfaces and implementations small and focused. The tradeoff is a small amount of code and a "factory" concept for creating the store instances.

This also fits well with Guard or any other request-scoped information that a store would need.

On hosting code

Github is great, but I wanted to do something different this time. It's not simpler or easier or better (arguably). It took much more effort to figure out how to set up, the file browsing is bare bones, etc.

It does feel somewhat empowering to have everything on my server, under my domain. I had fun setting it all up. I learned a lot. It (barely) provides what I need right now and no more. I don't need Actions, Discussions, Issues, Pull Requests, Releases, Insights, etc. It's nice to get back to the roots of the internet – have a server, put files up, configure things, own it.

Perhaps it's just that, I can.

Why do you want to climb Mt Everest? Because it's there. – George Mallory

On building with Claude

There's a lot of talk and turmoil about the role of LLMs in software (and everything else). There are those that love it, those that hate it, those that worry about it. So I thought I'd share my experience.

Claude has been a great partner on this atlas9 journey, but not because it's good at writing code. It's actually written mostly code that I was unhappy with, which is somewhat surprising to me. I've rewritten most code and docs Claude has written. Aside from code, I've also had long conversations where I needed to refine the initial plan proposed by Claude. If I had just "vibe coded" it and let Claude go off and do whatever it came up with first, I expect things would be quite messy.

Claude often seems to overdo it. It will add bits and pieces that are unnecessary or were not discussed, but that may be common in software in general. The result doesn't feel refined. I'm sure this is, in part, due to how I'm using Claude. It's great for some code though – the Pages code in the demo is a good example of a few hundred lines of tedious code I'd rather not type out (pulling form values, accessing storage, handling errors, passing back to render, etc), and that doesn't need to be particularly refined.

Claude is fantastic for learning new things. I have spent many hours deeply learning about postgres, access control, etc. The thought of learning those things by browsing the internet, with its messy pages and bad writing, sifting through SEO and clickbait garbage, dealing with popups and cookie prompts...ugh. It makes me cringe. Claude can almost instantly answer challenging questions I have on complex topics in a direct and concise way, without all the noise.

Claude is, for me, best for working through ideas. I can write a few dozen words about an idea I have, and then I can iterate on that idea for hours. It can be really hard past an initial idea sometimes – like a writer getting past the blank page. Having Claude sketch out some detail allows me to work with something less abstract, figure out what feels good and bad, reject parts and refine it. I've also loved asking Claude, "what else?" – after hours of pondering an idea, when my brain is drained and I'm deep down the rabbit hole and I have tunnel vision, Claude can give fresh perspective and summarize any gaps or tradeoffs.

Maybe LLMs will have another leap in the future and they'll write better code and design better applications or architectures than I could with no guidance from me, but so far that hasn't been my experience. They still need a driver. Perhaps software engineers have a perspective that is the distillation of years of experience, and perhaps LLMs haven't quite reached that level of distilled knowledge yet. They probably will though, which is crazy and scary and exciting.

Conclusion

I'm happy to finally get some real code published. Atlas9 feels slightly more like a real thing now.

And if you actually read all this and made it to the end here, thank you! If you have thoughts or want to chat, email me at [e-mail].

Until next time!