DEV Community: Connor Hicks

Choosing building blocks to move faster

Connor Hicks — Mon, 01 Feb 2021 13:31:43 +0000

I am currently running a developer survey to help build the best developer experience for WebAssembly on the server. Please consider taking it!

My open source focus for this year is building Atmo, and there is one aspect of the process that I would like to highlight. Since early 2020 I knew roughly what I wanted to build. The specifics of that thing changed over time, but the core idea of a server-side WebAssembly platform was consistent all throughout the year. I didn't write a single line of code for Atmo until late October, even though that was what I wanted to build the entire time. I want to talk about why.

Building blocks

Atmo is built on three core building blocks. When considering what would be needed to provide the kind of fully-featured platform that I was hoping to build, there were a number of core properties I knew it needed to have. I needed it to be extremely simple to deploy. I needed it to have zero external dependencies. I needed it to work in a decentralized environment. During this process is when I solidified the SUFA design pattern that I wanted Atmo to be a framework for. I worked on designing a solution, and ended up deciding that the project would require three core components.

The simplest part, the part that I knew the best, was the web server. I knew I wanted the platform to be centred around building web services like APIs, and I knew that a rock-solid server framework would be needed to ensure it was secure, fast, and easy to use. I had plenty of experience building web servers, so I wasn't too worried about this part.

Core to the idea of a server-side WebAssembly framework (or any function-based system) is the scheduler. The ability to dynamically control and distribute the execution of tiny modules while ensuring they remained coordinated is key to this project. I had several requirements for the scheduler, including modularity, being lightweight, and having tight control over how workers were managed.

The biggest unknown to me at the outset was how to reliably enable distributed computation, and even more so being able to do so in a decentralized manner. The message bus would end up being so essential to the project that choosing the wrong one would have been a huge detriment. I had an ambitious set of needs, and this was the aspect of planning Atmo I struggled with the most. I needed to sink a fair amount of research into the available options, and ultimately decide what was best for the project.

Each of these three components - the server, the scheduler, and the message bus - needed to be in place for this project to succeed, and I knew I needed to plan carefully to ensure they would all work in harmony. The classic choice in the software industry - build or buy - was coming at me from three sides, and I made a choice for each one.

I chose to build these blocks at the expense of moving fast. What I learned along the way is that this decision has its ups and downs - but it was the right one for this project, and I want to explain why. In the end, taking the time to build these components myself allowed me to build something of a higher quality, and actually allowed me to build Atmo itself more quickly than I could have otherwise - in just one weekend.

Dependencies, integration, and requirements

For each of the three components, the decision to build it myself or use something off the shelf came down to one or more of these three things; dependencies, integration, and requirements.

To start, one of the easiest way to disqualify a potential off-the-shelf component is when it introduces dependencies that you can't or won't use. In the case of choosing a message bus for example, most of the available options had a dependency on CGO (the Go language's framework for integrating C-style libraries). This was a dependency I was unable to accept because it makes cross-compilation and distribution of a library harder. I could have accepted this dependency and faced the consequences down the line, but ultimately I decided it was not worth the trade-off for this project.

The ability to integrate the building blocks together was another main factor that came into play. My design for how Atmo would function would mean a tight integration between the three components, and after some research I found that achieving this would be difficult. The message bus and the job scheduler needed to be able to work very closely with one another, and without being able to change the internal workings of the scheduler in specific ways, none of the available solutions were going to work for me.

Another common reason to decide against using an existing component is when it doesn't satisfy your requirements. For this project, I needed to prioritize the performance of the web server, and so I knew I needed an HTTP framework that performed well across a number of benchmarks, but I also needed some unusual capabilities such as being able to swap out the router in real-time. In the end, I decided on a hybrid approach for the web server, choosing to build a custom framework around the well-loved httprouter, which granted the majority of what I needed, and had some extremely well thought-out integration points that made it easy for me to build on top of it. In the case of the message bus, I also had a requirement of decentralization, which the majority of available projects are not.

The result

In the end, a combination of those factors led me to build the three main components of Atmo rather than use something off the shelf. Choosing to build the web server framework caused me to build Vektor. The need for a highly custom scheduler caused me to build Hive. The need for a dependency-free and decentralized message bus with tight integration with Hive caused me to build Grav. Some people may call this yak shaving, but I believe it was fundamental to building Atmo.

For starters, with the massive amount of infrastructure that these projects provided me with, I was able to build the first prototype of Atmo in one weekend. I'm not going to pretend that it was any good, but it worked, and I was able to immediately diagnose every problem I had with it because I knew the constituent components like the back of my hand. Understanding the internals of the various parts made it extremely easy to understand what was happening under the hood, and to gain control that I wouldn't have if I had chosen something pre-made.

Now that work on Atmo has been underway for a number of months, the dividends I am seeing from having built these components myself is staggering. I am able to finely tune the performance of the system, down to the execution of each function invocation, because I have full control over the internals. I am able to build abstractions at each level of the project that make the most sense in the context, which in turn makes each code base individually easier to understand and more useful.

I now have 4 usable and exciting projects available for others to take advantage of, and I am able to help them use each part to its full extent because I understand how they work internally. I don't want the message of this post to be 'you should build all of your project's infrastructure from scratch', because that's not true. For developers building any product, it's important to understand the real value that the build or buy choice means for your users or customers. If the end result to them is absolutely no difference, then why would you waste time building something when you can use something dedicated?

For Atmo and Suborbital as a whole, the goal is to build tools and frameworks to help developers build web services. The tagline is 'Helping you build web services that are powerful, but never complicated', and I believe that building these components meaningfully improves the quality of what I'm providing to them. I think it's important to always make these choices in the context of what you're building, and don't choose one path or another because of what I chose, or because of what you'd chosen for some past project. Make an informed decision each time and be sure to always consider what the end user gets out of it.

If you want to learn more about Suborbital, Atmo, Vektor, Hive, and Grav, head over to the Suborbital website, and sign up for the mailing list for occasional updates. You can reach out to me on Twitter or reach out via the Suborbital discussion forum

Cover photo by Steven Wei from Unsplash

P.S. If you look closely at the Atmo logo, you should be able to see the three components represented!

Building for a future based on WebAssembly

Connor Hicks — Thu, 21 Jan 2021 13:48:21 +0000

My goal with the Suborbital project is contributing to the WebAssembly community in the form of tools, frameworks, and a platform that makes building web services with WebAssembly useful in the real world. I believe that WebAssembly is a technology that will enable simpler, safer, and more flexible server applications. In order to acheive this, I want to outline a clear plan for the project, and I also need the help of the community to help shape the vision of what's being built.

I have a lot to say about WebAssembly, and I've spent the last year working on building server-side capabilities for it, so I wanted to share some of those thoughts here in one place. We have seen some great adoption of WebAssembly, including support in the major browsers, the evolution of WASI, more language toolchain support, and real-world use-cases such as Shopify and The Internet Archive.

I've laid out some of the design patterns that Atmo is being built around, and now I want to talk about some of the core aspects that make it compelling as a development platform. The first is declarative application development.

Declarative development

Just as Kubernetes popularized declarative infrastructure, Atmo strives to introduce declarative application logic. When developing an Atmo application, you write isolated, independent functions called Runnables that are compiled to WebAssembly. Since these functions are completely unaware of one another, there needs to be a way to string them together in order to create business logic. Atmo does this using a file called the Directive. Here's an example:

identifier: com.suborbital.test
version: v0.0.1

handlers:
  - type: request
    resource: /user
    method: GET
    steps:
      - group:
        - fn: modify-url
          as: baseURL
        - fn: get-user
          as: user
      - fn: fetch-details
        with:
          - "url: baseURL"
          - "user: user"

This includes a full description of how to handle a request to the GET /user endpoint. It describes not only what endpoints are available, but how they should be handled. The functions called are completely independent, and are able to carry out the request by interacting with Atmo's APIs.

As seen above, function groups allow multiple Runnables to execute concurrently to aid in fetching data from a large number of sources or performing many different data manipulation tasks simultaneously. Atmo's scheduler automatically handles this execution and coordinates running all of the functions needed for a request. The as and with clauses make it easy to manage state and function input/output.

The end of boilerplate

One of the key goals of Atmo's design is that no boilerplate code is needed. The only things you need to do are write your functions and define your Directive. There's no need to configure a webserver, bind to ports, build a router, etc. Everything is packaged into a Runnable Bundle and Atmo uses it to automatically build your server.

This is important for a number of reasons. Firstly, it is a boon for developer productivity, as creating new projects is vastly simplified. A simple subo create project command creates everything you need to get started. Secondly, it makes onboarding new developers easier as they don't need to reason about how you've set up your particular project. There's one consistent entrypoint and an easy to understand project structure.

Polyglot, for good reason

WebAssembly is a common format that can be leveraged to make application design easier, but it can also simplify vendor APIs as well. A long-term goal for Atmo is to allow Runnables from your own team as well as third parties to coexist in harmony. It shouldn't matter if a vendor wrote a library in Rust, C++, Go or Swift. Atmo will make it easy to chain together third party modules with your own to easily interact with third-party APIs. Vendors would only need to build their API once, rather than distributing libraries in multiple languages. Mocking out for testing would be made easier, as third party modules could be easily swapped for test versions in CI/CD to make testing your system simpler.

Flexible operations

Since the entire Runnable Bundle for an Atmo application is loaded at runtime, plenty of capabilities become available such as hot-reloading business logic, instantaneous rollouts and rollbacks, multiple simultaneous application versions, and more. Making changes will no longer require restarting a process and dropping or draining network connections, as your application can be upgraded in real-time while requests from an older version are still being handled. WebAssembly brings many capabilities like these that were previously difficult or impossible, and it really highlights what is possible.

Meshing to the core

As laid out in the SUFA design pattern, the ability to scale out a flat network of instances is one of the capabilities enabled by Atmo. This is more than just linking instances, it is truly meshing together your application's business logic down to the core scheduler.

Atmo uses Hive and Grav to mesh not only the instances together, but the execution of each request. Capability groups are designed to increase the security of your system, and meshing allows individual functions to be executed on privileged instances, and then immediately transfer execution back to non-sensitive instances, meaning that the exposure to resources is more tightly protected.

Request state

As shown above with the as and where clause, Atmo uses a novel approach to pass data between functions in a request chain. Each step in a handler produces output, and when the step completes, that output is saved to an ephemeral state object. This state is available to every Runnable in later stages, allowing access to the data produced by previous steps. as allows storing a function's output into state under a particular name, and with lets the developer define a 'desired state' to pass into a function, effectively choosing what should be provided as input.

Request state is automatically managed by Atmo, and is created for each request as it's handled. State has been designed to be serialized, such that it can be shuttled over the wire by Grav when request meshing is being used. This means that functions executed on a meshed instance gets access to the same state, no matter where it's running. This flexible data handling technique makes developing asynchronous and distributed compute simple and completely managed by Atmo.

What's next

I've just described a number of unique features that makes Atmo a new and exciting way to develop your web services, but the Suborbital project is still in its early days. Atmo is still in Alpha, and while it is showing great promise, it is not yet ready for production workloads. There is an immense amount of work that needs to be done in order to realize the goal of having a viable and well-adopted server-side WebAssembly platform, and for that I need the help of others. There are many ways you can help, and I welcome all of them, but there are two things in particular that I am asking for today.

Take the survey. I've created a 3-minute survey to learn more about the community's thoughts about WebAssembly on the server so that I can inform decisions that need to be made about developing Suborbital. I would greatly appreciate responses, as it will help ensure that I am building something that developers would be excited to build with.

Take the survey

Give it a try. While not ready for production workloads, trying Atmo and giving me your feedback is the most helpful thing that you can do right now. All of the tooling, infrastructure, architecture, and design patterns are brand new and hearing how you use it (or even better, fail to use it!) helps me to determine what can be improved, added, and fixed to make the project production-ready.

I am also open to any and all contributions from the community. I am more than happy to meet with anyone interested in working alongside me to build these capabilities so that I can help get you started developing Atmo, Vektor, Grav, Hive, and Subo. Developers with no experience working with WebAssembly, distributed systems, web services, or Go are encouraged to join and I will do whatever I can to help you learn what's needed to contribute. Open Source is not just about developing in the open, it's also about helping others learn.

This was a very long post, but I hope it helps communicate what I am working towards with the Suborbital project, and most of all I hope it gets you excited about the future of WebAssembly and how it can shape the next generation of applications in the cloud.

If you want to be kept up to date about Suborbital and WebAssembly on the server, please sign up using the form below or by providing your email address in the survey (which is optional, of course). If you want to get in touch, please feel free to reach out via Twitter or via our GitHub discussions

Cover photo by Nastya Dulhiier from Unsplash

How to familiarize yourself with a new codebase

Connor Hicks — Mon, 28 Dec 2020 19:45:24 +0000

A few weeks ago, a tweet made me take a second and think about something that I'd never consciously considered before; how can you approach an unfamiliar codebase and start to understand it?

Daniel Feldman

@d_feldman

Congratulations! It’s your first day as a senior engineer at a new company. Their code base is thousands of files/millions of lines and there are absolutely no resources to help you get on board, it’s your job to figure out how to contribute. How do you get started?

20:29 PM - 08 Dec 2020

143 2314

It got me thinking about how I would approach a new repo that I'd never seen before but needed to make a contribution against, like a bug fix. I remembered my early days of learning Kubernetes, and wanting to make requests to the its API (because using the command line wasn't good enough for me, apparently). I had been trying to work out how to automatically deploy a particular branch of a GitLab repo into a cluster every time someone pushed to it. I had big ideas about automating DNS, setting up automated certificates, and adding a Slackbot to notify you whenever a new deploy happened.

If I remember correctly, I got a proof of concept working, and then it never went much past that. Given how popular GitOps has become, maybe I should have stuck with it! When I started delving into the Kuberenetes side of the project, I was completely and utterly lost. The documentation didn't have much in the way of how to use the API (I'm sure nowadays things are much better), and reading the Kubernetes source code was a complete non-starter because well, that thing is a monster. I remember thinking to myself that I just needed to replicate what kubectl was doing to create a new Deployment.

So I gave up trying to read Kubernetes' source, and moved over to the source for kubectl. This is where I started to make some headway! I was able to follow straight from the main() function to the apply command, down through the logic until it started making API requests. It felt so good to finally get an answer, and to just import some Go packages to make it all work in short order!*

This is the background behind my answer to the tweet above:

Connor Hicks

@cohix

@d_feldman @richburroughs I always start with entrypoints - HTTP router, application init, etc. Everything flows from there.

20:32 PM - 08 Dec 2020

3 38

Since that project years ago, I've sort of instinctively followed this strategy whenever I need to reason about a new codebase because well, it works! Only recently did this tweet make me think about it concretely, and I'm glad it did. I tried to replicate this purposefully to test my strategy. I went to a large open-source repo and tried to find the code where it installed itself into a cluster. Using this strategy, I started with the tool's main() and then was able to find my way to the install command, which led me down to where the installation happens (funnily enough, by calling kubectl).

I think it's important for any developer to understand not only how to reason about an unfamiliar codebase, but also to realize that an important way that we learn is by trial and error. When we try something and it works, it brings us joy and we'll continue to do it, even if we don't realize it. I think it's a good idea to take a second to think about these moments when they happen, take a mental note of it so that next time you come across a similar problem, you can consciously use your previous learning and expand upon the strategies you've developed over time.

The reason I wanted to turn this tweet into a full blog post is because it made me realize that one of the goals for Atmo is to make it easier for developers to reason about applications. Since Atmo uses a declarative format for building backend applications (using WebAssembly modules), there is always one canonical entrypoint; the Directive. From there, you can easily reason about what the application is doing because it is declarative instead of imperative. This is one of the things that made Kubernetes so popular. Being able to describe your application in a simple format, and then have a system "make it happen" is a magical thing, and Atmo strives to do exactly that.

Atmo is gaining new functionality every week. If you want to learn more, check out the Suborbital homepage

When I say "short order", I'm sure it still took several days to get everything working, but once you unblock yourself on a big problem, everything after that just seems to fly by.

Cover Photo by Rafif Prawira on Unsplash

Meshing a modern monolith

Connor Hicks — Mon, 14 Dec 2020 13:17:36 +0000

In my last post, I described a model for a new type of web service monolith, the SUFA design pattern. A key part of Simple, Unified, Function-based Applications is the fact that the unit of compute is actually individual functions running on a job scheduler. The functions are composed to create the business logic of the application, and a framework is responsible for handling requests and running the functions as described.

One design consideration is ensuring that SUFA systems are able to scale efficiently, and this is accomplished with capability groups and meshing. I said that these topics deserve their own post, so here we are! The purpose of SUFA is to take the best elements of different design patterns to make something that is powerful, but never complicated. The ease and simplicity of building a monolith combined with the scaling characteristics of serverless or microservice systems is the ultimate goal.

Capability groups take full advantage of the decoupled functions within a SUFA system. Each function has some associated metadata, one of which is a namespace. A namespace is a logical grouping of functions, such as auth, cache, billing, db and others. Functions within a namespace are meant to encapsulate a category of logic, and require a common set of resources such as access to a particular data store, or secrets for a third party API. When a SUFA system is deployed, a single deployable is built, and then one or more Auto-Scaling Groups (ASGs) manage a set of instances to handle changes in traffic.

A simple SUFA system with one namespace (such as a simple REST API with a single database) could be easily deployed with a single ASG. In order to scale with increased traffic, a key property of microservice systems is that individual services can scale independently to handle endpoints and functionality that are being used more than others. With a traditional monolith, any increase in traffic requires scaling up instances, no matter which endpoints are getting the majority of the spike.

With SUFA systems, multiple ASGs are created, each designated as a capability group. Each capability group is given access to the resources required for the associated function namespace to operate (such as the datastore or secrets), and can then scale independently of one another. Since the application's functions are decoupled entirely from one another, it's possible for some functions to run on the host that receives the request, and functions from particular namespaces to be meshed into other capability groups. A SUFA framework such as Atmo is responsible for handling the meshed communication, completely absorbing the complexity.

Adopting this model of meshing request handling among multiple capability groups has several advantages. The individual groups can scale independently, allowing you to more flexibly handle increases in traffic without needing to scale up instances to only handle a sliver of functionality. It has desirable security properties such as "edge" instances handling incoming requests but only granting access to sensitive resources to particular capability groups that are not directly available on the public internet.

To give a more concrete example, imagine a blogging platform (like Hashnode where the Suborbital blog is hosted!). When an author requests their blog's analytics, a SUFA system could describe functions such as:

auth#checkSession
db#getBlogDetails
db#selectBlogPosts
analytics#getVewCountForPosts
frontend#renderPage

In this example, the auth namespace needs access to a cache, the db namespace access to a PostgreSQL instance, analytics needs access to a third-party service, and frontend needs access to static object storage. The SUFA system would be deployed with capability groups that grant access to said resources, allowing each to scale as needed when the various parts of the app experience increased load. The SUFA framework would automatically mesh instances in each group together to collaboratively handle the request and schedule the correct functions on an instance in the correct group.

Since all of this behaviour is provided by the SUFA framework, developers still need only concern themselves with writing the functions and defining how to compose them. Developers can even run a single instance in their local development environment, and then production environments can run in capability groups. Atmo runs functions compiled to WebAssembly modules and uses the Grav messaging mesh to provide meshing functionality, which provides the flexibility and simplicity needed to build powerful applications that are never complicated.

Please reach out to me on Twitter or visit Suborbital to learn all about Atmo, Grav, and the other projects that make the modern monolith possible!

Cover photo credit: Zak Podmore/The Salt Lake Tribune

Building a better monolith

Connor Hicks — Tue, 08 Dec 2020 00:58:22 +0000

The monolith gets a bad rap

When you hear that a company runs a monolith, you may think they're old-fashioned and they must have trouble scaling it, right? I'm here to tell you that some people (myself included) think monoliths are awesome for a whole lot of teams. That said, technology has advanced, and I truly think it's time to revisit the monolith with a new approach.

If you're new to back-end development, a monolith is a server-side system that runs as one.... thing. It's a single program that starts up, serves some network requests, and then terminates. Alternatives to 'the monolith' include service-oriented architectures (SOA), microservices, serverless functions, and probably a few that I haven't heard of. Each of these approaches has their time and place, and I salute anyone who has made an educated decision to build a system with these patterns. I'm of the opinion that a large number of web applications and services would be well served by a monolith.... but with some upgrades.

You need some services to lighten the load

The reasoning behind the alternative design patterns is very sound. By distributing the work among many different “things”, you make the system as a whole able to handle more. It is well known that by doing so, you introduce more complexity, which requires more effort, and therefore often more people-power and more money. The one possible exception I can see is serverless functions, which do indeed simplify many things, but whose downsides comes in the form of more difficult testing, vendor dependence, or the need for non-commodity tooling. When done right, the extra effort can lead to a very capable system whose benefits are truly remarkable. The classic example is Netflix, and they’ve done very well for themselves.

Finding a middle ground

If monoliths are hard to scale and microservices are too complex, then how do you design a system that can scale with your traffic and your development team without becoming a pain to operate, maintain and expand its functionality? Over the past few years, it has become clear that a middle-ground is needed. I don’t expect this solution to work for everyone, but most products aren’t serving the kind of traffic that really makes the microservice effort worth it.

Enter the SUFA design pattern

Before getting into the core of what SUFA is, I want to mention that this is not an entirely new way of thinking. Things like the actor pattern, the Neomonolith, and others have stipulated some similar ideas over the years, and SUFA is just one way of combining several concepts into one straightforward design pattern. So, what is it?

Simple, Unified, Function-based Applications.

Let’s break that down:

Simple

A system designed with SUFA can be run in the simplest of deployment scenarios. Auto-scaling groups have existed for a long time, and they’re made even easier by container orchestration systems. A SUFA system can be run on one ASG, or can be expanded with a service mesh to allow for capability groups (which we’ll talk about in a future post).

Unified

Rather than multiple services who each exist as something to be deployed, SUFA systems exist as one single deployable. This can be a Docker image, an AMI, or some other artifact, but there is only one thing that needs to be built. It should be built by CI/CD on a continuous or tagged release cadence, and it should be made available in an artifact registry such as a Docker registry or S3 bucket.

Function-based

A standard monolith probably includes a handler layer which is responsible for taking API requests and making calls to a business logic or data storage layer to handle those requests. SUFA systems instead handle requests by chaining together a series of functions, each completely independent and unaware of one another. Functions should expect a particular input, perform some operations, and produce an output to be passed into functions further down the chain. Functions should be easily testable and reusable across different scenarios (such as for different API requests). SUFA systems should also be designed to consume and produce event-based traffic as a primary method of communication.

Applications

Well this seems like it should be straightforward, but in SUFA design, “Application” has a very particular meaning. A SUFA system should serve one single application, meaning that it should encompass all of the capabilities needed for a fully formed product. This can be up for some interpretation (such as whether a company should have one SUFA for their whole business, even if they have distinct product areas), but the point is to avoid having multiple “things” serving one application. If functionality needs to be shared across multiple applications, the functions comprising the SUFA system should be easily reusable and composed for other purposes.

You'll notice that this is all very technology-agnostic and vendor-agnostic. SUFA is meant to span across languages, cloud vendors, and deployment environments. SUFA is a way of designing your server-side system such that it is testable, scalable, and secure. You'll notice I haven't touched on scalability yet, so let's discuss that

SUFA at scale

The critical factor that allows a SUFA system to scale is that it is composed of independent functions. SUFA systems should rely on an underlying framework to orchestrate the execution of these functions such that it can scale effectively. By using a function runner or job scheduler to run the required functions, a SUFA framework abstracts away how the functions are executed, and the programmer writing the code only needs to indicate which functions to run, and in what order.

Additional scalability is provided by capability groups and meshing, which I plan on writing follow-up posts for, as they deserve to be explored at length.

Suborbital Atmo

The SUFA pattern was designed in concert with Atmo, which is an all-in-one framework upon which SUFA systems can be built. Atmo uses a file known as a 'Directive' to describe all aspects of your application, including how to chain functions to handle requests. You can write your functions using several languages to be run atop Atmo, as it is built to use WebAssembly modules as the unit of compute. Atmo will automatically scale out to handle your application load, and includes all sorts of tooling and built-in best practices to ensure you're getting the best performance and security without needing to write a single line of boilerplate ever again.

The awesome capabilities of WebAssembly and the design thinking behind SUFA are being harnessed by the open source Suborbital Development Platform to introduce a new way to build your web applications. I've been working for over a year to realize this goal, and I'm extremely happy with the results thus far. Riding the wave of new technologies and practices such as JAMStack and edge computing means that we have a opportunity to bring the best of the old and the new to the next generation of software makers to do incredible things. I hope you'll come and join me!

Please reach out on Twitter (@cohix or @SuborbitalDev if you'd like to talk about SUFA design or the Suborbital project!

You can also sign up for occasional email updates by visiting https://suborbital.dev.