DEV Community: Dylan Anthony

Replacing Planetscale

Dylan Anthony — Tue, 26 Mar 2024 23:08:35 +0000

Now that Planetscale is ditching their free tier, I need a new database for the free GitHub app that I maintain. There were three reasons I picked Planetscale in the first place:

Aaron Francis
It was free
The branching workflow for schema changes

If you want more Aaron, consider buying his course. To address the other two reasons, I'm splitting this blog post in two parts: "The database" and "The workflow". That way, if you only care about one or the other, you can read just that piece. Now, on with the show!

The database

First, let's talk requirements. The database for this app needs to be:

Cheap or free. I don't want to spend more than the GitHub sponsors for Knope are bringing in.
Vitess-compatible. Data migration should be easy, and I don't want to rewrite all the queries.
Durable enough. Some basic periodic backups should be fine. I don't need high availability.
Close to the app. Less latency means less app runtime, which means less cost.

Hosting providers

I can broadly categorize the places to host a database into three buckets:

Database as a service (DBaaS)
Hyperscaler managed databases
DIY databases on smaller clouds

Databases as a service were eliminated pretty much immediately. The cheapest one I could find was $15/month. I guess Planetscale was the only one offering a real free tier (their cheapest is now $39/month, by the way).

If I were to pick a hyperscaler, it'd be AWS since this app was running on AWS Lambda within its free tier (I've since moved it to the same provider as the database). I also have managed RDS in production before, so the learning curve wouldn't be too steep. However, the free tier for RDS is time-limited, and the cheapest possible price I could find is twice as expensive as the cheapest DBaaS. So that leaves the third option.

When it comes to the smaller clouds, there's only one choice. My Railway account is old enough to still get $5/month for free, and I've accumulated some credits through referrals (with templates and links like the ones in this post). That means, as long as I keep it below $5/month, I can host the databases and the app for free. If I go over, I have some credits to sustain the app for a while until it gets sponsors 🤞.

Railway is not sponsoring this post. Any credits I get from the referral links will be used to power free resources like the GitHub app in question.

Also, as of writing, they have the best developer experience of any cloud provider. So I'd recommend them even without the referral links.

Database technology

Deploying my own database on Railway means I can pick whatever database technology I want. The two main options that are compatible with Vitess are MySQL and MariaDB. MySQL has an official template which can be deployed with a couple clicks, so that's what I chose. Yes, that really is why. Developer experience matters.

Setting up MySQL

After deploying the template, I enabled app sleeping to save on credits (and energy usage) when the app is not in use. Next, I copied over the schema from Planetscale to Railway. I used Atlas, but mysqldump would work just as well for this part.

Juggling databases

There are a bunch of well-documented strategies for cutting over databases, so I recommend you look for one that best suits your needs if you're doing something similar. For me, most of the data is a cache of GitHub data, so I didn't need everything, just the frequently accessed things (to avoid flooding GitHub's API with requests). To achieve this, I:

Added a connection to the new database
Changed all queries to read from the new database first. If data was missing, it read from the old database and wrote the missing data back to the new one
Let it run for a few days to capture the majority of the data

Once I was satisfied that the remaining gaps were small enough, I removed the connection to the old database, and I was done! Well... almost.

How did I make sure I didn't forget any queries? The app is written in Rust, which means I can use "compiler-driven development" to make sure that even the most major refactors don't break anything. If you'd like to see a whole post about this strategy, let me know!

Backups

The official template doesn't have automatic backups. I don't need anything too fancy, just a recent snapshot so I can restore the necessary bits if something catastrophic happens at a datacenter. I checked out the existing templates on Railway, and none of them quite did what I wanted, so I made my own (as well as a template which bundles MySQL and the backup job for even fewer clicks!).

This leverages Railway's built-in cron feature to take a snapshot periodically—so the backup process itself is only running (and costing me credits) when necessary. This also gives me a handy "Run now" button to make one-off backups for testing.

Internally, it uses mydumper, a faster alternative to mysqldump, to take the snapshot. Then, it uses rclone to push that snapshot to a Cloudflare R2 bucket. My usage will stay well within the free tier of R2, so this is effectively a free backup for me.

Remember, no backup is complete until you've tested restoring from it! I did that on a local MySQL instance and it worked great.

How much does it cost?

This is not going to be indicative of your costs—my app is very bursty, so the database can spend a lot of time sleeping. Railway is also very bad at estimating costs for apps that sleep. Plus, there's some sort of bug in the per-service breakdown where disk usage isn't covered. But, after running for about 2 weeks, my two sleepy MySQL instances have cost around $0.61. So for a month of usage at the current rate, I'd expect the total cost to be around $1.50. That's plenty of headroom before I hit my $5/month free limit and need to start using credits (though, this isn't the only project I'm running on Railway).

In case you're curious, the app itself is set to cost around $0.04/month while sleeping at about the same rate as the databases. Rust on the backend can make for some very economical apps!

The backup job says it's costing $0.00, so I guess it's quick and infrequent enough to be a rounding error!

The workflow

One of the main reasons I picked Planetscale in the first place was the ability to propose changes from one database schema to another. Iterating directly on the test database while developing, then applying those changes to production is much nicer than the typical migration script approach.

I chose Atlas to replicate this. It has a lot of features, but the important parts for me are inspecting a database, storing the schema as HCL (HashiCorp Configuration Language), and applying that schema to another database. Using this tool, my workflow looks like this:

Make changes directly to a test database while iterating on a new feature.
In GitHub Actions, automatically inspect the database and keep the HCL up to date.
When the feature is done, review the HCL schema changes in the pull request in GitHub.
Once the PR is merged, apply the changes to the production database in CI before the service is updated.

As an extra layer of validation, the app connects to the database while building to ensure that the schema is up-to-date. If something goes wrong with this process, the build will fail (rather than queries failing at runtime). If your database library offers this feature (like sqlx does), I strongly recommend it.

Here's what the actions look like:

on:
  pull_request:
    paths:
      - ".sqlx/*"
jobs:
  update-schema-file:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4.1.1
        with:
          repository: ${{ github.event.pull_request.head.repo.full_name }}
          ref: ${{ github.event.pull_request.head.ref }}
      - uses: ariga/setup-atlas@master
      - name: Update schema file
        run: atlas schema inspect -u ${{ secrets.TEST_DATABASE_URL }} > .sqlx/schema.hcl
      - uses: stefanzweifel/git-auto-commit-action@v5.0.0

When there's a pull request that affects a file in the .sqlx directory, Atlas inspects the test database and saves its schema to a file. The format is HCL, though that doesn't really matter to me, as long as it's readable. This file is then committed back to the repository.

This prevents the HCL file from being updated for unassociated pull requests, like automatic dependency updates. That .sqlx folder gets updated whenever I change queries in the app, so it's a good signal that the schema might need updating. If I need to manually update the schema, say for an index-only change, I can do that by running the same command locally.

For extra credit, you can also set up a periodic job to check the schema and, if it's changed, open a pull request with the updated HCL. This would work as drift detection.

When the HCL file changes on the main branch, this next workflow updates the production schema automatically:

on:
  push:
    branches:
      - main
    paths:
      - ".sqlx/schema.hcl"
  workflow_dispatch:
jobs:
  update-prod-database-schema:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4.1.1
      - uses: ariga/setup-atlas@master
      - name: Update schema
        run: atlas schema apply -u ${{ secrets.DATABASE_URL }} -f .sqlx/schema.hcl --auto-approve

So my schema changes are tied to my pull requests, I can review the changes as part of the PR, and when I merge to main to deploy to production, the database gets updated too!

For extra security, you may not want your MySQL database to be reachable from the public internet. You can use Railway's private networking to achieve this with a few changes:

Turn off app sleeping. Private networking doesn't work with sleeping apps. I don't know whether you only need to keep the database awake or the app too, but it's probably both.
Deploy a your own GitHub Actions runner in each Railway environment. Private networking does not work cross-environment, so you'll need a runner that has access to each database which Atlas will be talking to. That runner cannot sleep either.

This workflow relies on having one test database per PR, which works for me because I'm only working on one major feature at a time. If you're working on a team, you'll want to check out Railway's PR environments or use a local workflow for updating the schema.

Let me know if you'd like a full guide on setting up a database schema workflow for teams.

Wrap-up

That's it! I've replaced Planetscale by running MySQL on Railway, sending backups to Cloudflare R2, and managing my schema with Atlas in GitHub Actions. I'm actually happier with this setup than I was with Planetscale. My metrics are showing the slowest queries as 10x faster, even when the database was sleeping! Atlas is also a nicer solution for schema management; auto-deploys were much easier to set up, and I can see the schema diff right next to my code diff.

If there's any part of this solution you'd like me to dive deeper into, let me know! I'd also love to hear about your database setup and how it's working for you (particularly migrations).

Making my blog more climate-friendly with Astro

Dylan Anthony — Tue, 05 Mar 2024 01:44:24 +0000

It's been two years since I last rewrote my personal website, so naturally it's time to do it again! The immediate motivation was simple: my website stopped building with an inscrutable error message. Rather than taking the time to learn more about Zola and what could be causing the issue, I figured I would switch to something with a bit more community support: Astro.

Why Astro?

I recently created a new documentation website for Knope using the incredible Starlight theme for Astro. The developer experience, community resources, and performance of the site were all so good that I knew I wanted to build more with Astro, so it was the natural choice for this rewrite.

But... climate-friendly?

Yes, climate-friendly! There's this neat tool called Website Carbon that estimates the carbon emissions of a website. Here are the results for my old website:

A B is pretty good, better than 79% of other websites, apparently. But... "good" is never good enough for me, especially when it comes to climate change. So, how does the Astro version of my website compare? I used almost identical styles, markup, and content and I hosted it in exactly the same way. Here are the results:

A 70% reduction in carbon emissions! Now that's something to be proud of. But... how? What is it about the Astro site that makes it better for the environment? Primarily, it's reduced data transfer. Check it out, here's the network view from my old site:

And here's the network view from the Astro site:

On first load, the Astro site is 90% smaller than the Zola site! There are two reasons for this, first, Astro is lazy loading the images that are "below the fold", so it doesn't download data until it's needed. Not only that, look what happens when we scroll all the way down and load everything (even more than the Zola site had loaded):

The Astro site is still 73% smaller than the Zola site! If we look a little closer, we can see why: every individual image is smaller. I'll be honest, I don't know what Astro is doing to make this possible, but to my eyes there's no noticeable difference in quality, and that's incredible!

What about clean energy?

My website is hosted on a combination of Netlify and Cloudflare, and both have strong commitments to sustainability. However, all the clean energy that is produced will be used, so even though my energy is coming from a sustainable source, using less of it will still reduce carbon emissions.

I like to think of it like the solar panels on my house. They produce electricity as long as the sun is up, and on most (non-cloudy) days, they produce more than we use. So what happens to the excess? It flows back into the grid and helps power my neighbors' homes, replacing energy that may have come from fossil fuels otherwise. The energy I consume is not generating carbon emissions, but by consuming less I'm still reducing the overall emissions. It's the same with my website, and yours! So, even if your site is powered by renewable sources, it's still worth trying to reduce the energy consumption (by reducing data transfer).

How hard was it?

Like I said, I'm not a frontend developer, so changing up my website was very intimidating. Luckily, Astro makes things relatively easy. The Astro CLI gave me a blog template that worked great as a starting point. Next, I copied over my Markdown files and images. The only changes I had to make were to the frontmatter. For styling, my old site was using Tailwind, and there's an Astro plugin for that (much easier than it was to add Tailwind to Zola). Really, the hardest part was figuring out how to translate the jinja-like templates of the old site into Astro components. To be honest, the toughest part of that was understanding the templates of the old site, since I found Astro's syntax much easier to work with.

All-in-all, it took me a couple of afternoons to fully translate my site to Astro, and I'm extremely happy with the result. There are some minor style differences, but making it look identical wasn't the goal, I just wanted to preserve the general design and content.

Should everyone switch to Astro?

Probably not 😅. But, I think it's worth checking on the carbon impact of the websites you maintain to see if there's a clear path to improve them. Astro is definitely a good choice for new projects, or if you're planning on rewriting anyway. If you do find a great way to make your site more sustainable, I'd love to hear about it!

GraphQL is for Backend Engineers

Dylan Anthony — Mon, 05 Feb 2024 22:22:42 +0000

Most articles explaining the benefits of GraphQL focus on advantages for the frontend: things like preventing overfetching, reducing round trips, and iterating faster. But GraphQL provides just as many advantages for backend developers, which is why I choose it by default for new APIs and why you should consider it, too.

Improved communication

The goal of building any API is to enable someone to use it, which requires excellent communication between API producers and consumers. However, the state of the art for REST APIs doesn’t quite live up to this expectation.

On the backend, developers either need to manually document the entire API or rely on auto-generation tools that don’t fully meet their needs. Consumers face the same choice, write code by hand or workaround the bugs in their SDK generator (stated, lovingly, as the maintainer of an OpenAPI client generator). On top of this, these solutions result in inconsistent understandings of the API. Reproducing errors becomes time-consuming and frustrating, which feels like a battle instead of a collaboration. What we need is a shared language to describe how the API works—one that doesn’t add unnecessary layers of abstraction or manual work.

Meet GraphQL, where every team uses the same schema definition language (SDL) to talk about the API. SDL is designed for developers, omitting extraneous details while describing all the essential bits. This serves both as your documentation and a universal language to communicate between teams.

On the backend, developers can write code in their language of choice and have the SDL created for them. Alternatively, they can write the SDL by hand, and generate the required code. Either way, the documentation is always kept in sync with the implementation.

Consumers use this same SDL directly, so there’s no need to maintain a separate documentation website. Instead, they can get information, auto-complete names, and see potential mistakes right in their IDEs. The same request they write in their code can be run by itself for easier debugging, which makes sending snippets to the backend team much more straightforward. With clearer communication, there will be fewer open issues and less wasted time tracking down minor bugs.

Less bikeshedding

Bikeshedding is when you spend time discussing decisions that don’t have a significant impact, and it happens quite a bit with RESTful APIs. To name a few, I have been in bikeshedding meetings about:

Which verb to use? PUT and POST could have a lot of overlap. Maybe you *should*use GET, but the fact that it can’t contain a body is too limiting for the amount of filters you need.
Should this be a query parameter, path parameter, form data, or JSON body? For path parameters, how should we serialize arrays? What about objects?
Do we completely normalize endpoints to reduce duplication, even though that causes slower client performance? Or include the kitchen sink even though that causes slower individual endpoint performance? Maybe a compromise with shallow nested objects containing only the most common parameters or a parameter that allows selecting fields, emulating GraphQL without the type safety.
Which HTTP status code is the right one for each response?
Should relationships be indicated with IDs or URIs?

Hopefully, these discussions will lead to a comprehensive style guide, but the meetings will continue even with those guides. While there will always be some debate around things like naming, GraphQL eliminates entire categories of potential style issues (including all those above) by being simpler. You can spend less time on the little details and more time solving more interesting problems.

Reduce tech debt

Imagine you have a /v1/data endpoint called by all your frontend teams: web, iOS, and Android. You decide that the date field should return UTC instead of localized time, so how do you make that breaking change? It usually looks something like this:

Create a new /v2/data endpoint.
Inform all the client teams that this new endpoint exists and that they should migrate by some date.
Track metrics on /v1/data and send reminders to consumers who haven’t updated.
The deprecation date comes and goes, and you’re still receiving calls to the old endpoint.
You either pull the plug and start breaking things or, more likely, continue to support the /v1/data endpoint indefinitely, creating tech debt with no apparent payoff date.

Now, let’s play through the same scenario with GraphQL. First, you add @deprecated(reason: “use utcDate”) to your old date field and add the new utcDate field. API consumers immediately start seeing warnings in their IDEs and potentially CI/CD about this deprecation since most GraphQL tooling supports it.

As it turns out, the iOS and Android teams weren’t even using the old date field, so no changes are necessary for them!

Someone on the web team notices some squiggly lines in their editor and decides to fix them, following the instructions left in the reason field. Your backend metrics tell you that the date field is no longer in use, and you’re safe to remove it!

Creating less churn and an easier upgrade path for clients makes them more likely to upgrade. If they upgrade faster, you can remove that stale old code faster, too!

GraphQL is for you too

One of the main features of GraphQL is its developer experience. By creating a shared language between teams, reducing complexity, and providing a smoother upgrade path, GraphQL makes building and maintaining APIs more delightful. That’s why, even if your frontend engineers aren’t pushing for it yet, you should start evaluating GraphQL for your tech stack. If you have any questions or disagree with anything I’ve said above, let me know on our Discord server! If you want to be notified of future posts, there’s also a channel for that.

Secure GraphQL Microservices

Dylan Anthony — Wed, 02 Aug 2023 18:17:49 +0000

Federation unlocks superpowers for our queries, enabling us to split up business logic and improve performance with features like @defer. However, these same powers can be abused if placed in the wrong hands, so it’s essential to limit who has access to them.

The threats

Many coordination features of a federated graph rely on an important assumption: clients always query your router—never individual subgraphs. If this assumption is violated and clients query your subgraphs directly, you might expose data and capabilities that you shouldn’t:

The @inaccessible directive enables subgraphs to define schema fields that other subgraphs might need, but that the router shouldn’t expose to clients. If clients can query a subgraph directly, those queries can include @inaccessible fields. And even if you don’t have any sensitive @inaccessible fields in your graph today, you might in the future!
The @requires directive enables a subgraph to define schema fields that depend on entity fields from other subgraphs. Subgraphs rely on the router to resolve this dependency, ensuring that trusted data is provided. But if clients have direct access to subgraph entity resolvers, that data can no longer be trusted.
The @override directive enables a subgraph to take responsibility for a schema field away from another subgraph. With direct subgraph access, clients can still query the original subgraph field, which can result in inconsistent and unexpected behavior.

This is just a small selection of the available features today, and more will be added over time. The key takeaway is that federation is a microservice architecture that expects a single entry point through a controlled router. If subgraphs can be accessed directly, it’s only a matter of time before something unexpected happens, and that unexpected event could be a security breach.

Protecting your subgraphs

Exposing subgraphs publicly opens the door to several attacks. To mitigate them, we prevent anything but our routers from accessing our subgraphs. First, we can protect our subgraphs at the application level by adding an extra layer of authorization on top of whatever user auth may be in place. We can configure our routers to send a header containing a shared secret to each of our subgraphs:

headers:
  subgraphs:
    orders:
      request:
        - insert:
          name: "Router-Authorization"
          value: "${env.ORDERS_SUBGRAPH_SECRET}"
    users:
      request:
        - insert:
          name: "Router-Authorization"
          value: "${env.USERS_SUBGRAPH_SECRET}"

Notice that we use a unique secret per subgraph, following a security best practice called “the principle of least privilege”. If there’s no reason for the orders subgraph to have direct access to the users subgraph, then we should not give it the ability to do so (by sending it valid tokens for another subgraph).

These values come from environment variables, so how you set them will depend on where you’re running your routers. Most secret managers have a way to inject values via an environment variable. For example, in GraphOS, you can set secrets via the UI.

How this looks on the subgraph side depends on your implementation. Here’s an example using FastAPI in Python:

@app.middleware("http")
async def check_router_security(
    request: Request, call_next: Callable[[Request], Awaitable[Response]]
) -> Response:
    router_secret = environ.get("ROUTER_SECRET")
    if router_secret is None:
        return await call_next(request)
    if request.headers.get("Router-Authorization") != router_secret:
        return Response(status_code=HTTPStatus.UNAUTHORIZED)
    return await call_next(request)

Here, we’re setting up the expected token as an environment variable called ROUTER_SECRET in our subgraph. This value would be injected via the secret manager for your hosting platform. If the secret is not set (such as in local development), we turn off this enforcement; you may want to add a more sophisticated method of ensuring this is never turned off in production. If the secret is set, we ensure every request has a matching Router-Authorization header. Requests without a valid header will get a bare response with a 401 Unauthorized code—we don’t send any additional information that clients don’t need, and unauthenticated requesters don’t even need to know that this is a GraphQL server.

We have examples for many languages and frameworks in our subgraph templates, so check them out for more inspiration!

Extra credit: network protections

As an additional step, depending on your hosting environment, you may be able to use network-level protections to prevent any incoming connections to your subgraphs except your routers. This is typically done via firewall rules (such as access control lists). Not all platforms or architectures will have this option, but if you do, we recommend adding it as a “defense in depth”.

Ruling out alternatives

There are a few other solutions to this problem that might seem viable but can cause issues. Let’s go over them!

First, you might be tempted to apply router authorization checks only for federation-specific subgraph fields (_service and _entities ), but there are problems with this approach:

Newer versions of federation might add more subgraph fields in the future, which would introduce new vulnerabilities that special-case checks don’t handle.
This approach doesn’t solve the problem of reaching @inaccessible or @requries fields via a top-level Queryfield (a standard, non-federated query).

Second, you might be tempted to enforce router authorization per field, maybe even automatically for any field that implements a federated directive. However, federation enables any single subgraph to influence the behavior of the entire supergraph. For example, if @inaccessible is applied to a field in any subgraph, it’s expected to be enforced on every subgraph. This means that per-field, per-subgraph enforcement is fundamentally inconsistent and can lead to unexpected data leakage.

Finally, you might consider disabling introspection as a solution—if attackers can’t discover the hidden fields, they can’t use them, right? This is called “security by obscurity” and is widely considered not security at all. A sufficiently determined attacker will eventually guess the names of fields that you don’t want them to find.

The only consistent approach to preventing undesired access to subgraph capabilities is to disable access to subgraphs entirely to any client besides the router.

Get secure

Now that you know the dangers of exposing subgraphs publicly and the best approach to mitigating the threat, there’s nothing left to do but start implementing! Start securing your subgraphs immediately.

Have any questions, comments, or concerns about this post? I’d love to hear about it in our Discord server! That’s also where you’ll be notified of upcoming security-related live streams.

Even better GitHub Actions in Rust

Dylan Anthony — Sun, 25 Jun 2023 03:34:13 +0000

In a recent post, I introduced a method for creating GitHub Actions (the reusable code that can be run in CI, not to be confused with the name of the CI platform itself) using Rust and Docker. This was the easiest way I knew to create Rust-based actions... until now.

In the GitHub discussion for that post, someone brought up a limitation I hadn't realized. Docker-based actions can only pull images from public registries, meaning you couldn't author a closed-source, private action using this method. Additionally, Docker-based actions can only be run on Linux-based runners, limiting utility even further.

With a lot of inspiration from that conversation and trial and error, I created version 2.0 of my Rust-based GitHub Action template. You can try it out today with cargo generate dbanty/rust-github-action-template! Before you do that, read on to see what's changed.

No more Docker

The only way to surpass the abovementioned limitations was to stop using Docker-based actions. This leaves us with only two other options: javascript or composite. We're writing our actions in Rust, so using JavaScript as an intermediary seems silly—let's go straight to composite.

A composite action in GitHub is basically a reusable workflow with inputs and outputs that can be published on GitHub Marketplace. Generally, this is good for scripting with Bash or PowerShell and not too much else. However, that scripting is more than enough to run a Rust binary (you do that every time you run cargo test in CI), we just need to make the correct binary available!

Distributing binaries

We could copy over the Rust source code for our action and build the binary right there in the reusable action—but consumers would have to then (even if just occasionally, with caching) wait for Rust to compile something, which can take a very long time.

The other option is to produce pre-built binaries somewhere and download them from the composite action, then run them—which is precisely what I did! Luckily, I already have several examples of building Rust binaries and storing them in GitHub releases (that's how my release automation tool Knope works), so I just needed to build up scripting logic which:

downloads the correct version of the binary for the current platform
passes through arguments from the action inputs to the binary
sets outputs for the composite action correctly

We get a GitHub Action that does not require Docker at all, builds and distributes Rust binaries in a much more standard way, works with private repositories, and works on macOS and Windows runners.

A new developer experience

The new setup has a few tradeoffs, but I think they're worth it. First, instead of tracking a constantly-evolving v1 branch, consumers of the action will target a particular GitHub Release where they can download artifacts. That means consumers need to update their workflows to get the latest action logic—but that's easy with a tool like Renovate and probably leads to more consistent CI/CD anyway.

The trickier part is for you, the maintainer. Instead of your consumers getting a new version of the action every time you merge to the default branch, you'll need to create a new GitHub release with all the required artifacts. That process today looks like this:

Run the "Release" workflow by clicking a few buttons in the GitHub Actions interface and putting in your desired, updated version (note that this does not update the version in Cargo.toml, it just sets the version in GitHub releases).
Wait for the integration tests to pass with the new version to ensure nothing broke.
Open the new release and fill in any release notes. Set it as the latest release and publish it to the marketplace.

I think this can get a little bit easier by integrating it with Knope; for example, the release notes can be set, and the new version can be determined automatically (plus, it'll bump Cargo.toml). There doesn't seem to be a way to publish to the GitHub Marketplace via API, though, so there will still be some manual steps 🤔.

Let me know

So what do you think about this new and improved version? Do you prefer the Docker method? Is there still something missing from the Rust-based-actions developer experience that you're waiting on? Let me know!

Rust is not hard! Part 1: GitHub Actions

Dylan Anthony — Tue, 02 May 2023 23:56:07 +0000

The most common description of Rust I hear is something like, "it's great for performance but too hard or cumbersome or annoying for most tasks." I don't think this description could be more wrong. Sure, it can be fast, but that's not the main reason to pick Rust. You should pick Rust because it's easy to build with. It empowers you to create incredible software while enjoying the experience.

Sound too good to be true? I'll be honest; although this is my fervent belief, I wasn't sure it would hold up in many situations. Until now...

TL;DR

I implemented the same reusable GitHub Action in both TypeScript and Rust
It took about 3 hours for each implementation, 5 minutes fewer for the TypeScript version
Rust was significantly less frustrating in debugging the problems I had, and I am more confident in the quality of the result
I created a template that should shave around 45 minutes off of the Rust time for future adventurers, meaning Rust should be even faster than TypeScript to implement moderate-to-complex actions from now on 🎉

The first experiment

I wanted to write a reusable GitHub Action which runs some quick checks on GraphQL servers. I've authored actions before, but they're usually short Python scripts distributable via composite actions. This new action is too complex for a simple script.

So, naturally, I decided to check out the other options for actions and found that the best-supported, most popular methods are JavaScript and TypeScript. So I wrote this action in TypeScript (my preference of the two) and had a terrible time doing it 😬. "There must be a better way!" I cried and immediately started over, this time in my favorite language.

That's when the idea for this blog post came about—if I were going to implement this same action in TypeScript and Rust, I might as well compare my experience in each. So, I implemented the action twice more, this time while recording my efforts for science! First, I wrote it in Rust, then in TypeScript (again).

Side note: I'm not going to release that raw footage because it's incredibly boring (trust me, I had to watch it and take notes to come to my conclusions), but if anyone would be interested in a sped-up, commentated version where I walk through each implementation and all the trouble I had, let me know!

I intend to do something similar for other types of projects and languages. If there's a particular comparison you'd like to see, please let me know!

Some Rules

I wanted the comparison to be as fair as possible, so I set a few rules for myself:

When I encounter a problem I've seen before, I must do my best to solve it as if I don't know the solution. This is especially important because I have more Docker + Rust experience than most developers. I have also written this exact thing in TypeScript before (so I have solved most problems before).
Any work which is duplicated between both implementations doesn't count against either. For example, I created a bunch of integration tests that were completely reusable for both implementations—so I cut that out of my analysis.
The code should be shippable—but does not need to be perfect. I must be confident enough in the action to push the "publish" button, but I don't need to go down every rabbit hole to make it optimal (for example, I didn't parallelize any of the checks or set up benchmarking). I do, however, need to clean up any warnings or errors from the build tooling and linters.

The Setup

I mostly wrote each implementation in one go—Rust, then TypeScript—because it was more natural. However, I will tell the story by swapping back and forth between the same section for each language. The first task was getting started—setting up a super simple working action to serve as a foundation.

In Rust, this took a fair amount of effort. I spent 58 minutes (about one-third of the total) setting up the action. First, I had to read the GitHub documentation on Docker actions, then find an article about making a Dockerfile for Rust. Once I had the file, there was some back-and-forth fighting with GitHub Actions—and that was just the beginning. The hardest part was figuring out how to bypass the rebuilding of the image on every run—Rust is notoriously slow to compile, and waiting several minutes for the action to start was out of the question.

This section highlights the worst part of Rust today—it is still a very young programming language, so it's missing a lot of resources available to other languages. After this project, I made a template and a corresponding blog post so that future developers (probably me) will have a much easier time implementing Rust actions. However, this template wasn't available to me yet, so it doesn't count for this experiment.

On the TypeScript side, setup was much easier. There was already a template from GitHub that took care of the basics. Most of the time spent here was updating dependencies and getting my editor to play nicely with it—18 minutes, about 10% of the total.

This was the only part of implementing this action that took me less time in TypeScript than in Rust—but it took 40 minutes less, which is enough to bring the total development time in favor of TypeScript.

Implementation

Writing the business logic of the action took the bulk of the time in both implementations—116 minutes for Rust and 145 minutes in TypeScript. Let's walk through each issue I spent time on in this phase.

Error Handling

Rust does not have exceptions. This means that every time there could be an error, you're forced to deal with it. I don't allow panics in my production code, so I wasn't going to .unwrap() as a shortcut. Instead, I produced meaningful error messages for every possible error condition I encountered and bubbled them up in a type-safe way. This requires more upfront thought and effort than exception-based languages but means you're less likely to show an unfriendly error message (like a stack trace) to an end user.

In TypeScript, there are exceptions. As with most exception-based languages, there are no static tools to help you know where or when they could occur. For production code, displaying an exception's stack trace is unacceptable (just like panicking)—I want to give users actionable advice. In TypeScript, you either have to encounter an error organically (hopefully not in production) or rely on documentation. It's also exceedingly rare for the type of all potential errors to be documented, so usually, I had to run the code through known error conditions and read the output (or set a breakpoint with a debugger) to find out what information was available to me. For example, I want to tell the difference between someone providing a malformed URL and a server being unreachable (so that I can provide users with relevant suggestions). The only way to do that is to know how the errors differ between those two conditions.

Overall, I think I spent more time handling Rust errors (I certainly had more checks), but I was more frustrated dealing with TypeScript errors (because what I was looking for was hard to find). I definitely have more confidence that my end users will have a better experience with the Rust version since I may have missed some possible TypeScript exceptions.

Learning about GitHub Actions

Even with the setup done, there was much to learn about using GitHub Actions in both environments. The TypeScript template mostly came with examples of functions to call to get inputs, set outputs, etc. For the Rust version, I mostly resorted to reading bash examples and translating those to equivalent Rust code. TypeScript is the clear winner, though the total effort for either language was quite low. Again, like the initial setup, this is because TypeScript is older and more popular than Rust.

Outdated idioms

This is the other side of the "Rust is a young language" coin. When following examples and IDE suggestions, there was only a single instance of copied code being out of date—and the code still ran fine; the linter just told me I should do it the newer way (format!("{thing}") instead of format!("{}", thing)). There wasn't a single time that an example or suggested code didn't compile and run correctly.

On the Node side, conflicting idioms are documented everywhere, and it's not usually clear whether they are outdated! For example, axios is documented using the Promise API and CommonJS require() imports, but neither worked for my project. My ESLint setup (inherited from GitHub's template) told me that async/await is preferred, which required rewriting and made the error-handling documentation for axios all but useless to me. My setup also wanted ESM-style import instead of require()—but switching broke Jest. Several articles and incorrect fixes later, I realized that I just needed to update Jest. These issues were constant in TypeScript, and the error messages and search results were rarely helpful.

Here we have a clear win for Rust, and I don't expect that to change. The official Rust linter, Clippy, is the best I've seen in any language at suggesting (or automatically fixing) updates to idioms. Rust also has a strong backward-compatibility guarantee and dependency management system that, together, mean your builds won't start failing in the future when new versions of the compiler or your libraries come out.

Paralyzed by choice

While there is often one idiomatic way to do things in Rust, sometimes the choice is unclear. For example, if or match could be equally valid in branching your logic—sometimes, you have to try both to know what feels better for a particular case. Likewise, choosing between functional-style iterators and imperative loops is not always clear until you've started down one path. It can be good to have options, but it's also easy to waste time deciding the best method.

Another example of this is going down the rabbit hole of references and lifetimes. Remember, premature optimization is never a good idea. Start by cloning if you have borrowing issues, only try to optimize with references if you're sure you need them.

TypeScript is certainly not immune to this, but I found myself spinning for options much less frequently. Usually, there is one preferred method and several older discouraged methods (which ESLint often catches). I give TypeScript the edge on this issue for less time spent on bikeshedding.

Debugging remote responses

When your code is talking to a remote data source—you will inevitably have to inspect what's happening at runtime when weird things occur. The two approaches I take are setting a breakpoint with a debugger and logging out the information I need to a console (printing). Using a debugger in Rust is easy (with an IDE like CLion), but the information you get from it is not always useful. For example, if I want to inspect a generic, parsed JSON payload without modifying my code, it's pretty much impossible. Because of this, I often resorted to printing the result, tweaking my code, and repeating until everything worked.

In TypeScript, the dynamically-typed nature of the code leads to a much clearer debugging experience. Actually, my process for figuring out what a remote server is doing is pretty much identical to figuring out how each exception worked. I suppose it makes sense that the debugging experience would be better for a language where less static information is available. TypeScript wins here.

The tooling

In Rust, you get a standard set of tools that all work very well together. The build tool (cargo) manages the compiler (rustc), your tests, and your dependencies for you. If you used rustup (the recommended installer/version manager for Rust), you also get a formatter (rustfmt) and linter (clippy) for free, which are guaranteed to work well with the rest of the toolchain. There are many more features (e.g., docs) that I didn't need here, but the picture is generally that everything plays nice and gets out of my way unless it has useful feedback.

In TypeScript, everything is separate. The compiler (tsc), formatter (prettier), linter (eslint), testing framework (jest), and runtime (node) are all distinct components, usually requiring extra dependencies to integrate them. NPM is generally pretty bad at telling you when you have incompatibilities (e.g., when my version of @types/node did not match my version of Node). The tools like fighting each other (e.g., when eslint was mad about an import that tsc needed for proper typing or when jest + ts-jest needed a different, special config from tsc/ts-node). You end up with many more config files and a lot more time spent fiddling with the tools than would be needed in Rust—sometimes to no end (I had to disable type-checking in a couple of places because I couldn't make it happy 😔)! This makes for a frustrating developer experience, especially when compared to the joy of Rust's.

Without question, Rust has better tooling. In fact, Rust has the best tooling of any language I've ever used.

Documentation

Every library I've used in Rust has docs on https://docs.rs generated with the standard cargo doc tool. You always know what to expect, examples can be tested with cargo test, and links to other dependencies are kept up to date. This is also the same format that the standard library is documented with. Overall, it's easy to learn how to use a new library.

In TypeScript, every library has its own custom documentation. Often, this is just a README file uploaded to NPM, which is usually insufficient. When you combine this with less descriptive error messages from the compiler (or exceptions from the runtime), it's much harder to learn how to use a library in TypeScript than in Rust.

Fighting the language

This is a much broader category, but sometimes the code you write doesn't work the way you expected it to. Rust and TypeScript have very different issues—Rust eliminates several categories of bugs that TypeScript is prone to but introduces some other challenges. For this, I'll focus on the one issue that stood out the most in each language when reviewing the footage.

For Rust, it was one of the things folks complain about the most: the dreaded borrow checker 🙀. When going about my business, an error popped up in my IDE that said something like "cannot borrow partially moved value". The easy solution was to add .clone(), but whenever the borrow checker indicates that I'm re-using a piece of data, I try consolidating them. As it turns out, I had two if statements that were easy to combine and made for nicer code. That diversion took about 2 minutes—though it required understanding ownership, a concept that doesn't have a parallel in most languages.

Learning about ownership taught me to see my data in a different light. I have caught several bugs in other languages because I was thinking like the borrow checker. I don't think learning ownership is an obstacle to overcome, but rather a useful tool that every developer should have.

In TypeScript, my biggest hurdle started with a failing unit test: the array of error messages coming back from the main function should have had a length of 1, but had a length of 0. My first instinct was that there was a bug in the error-checking code, so I set a breakpoint and started debugging. After carefully stepping through, I found that I had used a concat method instead of a push with a spread operator. With unit tests passing, I pushed up the changes to find that, in CI, the integration tests were failing! After adding some print debugging and pushing back to CI, I found that I had used concat in a second location, causing separate errors. Overall this took 20 minutes of messing around to get an answer for something that should have been caught statically—and would have in Rust.

One of the big differences between "fighting the language" in Rust and TypeScript is that Rust puts almost every problem in front of you immediately in the build and lint steps. Instead of failing tests, you get red squigglies in your editor, pointing you immediately to where the problem is. An example of this is #[must_use], which a function like concat would have in Rust. Basically, if a function returns a value, and you forget to use it (like, say, with errors.concat(new_errors)), it's a compile error.

While it can be annoying to see more of your bugs up front, I definitely prefer that to stepping backwards from a distant effect. Rust wins here.

Refactoring

After getting to the end of each implementation, I decided to take them a step forward by reworking some logic. Basically, I wanted to automate something that previously required a manual user setting.

First, I rewrote the required code in TypeScript; it took 4 minutes. Then, I did exactly the same thing in Rust—another 4 minutes! So that's a tie... right? Well, not quite. When I changed the Rust code, the compiler pointed out a bug that was easy to fix. After I was done, I returned to the TypeScript code to see if it contained the same bug.

Sure enough, I was consistent enough to write the same bug into both implementations 🤦. The tests I'd written didn't catch it, but the Rust compiler did. After another 8 minutes, I reorganized the TypeScript code to work properly. Without the Rust compiler, I'm unsure how long it would have taken me to catch the TypeScript bug.

One of the greatest strengths of Rust being so explicit and strict is that it can catch many bugs that other languages can't. This is often talked about in terms of memory safety (when compared to something like C), but it goes way beyond this. If you leverage the type system to reflect the expectations you're making as you code, it will catch bugs that would slip through in other languages.

Conclusion

Is Rust hard? As with most things, I don't think this is a binary yes/no question. However, I find Rust easier to work with than TypeScript (at least in this case—and I don't believe that TypeScript is considered a hard language. Personally, I would rather write production code in Rust than any other language.

Still not convinced? Let me know which language and scenario you'd like to see compared next.

Footnotes

A list of languages I've written production code in and consider the tooling significantly worse than Rust's, in no particular order: Python, Java, Kotlin, Swift, Go, TypeScript, JavaScript, C.
Regarding how much professional experience I have writing in programming languages, Python is the first, followed by TypeScript, then Rust. I have certainly spent more time writing Rust than TypeScript, but I've been paid for more hours (and had more peer reviews) with TypeScript. The order of my confidence in writing each language is Rust, then Python, then TypeScript.

How to write a GitHub Action in Rust

Dylan Anthony — Tue, 07 Feb 2023 04:26:13 +0000

Creating reusable GitHub Actions is an easy way to automate away everyday tasks in CI/CD. However, actions are typically implemented in TypeScript or JavaScript, and getting started in another language is much more challenging. My favorite language is Rust, so naturally, I wanted an easier way to oxidize my actions.

cargo-generate

Rather than walk through all of the manual steps like I had to, you can use cargo-generate to get started quickly. From the command line, run cargo-binstall cargo-generate followed by cargo generate dbanty/rust-github-action-template, then follow the prompts to fill in the boilerplate values.

Not familiar with cargo-binstall ? You can use it in place of cargo install to install supported binaries rather than compiling them from source! You should make your next Rust binary cargo binstallable!

Finally, you'll get a fully-functioning GitHub Action implemented in Rust, ready for customization! You can see an example of the output in my Sample Rust Action repo, which is also a GitHub template (in case you want to skip the cargo-generate step).

Next steps

There's a "TODO" section in the generated README.md that gives you a high-level set of next steps—so feel free to dive right in if you're a hands-on learner! For completeness, I'll walk through each of the steps here.

First, you'll want to update the README to describe what your action does and how to use it. I find it easier to describe the user experience I want to create before I try to create it, a sort of "documentation-driven development". As an example, you can check out the docs for my GraphQL Check Action.

Now that you've designed your action, you need to define your inputs and outputs in action.yml. Each input needs to be defined in two places:

inputs:
  error:
    description: 'The error message to display, if any'
    required: false
    default: ''
runs:
  using: 'docker'
  image: 'ghcr.io/<your_username>/<your_repo_name>:v1'
  args:
    - ${{ inputs.error }}

The inputs section is how you define inputs for the action itself—GitHub will do some validation here, and users might peak at the description to double-check what each input does. Then, in the runs section, you pass the input to your Rust binary. The order here matters—so it's easiest to add inputs one at a time.

The outputs section lets you tell users what they can receive when the action fails. I recommend including, at a minimum, an error output for easier testing:

outputs:
  error:
    description: 'The description of any error that occurred'

With inputs and outputs defined in action.yml, you need to consume the inputs and output the outputs! The generated code comes with an example of each:

//! src/main.rs
use std::env;
use std::fs::write;
use std::process::exit;

fn main() {
    let github_output_path = env::var("GITHUB_OUTPUT").unwrap();

    let args: Vec<String> = env::args().collect();
    let error = &args[1];

    if !error.is_empty() {
        eprintln!("Error: {error}");
        write(github_output_path, format!("error={error}")).unwrap();
        exit(1);
    }
}

Here we can see the list of arguments passed in from the args section of action.yml ends up in our args variable. The first entry in this Vec is the name of the binary, so the first argument is at index 1. The eprintln! line is to write a message to standard error—this will appear in the GitHub logs so that users know what happened. The usage of write() is an example of setting an output—you have to write to a file path which is set to the environment variable GITHUB_OUTPUT using the format <output_name>=<output_value>. Then, exit(1) will make the action fail the workflow (putting that little red x on a status check and preventing PRs from merging).

For more ways you can interact with GitHub Actions (like setting warning messages), I recommend reading https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions.

The last step is to change your default branch to be named v1. This is because the template assumes you will use semantic versioning and that users will always want the latest compatible release (v1). You can use whatever branching and tagging strategy you want, though you'll have to alter more of the generated code.

That's it! If you can handle inputs and outputs and set actions to failed, you're ready to start implementing basic actions! Just write Rust code like normal—you can even install any dependencies you want!

Testing branches and pull requests

The generated code comes with an included .github/workflows/integration_tests.yml file for testing the action. If we take a peak inside, we'll see that it consumes our GitHub action using the uses: ./ syntax:

jobs:
  test_success:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ./

This works just fine on the v1 branch, but if we look back at our action definition, we can see a problem:

runs:
  using: 'docker'
  image: 'ghcr.io/<your_username>/<your_repo_name>:v1'
  args:
    - ${{ inputs.error }}

The action uses the v1 tag of a Docker image built from this repo. That v1 tag corresponds to the v1 Git branch—meaning that if you run these tests on any other branch, you'll still be testing the v1 branch! The easiest way to override this (e.g., for testing a pull request) is to change that to image: 'Dockerfile' . This will test the code of whatever branch you are on, but be careful not to commit this change back to v1 or it will cause terrible performance for action consumers.

How it all works

Now that you know how to implement and test your actions, you're ready to go! But if you're still curious about how everything works, stick around for a deeper dive.

First, we use the Docker container action method—one of three ways to create GitHub Actions. This enables us to build whatever kind of binary we want, using whatever dependencies we want, without needing JavaScript or complex, dynamic install scripts. There are some limitations, though. Notably, these actions can only run on runners with a Linux operating system, making them less flexible or portable than JavaScript actions. Second, some capabilities may not be possible from your actions (like setting environment variables).

Another limitation of Docker actions is the one I mentioned in the testing section above. You either need to publish a Docker image and pin your action to a specific tag or rebuild the Docker image every time that action runs. Rust can take a long time to compile, especially when waiting for Cargo to download dependencies—so building the image every time makes for a poor experience for the action's consumers. However, pinning to a tag makes it harder to test multiple branches, a tradeoff we have to accept for now.

So, to get from your Rust code to a consumable action—we have to build a Docker image and publish it to a registry so that the action can pull it before running your code. Let's take a deeper look at each of these steps.

The template generates a workflow in .github/workflows/docker-publish.yml that builds a new image with every push to the v1 branch. It's a rather complicated workflow, so we'll take a look at a couple of snippets:

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  build:
    steps:
      # other steps omitted
      - name: Build and push Docker image
        id: build-and-push
        uses: docker/build-push-action@v4
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Here we see that we're publishing to the ghcr.io registry with an image named the same as our repository—that enables us to push to GitHub Packages with no additional authentication, all of your artifacts live with your repository! We use the cache-from and cache-to inputs to enable Docker layer caching—crucial given how slow Rust-based images can be to build. We're also passing the input context: ., which means it should build from a file called Dockerfile—let's look at that next!

FROM rust:1.67 as build

# create a new empty shell project
RUN USER=root cargo new --bin sample-rust-action
WORKDIR /sample-rust-action

# copy over your manifests
COPY ./Cargo.lock ./Cargo.lock
COPY ./Cargo.toml ./Cargo.toml

# this build step will cache your dependencies
RUN cargo build --release
RUN rm src/*.rs

# copy your source tree
COPY ./src ./src

# build for release
RUN rm ./target/release/deps/sample_rust_action*
RUN cargo build --release

# our final base
FROM gcr.io/distroless/cc AS runtime

# copy the build artifact from the build stage
COPY --from=build /sample-rust-action/target/release/sample-rust-action .

# set the startup command to run your binary
ENTRYPOINT ["/sample-rust-action"]

Going through this line by line, we:

Start with the official rust image for building—some slimmer images probably work, but this gives us maximum flexibility for template users.
We create an empty Rust binary with cargo new, this is a simple way to get Docker layer caching to work. For a more robust solution, you may want to check out cargo-chef.
The next few steps build just enough of our code to get dependencies to cache. Note that modifying Cargo.lock or Cargo.toml will bust the cache; this is partially why cargo-chef may be a better option.
The next couple of steps (starting with COPY ./src ./src and going through RUN cargo build --release) will build our finished binary
Now, we switch over to a smaller base image. You can make this even smaller by switching to gcr.io/distroless/static but it makes building harder (you have to use some musl toolchain stuff), and I found that it doesn't make the action any faster.
We pop our binary over into the fresh cc image, and set up the entry point (note that CMD doesn't work here; you have to use ENTRYPOINT). That's the whole image!

Once we've built and published the image, we immediately test it to catch any last-minute problems. Let's look back at the .github/workflows/integration_tests.yml file from earlier:

name: Test consuming this action
on:
  pull_request:
    branches: [v1]
  workflow_run:
    workflows: ["Docker Publish"]
    branches: [v1]
    types:
      - completed

jobs:
  test_success:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ./

  test_error:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - id: test
        continue-on-error: true
        uses: ./
        with:
          error: "This is an error"
      - name: Verify failure
        if: steps.test.outputs.error != ''
        run: echo "Failed as expected"
      - name: Unexpected success
        if: steps.test.outputs.error == ''
        run: echo "Succeeded unexpectedly" && exit 1

The workflow_run section tells this action to run after we publish to Docker—this ensures we're testing the version we just published and not an earlier variant. Then, the workflow comes with two tests as examples—you'll want to replace these with ones that exercise the actual inputs and outputs. The second job, test_error, is much more interesting—this is how you can test failure conditions (and why we set the error output). It's just as important to test expected failures as expected successes, maybe even more important!

Conclusion

My little cargo-generate template will hopefully make it easier than ever to write Rust-based GitHub actions. If you try it out and have any suggestions or questions, please open an issue on the repository. If you want to hear more about the motivation for this template—why I'm writing actions in Rust instead of TypeScript, follow me for that upcoming post!

Stop Writing DRY Code

Dylan Anthony — Tue, 05 Apr 2022 23:14:24 +0000

Cover photo created by Jazlyn Borkowski

"DRY" is an acronym introduced, seemingly universally, to software engineers early in their education. For example, when I searched "software engineering best practices," 4 of the top 5 results mentioned DRY. It stands for "Don't Repeat Yourself" and is one of the worst things you can teach a fledgling developer.

If you take nothing else away from this post, take this: DRY should never be a goal when writing code. It is not an indicator of code quality; it is, at best, a tool to be applied in some circumstances. Code that does not repeat itself is not inherently better than code that does, so "DRY" should never appear as a recommendation in your code reviews.

MOIST

Code is a bit like cake. A dry cake is brittle and likely crumble when you touch it. Likewise, DRY code—code that has no repetition—will resist extension and alteration. By combining two implementations into a single one, you bind them together, making it so you can't change one without changing both.

On the other hand, a wet cake will fall apart, not holding its structure. In software, WET usually means something to the effect of "write every time"—it's the opposite of DRY in that you constantly repeat yourself. If your code is WET, you can change one piece without breaking any others. However, two pieces of information that should be the same can easily fall out of sync.

The best cake, and code, is MOIST—"Maintain One Indisputable Source of Truth." Rather than being a hard and fast rule about writing code, it gives you a reason why to refactor code. MOIST gives you the best of both worlds by increasing rigidity to prevent bugs and keeping flexibility wherever possible. As with all things, balance is vital.

Let's look at an example where keeping your code MOIST is essential. We're writing a program that handles semantic versioning for projects. It has two commands: auto reads your commit history and generates a new version. The manual command takes a rule name from the user and bumps the version according to that rule. A first attempt might look like this:

fn auto(history: &[Commit]) {
    let mut version = Version::get_current();
    let mut major = false;
    let mut minor = false;
    let mut patch = false;

    for commit in history {
        if commit.message.contains("BREAKING CHANGE") {
            major = true;
            version.major += 1;
            version.minor = 0;
            version.patch = 0;
            break;
        } else if commit.message.contains("feat") && !minor {
            minor = true;
            version.minor += 1;
            version.patch = 0;
        } else if commit.message.contains("fix") && !patch && !minor {
            patch = true;
            version.patch += 1;
        }
    }
    write_version(version);
}

fn manual(rule: String) {
    let mut version = Version::get_current();
    if rule == "major" {
        version.major += 1;
        version.minor = 0;
        version.patch = 0;
    } else if rule == "minor" {
        version.minor += 1;
        version.patch = 0;
    } else if rule == "patch" {
        version.patch += 1;
    } else {
        panic!("Unknown rule: {}", rule);
    }
    write_version(version);
}

Here we're maintaining two different implementations for applying a semantic rule to a semantic version—two distinct truths about the rules! Unfortunately, this separation could easily lead to disparate behaviors between the two commands, confusing users. So let's maintain only a single source of truth for how to bump that version number!

fn auto(history: &[Commit]) {
    let mut rule = "patch";

    for commit in history {
        if commit.message.contains("BREAKING CHANGE") {
            rule = "major";
            break;
        } else if commit.message.contains("feat") {
            rule = "minor";
        }
    }
    bump_version(rule);
}

fn bump_version(rule: String) {
    let mut version = Version::get_current();

    if rule == "major" {
        version.major += 1;
        version.minor = 0;
        version.patch = 0;
    } else if rule == "minor" {
        version.minor += 1;
        version.patch = 0;
    } else if rule == "patch" {
        version.patch += 1;
    }
    write_version(version);
}

fn manual(rule: String) {
    if rule != "major" && rule != "minor" && rule != "patch" {
        panic!("Unknown rule: {}", rule);
    }
    bump_rule(rule);
}

There, all good, right? Well, not entirely. Our new bump_version function is the arbiter of applying rules to versions—but now we have three different places those rules are defined! If we want to add a "prerelease" rule, we'd have to remember to change every location separately, which could easily cause a bug! So, again, we'll maintain only a single source of truth.

enum Rule {
    Major,
    Minor,
    Patch,
}

fn auto(history: &[Commit]) {
    let mut rule = Rule::Patch;

    for commit in history {
        if commit.message.contains("BREAKING CHANGE") {
            rule = Rule::Major;
            break;
        } else if commit.message.contains("feat") {
            rule = Rule::Minor;
        }
    }
    bump_version(rule);
}

fn bump_version(rule: Rule) {
    let mut version = Version::get_current();

    match rule {
        Rule::Major => {
            version.major += 1;
            version.minor = 0;
            version.patch = 0;
        },
        Rule::Minor => {
            version.minor += 1;
            version.patch = 0;
        },
        Rule::Patch => {
            version.patch += 1;
        },
    }

    write_version(version);
}

fn manual(rule: String) {
    if rule == "major" {
        bump_version(Rule::Major);
    } else if rule == "minor" {
        bump_version(Rule::Minor);
    } else if rule == "patch" {
        bump_version(Rule::Patch);
    } else {
        panic!("Unknown rule: {}", rule);
    }
}

There we go, one source of truth for the rules, how to apply them to a semantic version, how to generate them from commit messages, and how to interpret a user's input.

But I see more repetition! What if we changed to a single commit.message.contains statement with a map from the keywords to the rule they represent?

When considering any refactor, it's crucial to think about the goal. In the case of MOIST, we have to ask ourselves, "what is the truth we're trying to protect?" Is it true that "breaking changes" and "features" should always be determined the same way? No! In fact, this implementation is not consistent with Semantic Versioning yet, and the two branches will eventually diverge further! Coupling these two pieces of information would increase rigidity without protecting a single source of truth, so we should not combine them.

What about the user input? Surely we should factor out the string-to-rule map and only have a single function call location! Let's try the same test—what is the single truth we're trying to protect? Is it "every rule should be determined by only a single string?" That feels more like an implementation detail. In fact, if we add a prerelease rule, we'll need some additional info to select a prefix. This change feels like it would be a reduction in repetition without a clear goal—it would bind the separate rules together, making it harder to change just one without an obvious benefit.

MOIST can get subjective and be a bit mushy—as can any coding practice, but it tries to draw a line between harmful repetition and benign code. The key to successfully applying any "best practice" is understanding the true goal and always keeping that in mind.

RAINY

Cake aside, there are a couple more worthwhile goals semi-related to DRY. I'll attempt to shoehorn them into more acronyms semi-antonymic to DRY. "Reusable Abstractions, Ideally Not Yours" is another way of saying "don't build when you can buy." It is far more efficient for one person to solve a problem and share that solution than for hundreds of people to solve it independently. This practice is applicable at many scales but achieves another one of the fundamental goals that DRY tries to stand for.

The best example of this is the open-source community. Rather than re-do work that someone else has done, you can build on top of what exists. Likewise, you can share solutions to problems you've faced so that others don't need to waste future effort—and you multiply the impact of your work. RAINY is like taking "don't repeat yourself" and applying it across our entire community. More like "let's not repeat ourselves."

As with everything, there is a balance to be struck. We don't want a MONSOON, where "Maintaining Open source is a Nuisance So Others Often Neglect it." 🤪 Basically, as a consumer, installing dependencies is a pain, and keeping them up to date is even more of a pain (you should be using Renovate to mitigate that, though). On the other side, staying on top of issues and pull requests as a maintainer is burdensome and often thankless. This level of effort on both sides can lead to relatively simple bugs never being fixed.

Balancing this is difficult, and I think a conversation about the good and bad of open source can go far beyond this article—so I'm going to leave it there. However, if you'd like a post titled either "Do You Really Need that Dependency?" or "When to Open Source it", let me know.

FRESH

"Functions Read Easier in Short Hunks." Yes—my acronyms are getting more and more unhinged and also straying further from DRY. I'm not sorry.

Often, reviewers use DRY as a code word for "make that into a function." Code readability is vital for maintainability, and an easy way to improve readability is to split large functions into several smaller ones. This way, a reader can get a high-level understanding of what the code is doing just by reading the function names in order—and they should read a bit like prose.

Of course, there's a tradeoff to be made here; a function call is not always easier to read than the statements that make it up, and calling functions can often have performance penalties. Still, as a goal, refactoring to readability is far more valuable than refactoring simply to reduce duplication.

A Test

Here's a set of questions I suggest you ask yourself next time you're wondering if you should be repeating yourself or not:

Are there two separate sources of truth for a single concept? If yes, try to unify them.
Is this code solving a general problem or something specific to me / my work? If it can be generalized, consider publishing a reusable module (whether open source or in a private/corporate location).
Can I read through this code and understand what it's doing without stopping to inspect it? For this one, ideally, have someone who didn't write the code answer the question in a code review. If it's hard to parse for humans, consider abstracting away details that aren't needed, like with functions.

Hopefully, all of my acronymic nonsense has provided you with enough alternatives to DRY that you never use it again. Remember, not repeating yourself is not a valuable outcome of refactoring. Instead, try keeping a goal in mind like improving correctness, benefitting the community, or making the code easier to read. Now go enjoy some cake; you've earned it.

Was this post super helpful to you? Tip me on GitHub, Patreon, or Ko-Fi.

Have a question or comment about this post? Leave it in the discussion thread on GitHub!

Want to be notified of future posts? Watch releases in the GitHub repo or follow me on Twitter.

Have an idea or request for a future blog topic? Drop it in the GitHub discussions under ideas.

New Year, New Website

Dylan Anthony — Sun, 02 Jan 2022 05:00:09 +0000

I really like blogging. Sharing my discoveries and thoughts with anyone interested is one of a few ways I give back to the developer community. Having a topic also gives me an objective—and learning with an objective is the only way I can learn anything. So why haven't I posted in over eight months? That's what I hope to answer here, as well as share the ideas I've had for picking back up where I left off.

When debugging any problem, the first question to ask is "what changed?" The system (that is, the way my blogs get produced) is wildly complex, but I've come up with three components to look at:

The Content—has there been a significant change in the topics I want to write about?
The Writer—have I changed? Do I have less motivation to blog?
The Process—did something change about how I write, which holds me back?

The answer is, of course, yes to all of the above, so let's dig in one by one.

The Content

The old adage says, "write what you know," and I sure have. I had the goal in my day job to develop the best web APIs possible, which meant using the best tooling available. I wanted to write an OpenAPI server in Rust and host it on serverless technologies, so I explored all the options I could find in that space for a while. However, I pretty quickly ran out of options, with none of them entirely fulfilling my needs.

To make matters worse, although some of the solutions came close, none of them would be ready in time to use for the project I had in mind. As a result, my motivation to write about them quickly decreased. So the content wasn't technically changing, but its relevance to me was.

However, this wasn't the case for the whole of my eight-month hiatus. I worked on a side project that needed a new web server, so I decided to try hosting it on Microsoft Azure. The experience ended up being pretty terrible, but I never brought myself to finish any of the blog drafts about it. Recently, a new OpenAPI web server framework for Rust has appeared—Poem. It's an incredible new entry that seems to satisfy most of my needs and could be a real contender for the "FastAPI of Rust" crown—but still no blog post.

What gives? Clearly, even though the content was an issue early on, it's not the only thing that's put a pause on my writing.

The Writer

As I hinted earlier, my priorities have definitely shifted over the year. It started with changing projects at my old job and accelerated when I switched companies entirely. Not working on Rust-based APIs during the day definitely lowered my motivation to write on the topic, but it wasn't the only cause.

I'm not sure enough to diagnose this, but switching jobs also reduced how much I work on side projects and contribute to open source. It could be that I'm more satisfied in my day-to-day work and don't need to seek out additional accomplishments. On the other hand, it could be that the increased code-writing of the new position leaves me with less energy to do it after hours. Either way, I think the solution here is to write on topics that require less coding in advance.

The Process

Here it is, the moment you've all been waiting for; why did I rewrite my website? I write my blog posts in Markdown, and that hasn't changed. It's a lovely, straightforward format that is supported by Dev.to (one of the places I post) as well as my note-taking app. What has changed is the way I convert that Markdown into a website for your viewing pleasure (and a place to point canonical links).

I used to use Next.js with many addons so complex that I can't describe them. I can, however, explain how that complexity caused me pain and made me much less likely to post.

First, it's essential to understand that I cannot leave things out of date. Am I leaving performance on the table and thereby wasting energy every time my site builds or loads? Is my site horribly broken for some users because of a bug? Is there some security vulnerability that could actually impact my readers even though my site is essentially static? These questions constantly plague me as long as dependencies are out of date. I did not have time to read and comprehend the dozens of updates per month that Renovate indicated were available. Beyond that, my JavaScript package manager of choice—Yarn—just didn't work with some dependencies (e.g., the latest TypeScript and Next.js). Switching to npm would be too painful, so I had to test every dependency update manually, which, again, I simply couldn't justify the time investment for. Plus, an update to one dependency would often break a seemingly unrelated part of my website, and I'd have no idea why or how to fix it.

Another problem with my website that I couldn't ignore was JavaScript itself. Even though my content is static, my combination of tools could not convert it into plain HTML and CSS. This, again, meant wasted energy every time someone loaded the site, and their browser had to download and run some useless code. These aren't big problems; the performance difference is insignificant on modern machines. Still, it felt awful to have a bunch of stuff happening that didn't need to happen.

What I wanted was a way to convert my Markdown into pure HTML and CSS. I wanted the syntax highlighting for my code blocks to not be done on the fly with JavaScript but instead rendered once at build time and shipped as styled text. And most of all, I wanted way fewer dependencies, ideally while avoiding the npm ecosystem altogether. I got all of that with Zola.

Zola is now the only dependency I need. It takes my Markdown content and converts it into a purely static (HTML and CSS) website. The documentation is excellent, the build process is speedy (4x faster on Vercel than Next.js builds were), and I'll never again have to dread a backlog of updates. It does come with some downsides, like not having Tailwind, but it's well worth the tradeoff for the relief I feel. The whole conversion process only took me a handful of hours, and most of that was deciding on a theme to start with.

Conclusion

Many reasons can (and did) cause burnout, and many little changes can help relieve it. Will my combination of changes get me back to posting again regularly? Only time will tell. One thing is for sure though, taking the time to "debug" my own burnout sure did make me feel better about it. I hope to talk to you all again soon.

Was this post super helpful to you? Tip me on GitHub.

Want to be notified of future posts? Watch releases in the GitHub repo or follow me on Twitter.

Have an idea or request for a future blog topic? Drop it in the GitHub discussions under ideas.

Best Supported Serverless Languages

Dylan Anthony — Thu, 29 Apr 2021 00:56:33 +0000

Cover image created by me using the Community TypeScript Logo, the Go logo, the .NET logo, and Ferris the Crab.

I'm a big fan of serverless functions, but my perspective is usually limited to what will run on AWS Lambda. So I took a tally of the supported languages on the serverless platforms that I've heard of.

Caveats

I didn't go searching for new serverless providers, these are the ones I already knew about.
These are only the languages prominently listed with tutorials or examples on the provider's website. You'll notice, for example, that Rust on AWS Lambda is not included here even though I've written several blog posts about it!
This is not an endorsement or review of any languages or providers, merely a count of official support. If you have specific questions you want answered (or a slightly different list ranking related things) let me know.
Some of these entries have their own specific caveats, they are indicated with a number at the end of the entry. Details on that number are at the bottom of the post.
I have not used all of these language/provider combos, I'm assuming if they are displayed prominently on the site then they are possible to use effectively.

Big Takeaways

The best supported language is JavaScript, followed closely by TypeScript. No shock there.
Python and Go are tied for third place in availability.
Cloudflare Workers are by far the most flexible in way of supported / documented languages with 12—though many of those rely on cross-compiling the language to JavaScript.
AWS, Azure, and GCP are all tied for second at 8 supported languages. Realistically, these are the most flexible platforms as those 8 languages don't require transpilation (except for TypeScript but... you know...).
Rust has an explicit tutorial on the Azure website! 🥰 I'm suddenly more likely to use Azure than AWS in my next big project.

The List

Here is the list of all the supported languages with the platforms that support them. Reminder, a number in {} after the language means there's a caveat, check the bottom of the post for it!

JavaScript

Netlify
AWS
GCP
Azure
Vercel
Cloudflare

TypeScript

Netlify
Azure
Vercel
Cloudflare
AWS {1}
GCP {1}

Go

Netlify
AWS
GCP
Azure {2}
Vercel

Python

AWS
GCP
Azure
Vercel
Cloudflare {3}

Ruby

AWS
GCP
Vercel

Java

AWS
GCP
Azure

C

AWS
GCP
Azure

PHP

GCP
Cloudflare {3}

Powershell

AWS
Azure

Rust

Azure {2}
Cloudflare {4}

C and Cobol

Cloudflare {4}

Kotlin, Dart, Scala, Reason / OCaml, Perl, and F

Cloudflare {3}

Language-Specific Caveats

The runtime is for Node.js and there is no TypeScript example, but TypeScript is fairly easy to compile to JavaScript.
There is a tutorial for using this language, but not an official runtime.
Supported by compiling to JavaScript, some language features / libraries may not work.
Supported by compiling to WebAssembly, your mileage may vary.

Was this post super helpful to you? Tip me on GitHub.

Have a question or comment about this post? Leave it in the discussion thread on GitHub!

Want to be notified of future posts? Watch releases in the GitHub repo or follow me on Twitter.

Have an idea or request for a future blog topic? Drop it in the GitHub discussions under ideas.

Running GraphQL on Lambda with Rust

Dylan Anthony — Wed, 21 Apr 2021 00:53:24 +0000

Cover image created by me using Ferris the Crab, the GraphQL logo, and the AWS Lambda logo.

Taking a break from OpenAPI for a bit, I decided to explore the primary alternative for web API development—GraphQL. I've been pleasantly surprised so far at how easy it's been to write a GraphQL API using Rust, much easier than doing so with the OpenAPI tools I've tested so far.

The Options

There are two real contenders for GraphQL crates: async-graphql and Juniper. Having read briefly through both sets of documentation, async-graphql seemed like the more feature-rich and well-documented option, so that's what I went with. If you'd like me to take a look at Juniper and do an in-depth comparison, please leave a note in the discussion thread!

The good news is that both solutions are fairly feature-complete using stable Rust, and neither is tightly coupled to a web server! You'll know that this is perfect for my use-cases if you've read my previous posts where I repeatedly jumped through hoops shoe-horning a web server into a serverless environment.

The Setup

As I mentioned, I went with async-graphql as the primary crate to build my experiment with. I used a similar AWS CDK setup as in my previous post since it was the easiest and most robust way to both test Rust lambdas locally and to deploy them to AWS.

Interested in seeing how to apply this CDK setup to Python applications? Upvote this idea thread.

I used cargo-make to orchestrate a bunch of tasks (I highly recommend you check it out if you find yourself using Makefiles with Rust). Postgres via docker-compose is my go-to database, though you can easily swap in MySQL or MSSQL since I'm using SQLx to run queries. Throw in a bit of tracing following the examples of the fantastic Zero to Production in Rust book, and we've got ourselves a pretty solid development platform for building our API.

The Code

You can explore the codebase yourself by looking at this repo, but here's quick rundown. The code consists of two main pieces: the handler and the API.

The handler is the same sort of thing I've written for previous posts—it's the bridge between the lamedh_ crates (for interfacing with AWS Lambda) and async-graphql. All of that is in its own handler.rs file, so it could be easily split out into a crate in the future. It's a bit messy with all the generic-handling, but if you were to couple it to your specific schema then the code would look a lot cleaner.

Would you be interested in a lamedh + async-graphql crate so you don't have to write your own handler? Let me know in the discussion thread.

The API code is fairly simple, for the most part you can just follow the docs. There are a few things that I struggled to find the answers to—and a few things I haven't yet solved—but generally you just start writing structs and resolver functions, and your GraphQL API comes to life.

I'm considering putting together a full tutorial on how to build a GraphQL API with Rust and host it with serverless technologies. It would cover auth, subscriptions via websockets, authenticating the GraphiQL UI, the works. If that's something you want, drop a note in the discussion thread and tell me how much, if anything, you'd be willing to pay for such a resource.

Conclusion

Building a GraphQL API in Rust using AWS Lambda to host it is way easier than trying to build the equivalent with OpenAPI. The main problem is the lack of an end-to-end guide covering everything you need to know. The async-graphql docs are great, but terse (as are the SQLx docs), so more are needed to make this setup more accessible. There are of course some major differences between GraphQL and OpenAPI (upvote this idea if you want a full comparison), so you'd have to be sold on GraphQL already to use this solution. All of that said, this is definitely my favorite solution for making APIs with Rust that I've tried so far.

Was this post super helpful to you? Tip me on GitHub.

Have a question or comment about this post? Leave it in the discussion thread on GitHub!

Want to be notified of future posts? Watch releases in the GitHub repo or follow me on Twitter.

Have an idea or request for a future blog topic? Drop it in the GitHub discussions under ideas.

Replacing FastAPI with Rust: Part 6 - AWS Lambda

Dylan Anthony — Thu, 04 Mar 2021 00:58:32 +0000

Cover image created by me using Ferris the Crab, the Rust logo, and the FastAPI logo.

The last blog post was a bit long, so I figured I'd take a bit of a break and tackle a shorter topic. In this post, we'll take a look at two different methods to deploy our Rocket application to AWS Lambda: the SAM CLI and AWS CDK.

Prerequisites

If you want to follow any of the instructions below, you'll first need an AWS account to experiment on. I did this on an account where the free tier has already expired and I didn't accumulate any charges so you should be able to do this for free, but I make no guarantees.

Once you have your account set up, you'll need to generate some IAM credentials, install the AWS CLI, and configure it to use your credentials.

SAM CLI

If you've followed along with previous posts, you'll know that I've been using the SAM CLI in order to test the application locally. This allows me to emulate the AWS environment for testing, which is invaluable, but it's not the only purpose of the tool. It can also be used to build and deploy the application to AWS.

To start off, you'll need to do a sam build before each deploy, just like was necessary before doing a sam local start-api to run locally. If you've run this before with the same settings I have in the experiments repo, you'll know that it feels like it takes forever because you get no feedback on the build process. There might be a way to fix this, but I haven't found it yet.

After the app is finish building, you'll want to do a sam deploy --profile <aws_profile_name> --guided (omit the --profile if you've only got one AWS profile configured) and follow the prompts. This will create a samconfig.toml file with some additional required information for SAM. In the future, to redeploy, you can omit the --guided because you already have this file. Now feels like a good time to note that if you've copied the template.yml from my repo previously, you'll want to update it from the one in the SAM experiments branch because it was missing a required attribute previously.

The sam deploy command is going to deploy, it's that easy! Answer any questions it has and wait for it to be done. Now you've got an API Gateway URL you can hit to talk to your API in the cloud! But wait... where's the URL?

I had to do quite a lot of digging to figure out where to call my freshly hosted API. Eventually I found these docs which informed me the url should be something like https://{restapi_id}.execute-api.{region}.amazonaws.com/{stage_name}/. The {restapi_id} is replaced with an ID I found in the API Gateway console, {region} is what I configured with sam deploy --guided, and {stage_name} appears to be Prod by default.

With that URL, I was able to test out my API and verify that everything was working. Now it was time to tear down that test infrastructure, so it didn't sit there in my account now that it was unneeded. Unfortunately there doesn't seem to be a sam command for that, so I had to go into the CloudFormation console and tell it to delete everything for me. There was one slight snag with that as I had to find the S3 bucket it created and empty it myself before it would delete.

Overall the sam experience was fairly painless. I'd say the toughest part was figuring out which URL to call once everything was deployed.

AWS CDK

AWS CDK is a tool that allows you to define your AWS infrastructure in a programming language instead of using CloudFormation syntax (e.g. in template.yml for sam). The languages available listed in the docs are TypeScript, JavaScript, Java, Python, and C#. The CLI tool also seems to indicate that F# is an option, so that might be worth looking into if it interests you. I went with TypeScript for this project because:

It's a language I know.
It's strongly, statically typed.
The CDK CLI requires node anyway.

Before I walk through how I got to a working solution (as it was fairly difficult to piece together all the answers), feel free to take a look at the CDK branch of the experiments repo to see the result for yourself. The relevant bits are:

The Makefile which contains rules for building and deploying via CDK
The various .json files which configure the Node/TypeScript things
The cdk directory which contains the actual infrastructure code

Here's the high level process I followed to get there:

Follow these instructions to install CDK.
Create a directory to keep your infrastructure code in and initialize it with cdk. I did mkdir cdk && cd cdk && cdk init app --language typescript.
Delete the TypeScript handler code since we won't be needing that.
Follow these instructions to add the necessary components for Lambda and API Gateway.
Take some inspiration from this blog post for setting up Rust builds with CDK.
Find and use apigateway.LambdaRestApi in CDK to make the infrastructure setup way simpler.

With the CDK code all set up correctly, the steps to deploy are:

cdk bootstrap --profile <aws_profile> to set up some basic infrastructure (you only have to do this once).
Run make deploy which builds the binary, copies it into a new folder and names it "bootstrap", then runs cdk deploy. CDK has been configured to upload this folder to Lambda to use as a runtime.

The solution is very nice once you put all the pieces together. I strongly recommend copying most of what I have if you're going with a straight proxy (all requests from API Gateway go to one lambda handler). The clear benefits over SAM are:

Using a typed language instead of YAML means you get autocompletion and don't have to fumble around as much through documentation to find what you're looking for.
Using normal cargo build then copying over the binary means you get normal cargo outputs instead of a blank screen while building.
CDK comes with a nice cdk destroy command which will tear down most of what it created (just not the bootstrap stuff).
It's easy to break up your code into reusable components. You can even create normal packages with whatever language you selected and share those to other projects, so you don't have to copy/paste infrastructure. I'll definitely be doing this in the future.

The only question left to answer was how to run the function locally. Luckily AWS provides official and easy instructions for running a CDK function using SAM, and it works super easily! In my repo, I can do make local which basically builds the app, runs cdk synth > template.yaml, then runs the same sam local start-api I was already doing.

There is another option for running locally which you can try, but I had very little success with it. There is a project called LocalStack for emulating AWS services via Docker containers. I've used it in the past for doing integration tests with SQS, however their API Gateway support is pretty atrocious. You have to dig around in their docs for a while to find the long, custom URL you need to call which requires copying and pasting the API ID every time. All the effort didn't seem worth it to me, but if you want to try it out yourself I found this GitHub repo which should get you started in the right direction.

Conclusion

AWS CDK definitely seems like a great option for deploying functions, as well as managing any other infrastructure you need (like maybe an RDS database?). The only real downside is that you have to add yet another tool on top of SAM and a bunch of dependencies for whatever language you choose to write in. Too bad there's no Rust CDK option... yet.

Have a question or comment about this post? Leave it in the discussions thread on GitHub!

Want to be notified when the next part of this series is released? Watch releases in the GitHub repo or follow me on Twitter.

Have an idea or request for a future blog topic? Drop it in the GitHub discussions under ideas.