DEV Community: Zach Zaro

The 2024 Web Hosting Report

Zach Zaro — Tue, 20 Feb 2024 20:05:26 +0000

For a certain breed of web developer, the debate over the optimal hosting choice for web applications is akin to navigating a minefield. Delve into a decade's worth of discussions on Hacker News, and you'll find a battleground of opinions, each with its own set of staunch defenders and detractors. The truth is, the diversity of web projects out there means one size does not fit all. Whether it's a matter of budget, learning curve, scalability, or control, the "best" hosting solution varies wildly from one scenario to the next.

However, amidst this convoluted terrain of options, it's crucial to recognize that the stakes of making a poor hosting choice are alarmingly high. Opting for the wrong hosting provider can be the digital equivalent of building your castle on quicksand. Slow website load times, frequent downtimes, and inadequate security measures can not only tarnish your brand's reputation but also significantly dent your revenue. Imagine the horror of your site crashing during a high-traffic event or sensitive customer data being compromised due to lax security protocols. These are not mere inconveniences; they are potential catastrophes that can derail your online presence. In an era where consumer trust is as fragile as glass, a single slip-up in hosting can lead to a domino effect of lost customers, negative reviews, and a tarnished brand image that can take years to rebuild.

We’re publishing this report to demystify the complex landscape of web hosting and deployment. Our goal is to illuminate the myriad of considerations that should guide your decision-making process, helping you navigate through the noise to find a hosting solution that aligns with your project's needs, without falling into the trap of making a catastrophic choice. By understanding the nuances of this decision, you can set your web application up for success, ensuring it is both robust and resilient in the face of the internet's ever-evolving demands.

Hosting on a VM

A Virtual Machine (VM) is a “virtual” server running on top of a physical server. It’s preferred to a physical server for a few reasons, most importantly that you can overprovision VMs onto actual servers, and because the “virtual” nature makes things like migration, backup, and automation easier when compared to a physical hard drive and motherboard in a rack.

A VM is the original “hosting” product of the cloud era. Over the last 20 years, VM providers have come and gone, as have enterprise virtualization solutions such as VMware. Today you can do this somewhere like OVHcloud, Hetzner or DigitalOcean, which took over the “server” market from the early 2000’s. Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft's Azure also offer VMs, at a less competitive price. In their case, the VMs are either a building block for other services or the value is in the ecosystem. See the section on public cloud below for more.

To manage a VM, you can use something as simple as just manual actions over SSH, or can use tools like Ansible, Hashicorp's Packer and Terraform or other automations. For an app where there is minimal load and security/reliability concern, VMs are still a great option that provide a lot of value for the buck

These are the most flexible product, but require a lot of knowledge and work to “do them right” and while you can scale very far in terms of requests per second on one machine these days, in practice there are other parameters that matter for scale (e.g. reliability and compliance and microservices) and which drive teams to other options for their hosting needs in 2024. But you’re not a true expert until you’ve done this from scratch a few times, and everyone should play with VMs at some point…

Ideal for old school infra pro’s looking to spend as little as possible

Hosting on a VM using a deployment layer

This category dates back to the “control panel” era of tools that are still used on shared VM hosts, such as cPanel. These tools automate common functions such as managing authentication, deployment of databases, etc.

The modern iteration of these tools has taken the developer experience learnings from the Platform as a Service (PaaS) category, and will bring them to your own VM, giving you your own personal PaaS. Example of this include Dokku, Coolify, Caprover, Cloud66 and many more!

These tools are great for making the VM hosting experience easier - upgrading, deploying, maintaining all get well-supported pathways, often with documentation. In some cases, you can buy support around the software as well. Some of the commercially-supported options can scale further, but in general the biggest limitation of this category is that you remain limited to a single VM for hosting, and so for most teams they do not choose this path for anything beyond a small project.

Ideal for SaaS soloprenuers, students, or side project hackers

JAMstack/SPA Hosting

At some point after AWS released S3, they made it possible for you to host a website in a bucket. This was one of the first “serverless” products. You didn’t need a VM at all!

Single-page applications (SPAs) existed before S3, but given that you still had to set up, scale, and maintain servers using something like Apache or NGINX in order to serve them, the advantages for “Ops” or “DevOps” were not so different to running a “real server” with a language like PHP, python, or Java.

After S3, people got excited about the potential for a truly simpler deployment experience, where you could scale infinitely without worrying about it - it was the cloud’s problem. And, it was super cost-effective when you put a content delivery network (CDN), which is basically an internet-wide cache, in front of your site. This drove a wave of single-page app design and delivery, which had mixed results but was undoubtedly popular.

One thing that smart teams noticed with the new “serverless” paradigm for their frontend was that it was now possible to easily deploy many versions of your site, for internal testing, incremental rollout, and other reasons. The rise of “preview sites” was initiated in earnest by Netlify in 2014 and was truly catapulted by Vercel in the late 2010’s.

The term “JAMstack” has fallen out of favor, but this style of hosting and the workflow it enabled with dynamic previews is still a major trend on the internet today, and big part of how deployment and hosting work.

Purely static hosting and SPAs are still part of a lot of stacks, and if you’ve got one in yours this is still a great option for that part of the system. But if you do server-side rendering or want to use a language besides JavaScript/TypeScript for your logic, other choices will make more sense for you.

Ideal for client-side applications for the web that are content-heavy and have a lot of interactivity

Functions as a Service / “Serverless”

The first product that popularized the term “serverless” was AWS Lambda, which is both the prototypical and archetypical function as a service provider. It also has a great name, which pings back to its envisioned place in the cloud of the future. In computer programming, a lambda, often referred to as a lambda function or lambda expression, is a concise way to represent an anonymous function, which is a function without a name. The concept originates from lambda calculus in mathematical logic and has been adopted by many programming languages, each with its own syntax and characteristics.

The definition of serverless is a bigger topic than we will cover in this post, but in general the 2 key aspects are that it scales to 0 in a request or event based timescale, and that there is no access to the underlying operating system, only to a well-defined safe runtime within it. In Lambda’s case, the original runtime was a javascript based one, which has now expanded to many languages including python and Java.

Serverless functions are now offered by many cloud providers, as well as having options like OpenFaaS, Knative, Apache's Openwhisk and more from the open source community that run in environments ranging from one server all the way up to globally replicated private clusters.

These are a great choice for a variety of uses cases. Simple API endpoints on the web, simple event-based microservices, edge functions that run close to users (often stateless), and more are the most obvious fits. But given their operational simplicity, wide availability, and simple operational mechanics - they are often seen as the “hammer” for a lot of “nails” in stacks across the startup and enterprise software world.

We see some great results from using these in conjunction with frameworks such as SST or Serverless, and also some real spaghetti from people who organically proliferate 100’s of functions over time and lose track of how they relate to each other or how to update them safely across time and service. Buyer beware!

Ideal for small bits of backend business logic integrated into a larger system, for example edge functions that need to access a 3rd party API with a secret key required

Backend as a Service

Backend as a Service (BaaS) goes back to early 2010’s with companies like Parse and Firebase. These products integrated everything a backend provides to a webapp in a single, integrated package that makes it easier to get started and enables you to offload some of the devops maintenance work to someone else.

Today, this ecosystem is going strong with new providers like Hasura, AppWrite and Supabase powering millions of projects. There are a few reasons people choose this style of hosting, especially if they are more comfortable with frontend development. BaaS lets them set up a database in a secure way, expose some business logic on top of the data, and connect via a dev-friendly SDK from their app or website code to save data easily. These modern tools build a blend of managed database with curated plugins such as authentication, great admin dashboards, and function as a service type capability - all in one package, and often offered as a integrated hosted service.

There is some overlap here into the “no-code” or “low-code” world, as sometimes the same teams will hook tools like Zapier up to the BaaS in order to integrate with third parties. For small projects this can lead to superhuman productivity! But over a certain line it can become a mess of complexity where it’s hard to track down where data lives and where it is mutated.

For solo hobby projects by frontend devs or beginners or small teams that just need to offload a bit of state to the backend but don’t have a ton of server-side requirements - BaaS is a perfect solution. It does seem that there has been a rising trend for these tools to do more - for example a lot of startups are building a large part of their backends on Supabase over the last few years. Those that end up with a high percentage of server-side business logic will often “graduate” over to a more traditional setup for their server as they grow, and migration can be painful and time-consuming if there’s too much logic spread out over different functions. We think these are not the right choice for building a SaaS software business or enterprise software product on top of.

Ideal for client-heavy applications that need to combine a function as a service with a database, or just need to synchronize state between clients but don’t need server-side logic

Platform as a Service

Platform as a Service (PaaS) is a term that captures tools built for developers to use where they don’t need devops engineers to deploy to the cloud. Heroku was the “category-defining” product here, and modern successors include Vercel, Render, Railway, Replit, platform.sh, Aptible, Northflank, and more. We have to say that there are some amazing products in this category. Developer Experience is at a premium!

There are many great reasons to use these products:

The best developer experience - it’s easy to get started, there’s minimal maintenance work, and common use cases are covered fully. You don’t need to do a lot of magic to get a database hooked up to your server, or to integrate a common service like DataDog into your app. Last, deployment and hosting are integrated, with CI/CD built into the PaaS
Great community and documentation - other devs have gone through the same experience the same way. On a VM, there are so many ways to do things that it’s hard to find the exact same issue on stack overflow (or ChatGPT). But in a PaaS - there’s probably a doc or forum of your same stack on the same platform that can help you through the issue you’re facing
Easy to get started - there are tons of resources for “starter templates” that support popular frameworks and platforms that you can use directly or modify into your project to make it easy to get going
Low-end friendly pricing - the economics of shared infra, commercial support from the provider, and a focus on user experience mean that you can get started quickly and cheaply, often for free. The premium price you’re going to pay to scale is a problem to worry about tomorrow if the project works. Ultimately your business is not hosting, it’s whatever your software does for your users - go focus on that

There are 3 big reasons why people outgrow a PaaS:

Cost - you pay a premium for the developer experience compared to raw cloud costs for CPU/memory if bought directly. This is especially true when you consider cloud credits/discounts/spot prices etc, and the numbers get big as your scale
Compliance - AWS/GCP/Azure are the most well-audited and trusted environments on the planet. There are some reasons folks get paranoid - e.g. Wal-Mart does not want their vendors storing data on AWS. But overall, they are 3rd party certified more than any other hosting option and can serve customers all the way up to federal governments with confidence at the data center layer, which is pretty amazing!
Extensibility - Heroku had 4 first party services (e.g. Postgres, Redis, Dynos) and 100 or so extensions, with varying degrees of true integration. Modern PaaS have more, and of course over the internet you can integrate them with a huge variety of services, but when you look at the variety of services available there’s no comparison between PaaS and AWS/Azure/GCP. The big clouds offer secure, deep integrations all the way down to things like IAM in ways that cannot be patched together easily and still be enterprise-grade.

These three categories are unlikely to change in the near term as reasons why teams outgrow a PaaS. The bottom line is that the best engineering teams in the world are either in the public cloud or are hosting in their own datacenters and building their own tooling. They’re not paying a double premium for a black box that they cannot control.

For small teams, side projects, personal or student use, or other use cases where simplicity is paramount - a PaaS is the right hosting and deployment choice. For anyone else, these are a great place to get started but are not the final destination.

Ideal for small or new software teams, students, development agencies that serve small clients

Containers as a Service

This is a new category that sits between a Platform as a Service and a Function as a Service. Inspired by the great feedback about the developer experience for “serverless” but stuck with some of the limitations - the major cloud providers expanded the category with products that could run “real” linux containers. This means that they can run legacy apps as well as anything else that can be packaged as a container - a much larger footprint than the Function as a Service with a much smaller migration delta.

Some of these products also integrate deployment pipelines directly, making them almost as easy to use as a PaaS and blurring the lines. Generally they are still distinguished by not offering prebuilt integration with other services like databases or message queues, and by not having the same developer experience across different environments and making things like previews very easy.

Examples for products in this category are: Google Cloud Run, AWS App Runner, Azure Container Apps. Each has different scalability, cost, and integration trade-offs.

These are a great choice if you’re already running services or hosting apps/sites in a particular cloud, and need to deploy something that doesn’t fit into that existing tooling. They can be placed into a VPC and add new services, quickly and easily.

Ideal for teams who have made an existing investment into a cloud provider, and are running single-service applications that will not be updated often, or which are not production-critical

Software Creation Platform

We mentioned Replit in the PaaS category - and in some ways their deployments tool is an equivalent of the rest of the PaaS category - an easy way to deploy and host without any cloud account or configuration needed.

Additionally, Replit is an example of a broader and more unique category of software as well, one that does not have a well-defined name but which is defined by having an integrated solution for both development and deployment. Usually, this involves having a Cloud Development Environment with a IDE, a dev server, and then an integrated deployment and hosting option that runs the server or serves a single-page app and other assets via a CDN.

Replit is the category leader here, but other products in this space include: Glitch, Codesphere, StackBlitz. Coherence fits here as well, with our “Workspaces” Cloud IDE. We’re also the only option where the PaaS is replaced by an Internal Developer Platform.

These tools are great for a few reasons:

Developer experience - you’re able to build, test, deploy your app all in one place, with one configuration. This means that new team members can get up to speed quickly, new projects are easy to start and are consistent with other projects, docs are maintained by a third party instead of internally
Environments are ephemeral across the SDLC - per-branch previews for JAMstack are one thing. Being able to spin up usable dev/test/prod environments easily, and deploy different versions of the app to different environments easily, can be a game-changer for workflows
Integrated tooling - no more “works on my machine” or fighting with webpack or docker locally. It just works

So far, the only place these are a preferred choice for hosting is for education, interview process for engineers, and some edge cases internally at enterprises. We think that this is an interesting space to watch, especially as it matures. Try one of these products out if you’re not familiar with them!

Ideal for developers who are writing complex backend applications but who have no devops experience and are not able to hire a consultant to support them

Container Orchestrators: k8s/docker swarm/mesos

The single most important development in hosting since the invention of EC2 is defined by its own 3-letter acronym: k8s. Kubernetes has won the “container orchestrator” space, becoming the default way that teams across industries are managing their compute nodes and scheduling their workloads, from data pipelines to web services.

In the beginning, there was docker. In 2013, building on linux internals, docker packaged containers for mass adoption and made it easy to share a complete runtime environment for an application across the network. Check out their first demo at PyCon 2013 (I was there!) At the time, serious workloads ran on something like Mesos, which was not “container-native” and had its own way of packaging and distributing software across a cloud or data center footprint.

Docker didn’t have a default way to run on multiple hosts, and so in the wake of docker’s explosive adoption there was a rush of different solutions offered for scheduling containers across a fleet. One of the first well-adopted solutions was actually called fleet - it was part of CoreOS, whose team went on to be very influential throughout the container revolution. This was in the systemd era, and was basically seen as a multi-host systemd. It was very cool and it worked great!

Docker itself intended to build the answer here, it was the deployment platform of the future and was going to be the heart of their business model. It was called “docker swarm” and unfortunately by the time it arrived there was a fragmented market of already-adopted solutions. Docker as a package format was a runaway winner, and Docker Hub (which became the foundation for the current Docker, Inc. business) was widely used. But the fragmented runtime market left room for a new entrant: (drumroll please…) enter Kubernetes. It was a “community-driven open source project” that was in fact funded and well-staffed by Google engineers, and rose quickly on the claim of being an open-source implementation of Google’s storied internal cluster management system called “borg.”

Docker swarm still exists, it still works, and some of these other container orchestrators are still hanging on, but for the most part, you’re using Kubernetes if you’re doing this stuff at work. Generally it's well-understood that kubernetes is hard to get right, and so most people use it via a managed provider like Elastic Kubernetes Service from AWS, Azure Kubernetes Service from MSFT, or Google Kubernetes Engine from Google Cloud. And there’s many more: Tanzu from VMWare, RedHat's OpenShift , the list goes on…

It’s also well understood that having a k8s cluster is not enough to make developers able to host their services - you need a devops team to work with them, using tools like delivery pipelines, Helm, kustomize, infra as code, service mesh, ingress, secrets management, key management - the list goes on! Developer Portals like Backstage, Port and Cortex have started to emerge to help manage some of this complexity.

In general, k8s is not a good choice for hosting, in and of itself. Even the managed variants are really helping you with the management of Kubernetes itself, not your application on top of it. Kubernetes is a tool for devops or platform engineers to then build a deployment solution for developers on top of. If you work at a company where you’ve got those resources, and they are capable of building that solution, and you’ve got time to wait, and they’ve got time to build, and you’ve got budget to buy all the pieces you’ll need to integrate and maintain to do that - congrats! Market-leading engineering teams have already done this work in many cases, and so maybe you already have this experience. If that’s you, k8s is a great solution for you. If not, stick to one of the other options outlined here.

Ideal for platform teams who are building their own internal developer platform and need a high degree of customization in order to meet their application requirements

Hosting in the Public Cloud

There are a huge variety of hosting services available on the large public clouds: Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, Oracle Cloud. These range from simple VMs that work very much like those described in the first section above, through more complex dedicated services, some of which are included in other hosting options below.

Infrastructure as Code (IaC) is an important part of any true hosting operation in the public cloud. Each of these platforms has their own IaC solution, e.g. AWS CloudFormation. But they also support popular open-source IaC tools like Pulumi or Terraform. A category of tools that also needs to be discussed is API gateways and other app-specific load balancers. There are applications for internal consumption, which can be called microservices if you have a lot of them. And often microservices use advanced networking options such as a service mesh instead of just the native private network offered by a VPC.

This topic is very complex, and is covered by entire disciplines: DevOps and Platform Engineering. Many teams take the various components above, plus a host of other tools, and build a “Developer Platform” for their company. Some of these are as simple as some bash scripts wrapping CLI tools. Others are complex apps in themselves, with a developer portal, service catalog, layers of security analyzers, IaC execution managers, compliance audit tools, workflow integrations into sprint managers like linear or Jira, engineering productivity analysis, and even more.

This category has blended with some of the Kubernetes options discussed in more detail above. The “raw” public cloud is a fit for the same types of organizations.

Ideal for backwards-compatibility with existing on-premise applications

Internal Developer Platform

Internal Developer Platforms are essentially a bridge between complex cloud infrastructure and developers who seek to deploy applications quickly, securely, and efficiently. By abstracting the underlying infrastructure, IDPs offer a simplified interface for developers, akin to what Platform as a Service (PaaS) provides, but with greater control and flexibility.

Some popular examples of an IDP are:

OpenShift: Red Hat's OpenShift is a prominent example of an IDP that extends Kubernetes with a developer-centric view. It integrates CI/CD, source code management, and automated scaling, offering a comprehensive environment for application development and deployment.
Qovery: Qovery simplifies the deployment process for Kubernetes by letting you declare your project's structure and dependencies, making it an excellent choice for startups looking to focus on development without worrying about infrastructure.
Cloud66: Cloud66 offers a suite of tools for container and server management, supporting both traditional applications and containerized workloads. It provides a clear pathway for organizations transitioning to containers and microservices.
You can also check out our own product, Coherence! Coherence supports a wide variety of integrated tools to provide a developer experience on top of your own cloud account. Read more in our docs.

IDPs vary widely in their approach to balancing usability, customizability, and the deployment footprint. Some, like Zeet and Flightcontrol, prioritize ease of use and rapid deployment, making them ideal for small teams and projects with tight timelines. Others, like Architect and Porter, offer deeper customization options, catering to enterprises with specific compliance, security, and scalability requirements.

The choice between using high-level managed services like Google Cloud Run and Kubernetes-based services is crucial. High-level services offer simplicity, lower costs, and managed infrastructure, but with Kubernetes, you gain unparalleled flexibility and control over your environment. This decision impacts not only the developer experience but also factors like total ownership cost, compliance, and maintenance.

Here are some great IDP use cases:

Startup Teams Outgrowing PaaS: Startups initially benefit from the simplicity of PaaS but often outgrow it as their team and application complexity increase. IDPs offer a scalable solution without the steep learning curve associated with directly managing cloud services.
R&D Teams in Large Enterprises: These teams need the agility to experiment without being bogged down by infrastructure management. IDPs provide a compliant and manageable environment, freeing them to focus on innovation.
Cloud Migration Projects: For organizations moving to the cloud, IDPs offer a streamlined path to migrate applications without the overhead of building and maintaining their own cloud-native infrastructure from scratch.

Internal Developer Platforms represent a significant evolution in how organizations deploy and manage applications. They combine the best aspects of PaaS with the flexibility and control of managing your own infrastructure, tailored to the needs of modern development teams. Whether you're a startup looking to scale, an R&D team pushing the boundaries of innovation, or an enterprise migrating to the cloud, IDPs offer a compelling solution that balances ease of use with the depth of control necessary for today's complex cloud environments.

By exploring the diverse landscape of IDPs, organizations can find the right balance of features, usability, and control to suit their specific needs, ensuring a future-proof foundation for their development efforts.

Ideal for software teams writing significant backend software with business logic, looking to deploy to the public cloud without building their own platform for delivery. A VC-backend startup, a new project inside the enterprise, a large open-source project with cloud credits for hosting - these are the best use cases.

No Code or Low Code [Bonus]

In many cases, no or low code tools are integrated with hosting themselves, and so you aren’t really choosing where to host - you’re choosing when to use these tools and when not to. There are also some open-source versions of these tools that you can self host them.

In general, these are great tools for a few use cases:

Semi-technical stakeholders who want to build a small business application
Devs who don’t want to write or maintain code for a particular project
Within certain ecosystems the data integrations and prebuilt capabilities let you go so much faster than “doing it from scratch”

For the first category, tools like retool, Zapier, Airtable are best in class and are used in a similar way to spreadsheets with much more power (which are still well-used and important - a large % of the world economy runs on excel macros).

For the second category, tools like bubble, Unqork, Glide are awesome (there are a lot more of these). But the risk is to go too far, and build something that really needs to be built at a lower layer in one of these tools. The providers of course want to push every use case, but in our view these are not a replacement for traditional software, and AI-assisted programming is a better path for dev augmentation than a restricted platform. In some cases though, with the right escape hatches and system design, these can truly be a 10x superpower type tool!

For the third, examples here might be analytics plugins in specialized databases like Clickhouse, data-transformations in places like your ETL pipeline using Airflow or Fivetran, or special integrations in your authentication workflow with Auth0 hooks and rules.

Ideal for internal tools, technical-adjacent folks who are not developers but who want to build data-heavy integrations that require data transformation or aggregation, prototyping simple input-driven applications without complex business logic.

If you've made it this far, thank you! We hope you found this report comprehensive and helpful. As you can imagine, this is a rapidly evolving landscape. We'd love to hear your feedback, feel free to send me an email at zach@withcoherence.com.

What comes after Heroku?

Zach Zaro — Wed, 31 Jan 2024 23:30:55 +0000

As the saying goes, “All good things come to an end.” And unfortunately that reality has now started to be true for the most amazing deployment platform of the last 20 years, Heroku. There are endless debates across HN, reddit, and twitter about why things are trending in that direction — for my part I’ll leave that to the scholars. We’re now seeing the question posed — what comes next?

What made Heroku Awesome?

If you had to choose one thing, it would be “git push heroku master.” This was a revolutionary integration of source control, CI/CD, and devops automation that hit just the right blend of opinion, power, and flexibility. The “12 factor” apps designed to fit onto it became a pervasive and still-thriving industry trend. Going a bit deeper, 2 other things stand out. The breadth of the free offering included common “batteries” such as database, cache, and metrics. And the depth of detail the team cared about borders on obsessive, from the early inclusion of SSL domains to the magic of buildpacks, made a wide variety of apps “just work” when they were deployed.

What is the “Next Heroku?”

For those who believe that an opinionated workflow over simple abstractions was the magic of Heroku, platforms like Northflank, Render and Railway are viewed as the places to go next. They take the best parts of Heroku in terms of simple pricing, configuration, and commonly needed services — and go further than Heroku in terms of baking in advanced features such as preview environments and CI/CD. In our view, these services cannot compete with the hyperscale clouds in terms of the breadth of services they offer, and so might struggle to serve more enterprise-scale teams, just as Heroku did.

Some people did not use Heroku for the Dynos, but rather for the shockingly good Heroku Postgres product. For them, services like Crunchy Data, MongoDB Atlas, Neon, CockroachDB, or PlanetScale are the “next Heroku.” At this point all of the major cloud providers also have a spectrum of awesome data persistence tools.

For folks who saw the demise of Heroku in its cost, the alternative is to go back to basics with VM-based hosting products like Hertzner. There, you can self-host using open source products on beefy hardware and you won’t get the rug pulled out from under you by a tech bigco anytime soon. If you highly value your (or your teams’) time, however, that’s not what the “next Heroku” looks like, and it’s not in the spirit of the question to accept that as the answer, in our view.

For others, Heroku did not pioneer solutions for the cutting edge, for example serverless or edge computing. For teams with a focus on these trends, many are interested in services like Fly.io, Cloudflare Workers, Vercel/Netlify, or Firebase/Supabase/Appwrite. These projects bake in Heroku’s developer experience magic and are creating such good products that they are pulling in projects that don’t care about being on the cutting edge, either.

Some of us were so burned by the lock-in we experienced with Heroku that the only solution for the next Heroku is something we own. If that’s you, something like Okteto, Ambassador, porter.dev or quovery is probably what you’re looking at as the “next Heroku.” A PaaS experience built on top of Kubernetes in your own cloud, ideally with multi-cloud capability, means that the community and your direct relationship with the cloud will ultimately protect you from getting pushed off your stack again.

The most interesting candidate for the “next Heroku” is replit. Heroku’s first product was an integrated cloud IDE and deployment system for Rails apps (Heroku Gardens). Replit provides the same magical integration from building to shipping, while keeping a beginner-friendly community and generous free tier to provide a home for side projects and new ideas. Heroku similarly was a magnet for learners, and in this way it’s clear that Replit is carrying some of the torch.

It’s important to note that Heroku isn’t gone. Salesforce has, despite what people say on the internet, invested tremendously in the platform and it retains best-in-class and unique features like Heroku Connect, the add-on marketplace, and a broad universe of integrations to other systems. Heroku just might be the “next Heroku” and it’s up to the web development community to define what comes next in the world of developer experience.

The ground truth is that there probably is no one answer to “What comes next.” The diversity of reactions and realities for technologists means that all of these projects have a home somewhere. At Coherence, we are building the tools that provide the most integrated and powerful “golden paths” to serve teams of professionals over the long haul on real-world enterprise-grade projects. We believe that a blend of the “in your own cloud” and “dev to prod integration” are what create the most leverage for engineers over the life of a project, and we’d love you to give our next-gen experience a spin at our site.

The 9 Principles of the Perfect Startup Engineering Stack in 2024

Zach Zaro — Wed, 31 Jan 2024 20:17:00 +0000

As a startup, a major factor in your success is how fast you can ship your product and iterate based on feedback from your customers. A well-chosen set of developer tools can be the key to moving quickly. The ideal stack should be easy to set up, affordable, and integrate services that your team already knows and trusts. In this article, we'll explore the nine principles that make up the perfect startup developer stack.

1. Easy to set up, free to get started, and affordable to scale

As an early stage team, you want to build a great product and not worry about your infrastructure time or cost. The best dev tools focus on ease of setup, provide free tiers to get started and scale without huge jumps from a cost perspective as your team scales.

2. Supports tools you already know and trust

You want to use tools that you already know and trust. Tools like Slack, GitHub, AWS/GCP, Linear/JIRA, Docker, and VSCode. Your dev stack should meet you where you already are and support databases like Postgres and Redis which are popular choices for startups.

3. Doesn’t require using a new framework or language

Your stack should be flexible enough to support the tools and languages that your team is already using. The rise of “Infra from Code” frameworks such as serverless.com and Encore are cool for new projects. But they require you to rewrite your application into a new format, or to have a greenfield project and be open to using novel tools or languages. You don’t want your team spending time learning a new framework or language when they should be focused on shipping your product to customers.

4. Optimizes cloud cost

As your team scales and you move beyond the free tier and/or your startup credits, you’ll want a stack that is proactively optimizing costs for you without compromising performance, security, or reliability. You’ll need visibility into your cloud usage and will want to use monitoring to identify areas where you can save money.

5. Supports key parts of the SDLC

Workflows that achieve industry best-practices will pay dividends in the long run and will allow you to align with critical standards like SOC2/HIPAA/ISO27001. The SDLC (Software Development Life Cycle) should include tools to enable the various phases such as coding, testing, deployment, maintenance, patching, monitoring, and updating.

6. Branch previews for the whole app

The next-gen development workflow offers branch previews for the whole app, allowing your team to preview changes before they are merged. This increases productivity by removing bottlenecks of a shared staging environment. Just as important, it improves quality by giving a dedicated place to run integration tests and do manual testing on this code in isolation from other branches, reducing the risk of introducing bugs to your next release.

7. Complete build pipelines

You want to ensure that your code is tested and analyzed before it is deployed. Build pipelines with testing, static analysis, database migrations will get you there.

8. Continuous Deployment

Continuous Deployment (CD) allows your team to deploy changes quickly and frequently, reducing the risk of introducing bugs and maintaining velocity.

9. Security best practices built-in

Your stack should include RBAC (Role-Based Access Control) and the "two-person rule" for infrastructure. Your tools should also eliminate the need for your team to manually manage credentials to access environments, reducing the risk of security breaches.

How does a startup get all that?

Unfortunately, most teams end up using a stack built on a PaaS (Platform as a Service) and kick the can on worrying about the development process. They build their own integrations based on copy/paste from Stack Overflow and manage the process in their heads. When an issue arises, they figure out how to solve the problem without building the fix into their process, causing future delays and downtime.

As startups scale, they outgrow their PaaS solution and their cobbled together stack falls apart. They end up having to rebuild and often need to hire experts to manage and maintain their new infrastructure. These experts need to migrate the existing systems to new platforms without causing downtime, which can be a time-consuming and expensive process.

They also need to keep legacy decisions, which can make things 10x slower and harder, as they may have to work around existing infrastructure and systems. An example might be that they chose to use a config file instead of an environment variable for some kind of information, and thus they cannot move to a system that only supports environment variables, even if it were better and simpler and cheaper in every other way. For some cases config files are the right choice - so this isn’t about that one particular item, but rather the kinds of snowballing decisions that cause long-term speed impacts on startup teams.

Furthermore, as the team grows, even a small percentage of the engineering salary budget can be a significant amount of money. It can take years to get ahead on the infra roadmap and implement the necessary changes to optimize costs and improve the platform's performance. This delay can be a real dollar drag on the startup's finances and may impede its ability to compete with other companies in the market. Therefore, startups need to consider the cost of engineering time when building and maintaining their platform to ensure that it is scalable and cost-effective in the long run.

Building the perfect developer platform for startups is critical to moving quickly and achieving success. It should be easy to set up, affordable, and support the tools that your team already knows and trusts. It should also provide a best-practice workflow that aligns with industry standards, complete build pipelines, and security best practices. That’s why we built Coherence. You should be able to get the ease of use and setup of a PaaS AND these nine principles (and more!) on day one, which will allow you to focus on building great products AND scale once you find product market fit. Be ready for the long term on day one with Coherence.

Common Serverless Runtime Limits

Zach Zaro — Wed, 31 Jan 2024 20:13:32 +0000

When does Coherence get useful instead of purely serverless functions, like those offered on Vercel/Netlify/CloudFlare? There are a few reasons why you might hit limits and need to go beyond their hosting, especially for backend APIs.

One set of limits is execution time limit on functions:

e.g 10 seconds on free on Vercel, 10 seconds on Netlify
You can pay to increase this on some platforms, for example the paid tier on Vercel is 60 seconds. This is still not enough for many real-world use cases, especially if you talk to 3rd party APIs from your code.

Another set of limits is the package, request size, or memory limit:

50 MB total deployment size limit on Vercel serverless
1024 MB memory limit on Vercel or Netlify serverless
4.5 MB max request body size

In addition to serverless functions, some platforms like Vercel also offer edge functions. These have their own limits.

Edge functions are even more limited, e.g. 30s max timeout for all plans or 4 MB size limit even for enterprise plans. There are some caveats here such as that streaming responses can consume more time as long as they are actively sending data back to the client.

There are additional limits that apply to some applications, for example:

No outbound TCP connections, e.g. to redis. There are workarounds here, such as using a HTTP wrapped API via a provider like Upstash, but these mean that you're locked into a vendor's ecosystem instead of the open standard software itself.
No ability to connect to database resources in a VPC, must use public IP and open your database up to the internet. In addition to security risks, this is a performance bottleneck and a cost drawback

Coherence has no limits aside from the underlying limits on Cloud Run (GCP) or Fargate (AWS). With Coherence, you can easily deploy an application to your own AWS or GCP account using a container and remove the limits on slug/package size, memory, and execution time right away. You’ll still get the developer experience of branch previews and an intuitive UI that Vercel provides. These features might be useful for your application, for example if you are doing long-running jobs that process data or talk to external APIs where you don’t control the response speed. You might also want to use:

A database instance or redis instance in your VPC on the cloud provider, for example postgres or mysql
Separate worker tiers running with different machine types, for example using libraries like celery, sidekiq, or bee-queue
CI/CD with database migrations or seeding built in, similar to what Jenkins, Drone, CircleCI, gitlab CI, or GitHub Actions might provide
A Cloud IDE based on VSCode to develop your project, similar to what GitPod or GitHub Codespaces might be used for

Feel free to reach out zach@withcoherence.com with any questions if we can help you get started!

Measuring Engineering Velocity on a Software Team in 2024

Zach Zaro — Wed, 31 Jan 2024 18:36:58 +0000

The concept of "velocity" has become a cornerstone metric for modern software development teams, aimed at quantifying the rate at which a team completes work. However, defining and accurately measuring velocity is not without its challenges. Below is a guide to understanding and effectively measuring engineering velocity on a software team.

1. What is Engineering Velocity?

Productivity management has been a part of “knowledge work” for a long time. The history of productivity management, particularly in the context of project management in non-manufacturing sectors, has evolved significantly over time, influenced by various theories, methodologies, and technologies. Here's a brief overview:

Early 20th Century - Scientific Management: The roots of productivity management can be traced back to the early 20th century with Frederick Taylor's Scientific Management theory. Taylor's approach focused on improving industrial efficiency through the analysis of workflow and labor processes. Although initially applied to manufacturing, these principles later influenced non-manufacturing sectors in terms of structuring work and managing projects.
Mid-20th Century - Project Management Emergence: The concept of modern project management began to take shape in the mid-20th century. Two notable methodologies, the Critical Path Method (CPM) and the Program Evaluation and Review Technique (PERT), were developed in the 1950s. These were initially used for large-scale government and defense projects, but they later found applications in various non-manufacturing sectors like construction, IT, and services.
1960s and 1970s - Management by Objectives (MBO): Developed by Peter Drucker in the 1950s, MBO became popular in the 1960s and 1970s. It focused on setting clear, measurable objectives and provided a framework for organizations to align their goals with the performance and development of their employees, including in non-manufacturing industries.
1980s and 1990s - Agile and Lean Methodologies: The emergence of Agile methodology in the late 20th century marked a significant shift in project management. Initially developed for software development, Agile's principles of flexibility, iterative progress, and cross-functional teams have been widely adopted in various non-manufacturing sectors. Similarly, Lean methodology, with its focus on reducing waste and adding value, also began to influence non-manufacturing sectors.
21st Century - Digital Transformation and AI: The advent of digital technologies has profoundly impacted productivity management. Project management software, collaboration tools, and AI-driven analytics have enabled more efficient workflow management, real-time communication, and data-driven decision-making in non-manufacturing sectors.
Recent Trends - Remote Work and Sustainability: The increase in remote work, accelerated by the COVID-19 pandemic, has led to new challenges and innovations in managing productivity and projects outside traditional office settings. Additionally, there's an increasing focus on sustainability and social responsibility in project management, reflecting broader societal and environmental concerns.

Throughout its history, productivity management in non-manufacturing sectors has continuously evolved, adapting to technological advancements, changing market demands, and shifting organizational priorities. The field remains dynamic, with ongoing innovations and methodologies emerging to address new challenges and opportunities.

Pre-Agile Era: The Waterfall Model

More specific to software development, the important history starts with waterfall. Before the rise of Agile methodologies, the predominant software development approach was the Waterfall model. This linear, phase-based model did not emphasize iterative development, and thus, the concept of velocity as we know it today wasn't relevant. Of course, deadlines and speed have always been part of any project - I’m sure they had some kind of scrum meetings when building pyramids, castles, or railroads - but the focus on how to measure and increase iteration speed for a project team was less focused on the metrics we will discuss below.

Birth of Agile

The seeds for the concept of velocity were sown with the rise of Agile methodologies in the late 1990s and early 2000s. Agile's emphasis on iterative and incremental development created a need for metrics that could gauge progress within short development cycles.

Scrum and the Introduction of Velocity

Scrum, one of the most popular Agile frameworks, introduced the term "velocity" to represent the amount of work a team could complete in a sprint (typically a 2-4 week period). This measure was used to help teams estimate how much work they could commit to in the next sprint. Over time, tracking velocity also helped teams predict their capacity for future work, allowing for better planning and stakeholder communication.

XP and the Emphasis on Estimation

Extreme Programming (XP), another Agile methodology, placed significant emphasis on continuous feedback and estimation. While XP didn't explicitly coin the term "velocity," its practices like "Planning Game" and "Iteration Planning" contributed to the broader Agile community's focus on adaptive planning and metrics, indirectly influencing the adoption and refinement of velocity as a concept.

Broader Adoption and Critiques

As Agile methodologies gained traction, so did the use of velocity as a measure. However, with broader adoption came critiques:

Misuse as a Performance Metric: While velocity was intended as a planning tool, some organizations began using it as a performance metric, putting undue pressure on teams to increase their velocity without regard for quality.
Variability Across Teams: The lack of standardization in story point estimation led to significant variability in velocity metrics across teams, making it hard to compare or aggregate velocities.

Modern Iterations and Extensions

As the software development community recognized the limitations and potential pitfalls of velocity, various adaptations and extensions emerged:

Throughput and Cycle Time: These are metrics focused on the actual output (e.g., features shipped or bugs fixed) and the time taken for a task to move from start to finish, respectively. They can serve as complementary or alternative metrics to velocity.
Commit-to-Complete Ratio: Measures how frequently teams meet their sprint commitments, providing an alternative view of consistency and reliability.

The history of measuring engineering velocity is intertwined with the evolution of Agile methodologies. As with many concepts in software development, velocity has seen its share of praise and critique. Its history reflects the software industry's ongoing quest for better ways to plan, measure, and improve the development process. The key lies in understanding its purpose, benefits, and limitations, ensuring it's applied thoughtfully and judiciously.

At its core, engineering velocity measures the amount of work a software team can complete within a given time frame, typically a sprint or iteration. It is expressed in terms of "story points" or other units that the team assigns to represent the complexity or effort required for a task.

DORA and SPACE are two frameworks that bring these concepts into the year 2023. Most of you probably have some exposure to teams that measure these stats - maybe alongside story points or maybe on their own. In our conversations at Coherence, we hear about these new frameworks a lot more than about agile terms like sprint velocity.

2. Why is it Important?

Velocity offers several benefits:

Predictability: It helps in forecasting how much work the team can tackle in future sprints.
Transparency: It provides a clear view of team progress to stakeholders.
Continuous Improvement: Tracking changes in velocity over time can highlight areas for optimization.

3. How to Measure Velocity

There are many different metrics monitored around velocity by different teams and the common theme is that there is no industry standard for measuring velocity. Below are some of the common metrics we see teams using. Some of these are related to planning and task management, and others are more output-focused and help to quantify a team’s performance and shipping speed.

Throughput

This metric counts the number of work items (user stories, tasks, features, bugs) completed in a specific time period, like a sprint or week. It offers a tangible count of output without resorting to the abstraction of story points.

PR Size

PR Size can be measured in lines of code per changeset. Smaller PRs generally move faster and have less defects.

Cycle Time

Cycle time measures the amount of time it takes for a work item to move from the beginning to the end of a process. For software teams, this could be the time taken for a user story to move from "Ready for Development" to "Done." Lower cycle times usually indicate higher velocity.

Lead Time

Lead time calculates the duration from when a new work item is created until it's completed. It encompasses the entire process, from the moment a new requirement is identified until it's fully implemented.

Commit-to-Complete Ratio

This ratio measures how often a team fulfills its commitments. For example, if a team commits to completing ten stories and finishes eight by the end of the sprint, their commit-to-complete ratio is 80%.

Work in Progress (WIP)

Though not a direct measure of velocity, WIP helps teams understand their capacity. By limiting WIP, teams can focus on completing tasks and potentially increase their velocity.

Burndown Chart

While this is a visual tool, the slope of a burndown chart (showing the amount of work remaining over time) can be indicative of a team's velocity. A steeper slope might indicate a faster pace of work.

Burnup Chart

This is the opposite of a burndown chart. It represents the cumulative work done over time. The slope of a burnup chart provides insights into the team's pace.

Escaped Defects

This metric counts the number of defects or bugs reported by customers after a release. Although not a direct measure of velocity, a high number of escaped defects might indicate that a team is prioritizing speed over quality.

Technical Debt

A measure of how much "quick and dirty" work is in the code, which will need to be paid back later for maintaining the software. Like escaped defects, rising technical debt might be a sign that velocity is coming at the expense of code quality.

Customer Satisfaction

While more qualitative, customer feedback can offer insights into whether the pace of delivering features and fixes aligns with user needs and expectations.

Story Points

During sprint planning, the team assigns a point value to each user story based on its estimated complexity.
At the end of the sprint, add up the points for all completed stories. This sum is the team's velocity for that sprint.
Velocity can be volatile. To get a more consistent measure, average the velocity over the last three to five sprints.

4. Common Mistakes

One of the most important parts of looking at velocity on your engineering team is understanding why you are tracking velocity and what you are hoping to optimize for. There is no “universal standard” of objective velocity across the industry to benchmark against, since codebases, tech stacks, and businesses are so different. Some things to keep in mind are:

Comparing Teams: Every team is unique. Avoid comparing velocities between teams as it can lead to incorrect assumptions and unhealthy competition.
Overemphasis on Velocity: Velocity is a diagnostic metric, not a performance metric. Pushing a team to increase velocity without addressing underlying issues can lead to burnout and decreased quality.
Inconsistent Point Estimation: From a planning POV, ensure that the team has a shared understanding of what each point value represents.

5. Best Practices

Measuring velocity can certainly help your team move faster, if you do it right. Keep these best practices in mind when you are looking at your team.

A. Regularly Refine Estimations and Metrics:

Hold estimation sessions to ensure alignment in understanding the size and complexity of tasks or of code shipped.

B. Factor in Non-Development Work:

Include activities like code reviews, bug fixes, and meetings. They consume time and should be accounted for.
Also account for infrastructure toil and coordination. Often this is some of the lowest-hanging fruit in increasing velocity. If you want to get some help, feel free to chat with us at Coherence!

C. Review and Adjust:

Continuously monitor velocity and discuss anomalies or changes during retrospectives.
Getting input from your team is the best way to improve your metrics and your understanding of them.

6. Remember the Bigger Picture

Velocity is just one metric among many. While it can provide insights into a team's capacity and performance, it shouldn't be used in isolation. Other qualitative measures, like code quality, customer satisfaction, and team morale, are also important.

Conclusion

Measuring engineering velocity is a valuable exercise for software teams aiming to improve their processes and deliver value consistently. By understanding its nuances and potential pitfalls, teams can use velocity as a tool for growth and continuous improvement rather than a blunt instrument of judgment.

We’ve been speaking with a ton of people as we look at our velocity metrics feature set at Coherence. The most important thing we’ve learned is that teams are hungry for more metrics around velocity, and they often still rely on manual scripting or cobbling together metrics from github and spreadsheets. If you're interested in measuring your team's efficiency check out our new PR Efficiency Scorecard which we just released.