DEV Community: Dan

Does a ‘perfect’ dev environment exist?

Dan — Tue, 13 Feb 2024 22:25:26 +0000

That’s likely a loaded question and it comes with a further set of qualifiers:

How resource intensive is the work? Cloud or local?
Are we talking about teams or individuals?
What types of machines are we working with?
What do you mean by environment? Do you mean your IDE? Internal dev stack?

To avoid the risk of sparking any holy wars, let’s lay some ground rules. Imagine a scenario where we have a team of 15 or so engineers and we’re working on an increasingly complex product where we want to iterate quickly. Our product has a lot of moving pieces and we’re starting to struggle with devs not having shared context for their dev environments. Different setups and inconsistent local environments are slowing us down. Sound familiar?

We’ve experienced it, we have friends who’ve run into it, some of this might even resonate with you.

Development is a fundamentally creative act and the canvas of development environments needs to enable engineers, not deter them.

This leads us to our quintessential question, what makes the ‘perfect dev environment’? For starters, we believe they should have all of the following characteristics:

Parametrizable: Save time by encoding common use cases in the dev environment.
Overridable: Retain flexibility by allowing for changes outside the predefined cases.
Approachable: Interfacing with your tooling should be intuitive.
Reliable: The environment should consistently spin up and work as expected.
Ephemeral: Easily spin up and clean up after yourself.
Distributable: Ensure everyone can easily access the dev environment.

Let's break these down.

Parametrizable: Save Time with Common Cases

Parameterization allows you to encode common use cases in your dev environment, letting you flex common interaction points while skipping the boilerplate repetition. Picture your dev environment with a Postgres database. You’ll need to input a username, generate test data, load that data, connect an API to your database, and more. These steps have different inputs however stay consistent.

In the near term, this saves you time in skipping tedium, but in the long term, especially as your setup grows in complexity, it saves mental bandwidth as every dev doesn’t need intimate knowledge of a system to interact with it. Allowing devs to simply configure the parts they need (username, API keys, etc.) while abstracting away the rest (loading data) ensures devs get the exact configuration they need without needing any knowledge of how the environment was set up.

Overridable: Stay Flexible for Uncommon Cases

So we can handle common cases, but what about the uncommon ones? Can we make changes outside the tyranny of what our environment author has set for us? This leads us to environments being overridable. Without the ability to override your dev environment, you’re stuck with the set of tools already written by someone else. If you aren’t already very familiar with those tools, you’d have to dive into that code to understand it, especially well enough not to break it.

Taking our Postgres example, imagine an internal tool that spins up a Postgres database, web services, and test data. If we wanted to change something that wasn’t yet parameterized — e.g. the flags Postgres starts with — we’d have to dive into Docker, k8s, or whatever else is used to start the dev stack. If you’re using bash scripts, you might break something in your local dev environment requiring more time for debugging. Best case scenario, nothing breaks and you’ve made the edits you expected but you’ve devoted quite a bit of time to achieve that. By allowing your environment to be overridable, you don’t need to deeply understand the system to make changes.

Approachable: How Intuitive is Your Setup?

Depending on the sophistication of your setup, it may be challenging to interact with your dev environment. Let's say your DevOps team has decided on a CLI and made this tool fully featured with a myriad of wonderful detailed options. As the tool becomes more powerful, this CLI becomes more complex and challenging to use.

When a dev is dumped into a tool that isn’t walk-up intuitive, they’re met with the choice of “Do I learn this tool or do I just hack something that’ll work right now?” Increasing approachability with an intuitive tool helps them pick the former. This is easier said than done as achieving this requires both a product mindset as well as a DevOps/platform mindset, or at the very least great docs which no engineer likes to write.

Reliable: Your Environment Should Just Work ™️

When spinning up the environments, it needs to just work. These environments need to spin up as expected when presented with different directories, different configurations of dev machines, and more. As many would guess, this is easier said than done. Flaky dev environments not only are frustrating but also result in costly delays.

Scripting languages are challenging for writing safe code. It’s challenging to catch configuration errors in advance, instead, they need to be caught at runtime. You need to implement waits and health checks correctly i.e. docker by default doesn’t wait for ports to become available and k8s has health checks but users need to configure them.

Ephemeral: Clean and Consistent

An ideal dev environment is also ephemeral. It’s important that you can spin these environments up and clean up after it trivially. This ensures consistency as team members work off a clean slate baseline environment. They’re also resource-efficient and scalable as you can clean up environments not in use and scale multiple instances as needed.

Ephemeral environments are useful for a variety of uses across quick dev iterations and end-to-end testing, they do require sophisticated infrastructure or tooling due to resource management, data persistence, and more, however, some upfront pain pays dividends.

Distributable: Get Your Tooling to Everyone

The last I’ll mention is the environment being distributable. So you have a highly flexible dev environment that is incredibly easy to spin up for common use cases, but easy to revise for uncommon ones. Sounds great but it’s only valuable if everyone can get their hands on it.

Smaller teams can likely get by as everyone can clone the repo and individually ensure they have the latest. However as the team grows and work becomes more complex, it becomes challenging to distribute the latest environment, ensuring everyone is working on the same thing and it's in sync with relevant tools. The best tooling is most valuable if it’s uniformly adopted by the people it’s meant for.

So where do we land?

Often, how easy achieving these depends on how good your platform engineer is, their familiarity with bash, compose, k8s, and their product mindset for aforementioned approachability. It’s a tall ask creating the ‘perfect dev environment’ given high demands for optimal developer experience, a fast-moving landscape, and technical barriers.

Luckily, we built Kurtosis so that you don’t have to make that choice. Development environments written into Kurtosis packages achieve all of the above and more. If you’d like to learn more, check out monorepo and consider dropping a star while you’re there.

Pkl and the Goldilocks problem of configuration languages

Dan — Wed, 07 Feb 2024 00:30:25 +0000

Apple’s newly open-sourced config language Pkl has been making waves on HackerNews, and the responses seem split between “This is a breath of fresh air” and “Why do we need another config language?”.

Kurtosis has forced me to think long and hard about configuration, so in this article I’ll provide a framework for evaluating configuration languages, explain why configuration languages face a difficult tradeoff, and make a prediction on Pkl’s trajectory.

What even is configuration?

This seems like a silly question, since we as an industry generally grok what config is: it’s the thing that controls how a piece of software behaves. It can be expressed in various ways — perhaps as an object in your favorite language, perhaps as flags or environment variables, perhaps as a YAML file — but in all cases config represents the top-level parameters of the piece of software.

However, config is interesting because it sits at the seam between the program being configured and a user (either a human, or another program). Configuration is an API.

The constraints on configuration

Because it sits at the seams between two systems, configuration is subject to the following pressures:

Approachability pressure: configuration must be walk-up approachable in a way that the program being configured need not be. It’s fine to write your entire program in Brainfuck if your configuration is understandable and well-documented YAML. Minimalism, readability, and documentation are important here.
Integration pressure: configuration lives at the seams between two systems, and it’s very common for engineers of the outer system to automate configuring the inner pieces. Configuration that can’t easily be automated is problematic.
Maintainability pressure: the users of the configuration need to evolve it over time, especially if they have an automation system built on top. Configuration reuse (DRY) is an important tactic for preventing bugs and increasing maintainability, so higher-order logic like conditions, loops, and packaging systems become necessary somewhere.

NOTE: configuration is also sometimes subjected to validity pressure, under the idea that config shouldn’t even be usable unless it’s valid. More on this later.

Configuration languages today

Let’s take a brief tour of some popular configuration languages and see how they deal with these pressures:

YAML — the industry standard — is quite approachable, with its comments and whitespace-delineated blocks and insistence on key name duplication. However, it has very limited configuration reuse capability and lacks conditionals or loops, so its maintainability is low. Despite this, its ability to be integrated into a broader system is very high: every general-purpose language has YAML parsers and generators, and templating is relatively easy.
TOML takes a similar “it’s just data” approach to configuration as YAML, and has similar high-approachability, easy-to-integrate, low-maintainability characteristics to YAML (though with the asterisk that it’s not as well-known and therefore has a larger learning curve).
Jsonnet is more maintainable than YAML because it provides code reuse features like variables, references, conditionals, and imports. However, its custom syntax on top of the verbose JSON syntax reduces its approachability, and the lack of Jsonnet generators reduce its ease-of-integration (which might explain its lack of industry dominance).
Dhall plays in the same “JSON with logic” space as Jsonnet, but adds a typing system as well. This means that the language itself enforces configuration validity, which can be helpful in shortening feedback loops. Like Jsonnet though, the extra logic with custom syntax comes at the cost of approachability and ease-of-integration.
Cue also focuses on code reuse features using a custom JSON-like syntax similar to Dhall and Jsonnet, but goes even further by baking data validation into the value system. This leads to very strong validation, but with similar approachability and integration problems as Dhall and Jsonnet. In particular, Cue’s matrix model — while powerful — is hard to grok quickly.
Starlark is Bazel’s configuration language, and has high approachability as a minimal subset of Python. However, users are free to write quite complex logic so its maintainability suffers, and Starlark generation is difficult so its ease-of-integration is low.

Analysis

The projects above are sorted by descending star count on the project repo. Assuming star count is a rough proxy for popularity, this suggests some interesting conclusions:

Approachability and ease-of-integration tend to move in tandem
Approachability and maintainability tend to move opposite each other
The two most-approachable, least-maintainable languages are by far the most popular

In other words, people prefer to have configuration that’s difficult to maintain so long as they can easily understand what it’s doing, and integrate it with a higher-level system to flex the configuration as needed.

This makes sense to me, as configuration’s primary purpose is to hold parameters for the lower-level system. Secondary features like code reuse aren’t always worth their complexity weight, because code reuse can be done in a higher-level configuration automation system.

This is especially true for validation. In theory, it sounds nice to disallow invalid configurations from being instantiated, but in practice doing validation in the configuration language often means the author of the program-to-configure needs to duplicate validation: once in the program to validate the flags and environment variables their program inevitably accepts, and once more in the configuration file schema.

There is, however, an environment where the calculus changes: a large company with a powerful internal platform and a standardized configuration language. In such an environment, a more powerful top-level configuration language is worth the approachability hit to empower engineers to reuse configuration from a large galaxy of existing configuration.

It makes sense to me that Jsonnet, Cue, and Starlark were all born of Google’s configuration needs, and that Pkl might be born of Apple’s.

The Goldilocks problem, and Pkl

We’ve seen that configuration languages face a Goldilocks problem — too little complexity and they struggle to keep code DRY and maintainable, too much complexity and they’re difficult to learn and integrate outside of corporate environments. The balance has to be just right.

So where does Pkl fit in?

Approachability: Pkl uses a completely custom syntax, which is a big barrier to adoption, and the system itself seems decently complex. I’d rate the approachability as low.
Integration: This is a wildcard for me, as it’s not clear to me if I can import third-party schemas, generate bindings in my favorite general-purpose language, and use that to build a configuration automation system on top.
Maintainability: Pkl’s embrace of schemas reflects configuration’s purpose as an API, and much of Pkl’s feature set seems geared towards code reuse and typing. I imagine that Pkl code will be quite maintainable.
Validation: Pkl seems to prioritize validation highly; I’d anticipate Pkl config to be very safe.

This analysis suggests to me that Pkl is trying to fill a spot somewhat similar to Cue: a very safe, highly-reusable language that’s intended to be your top-level configuration language at large scale, but whose approachability and ease-of-integration can be constraining for smaller projects and companies.

I therefore hypothesize that, on the current trajectory, Pkl will probably see heavy adoption at Apple, but will remain niche outside. If the Pkl team is aiming for broader open-source adoption, I’d recommend smoothing the approachability and integration paths with the following tactical suggestions:

The ability for YAML and TOML to be parsed as basic Pkl files (thereby allowing me to write my config in a language I’m familiar with, but get Pkl schema validation)
Smaller example repos with clearer identification of the Pkl-specific parts so I can see the full cost of buying in
The ability to generate Pkl from my favorite general-purpose programming language, so I can build a configuration automation platform on top
Perhaps, features in the toolchain to put backpressure on writing less complex (and therefore more approachable) Pkl files

Afterword

Configuration languages fill an important role but face a difficult game of tradeoffs, and Pkl is no exception. The proliferation of tools attempting to be the top-of-stack configuration language reminds me of the famous XKCD comic about standards, and I’m becoming bearish on any non-YAML language as the open-source standard.

In the next article I’ll detail how we wrestled with these same dilemmas at Kurtosis, and the conclusion we came to. Until then, if you’re a microservice developer who doesn’t want to deal with infra, I encourage you to check us out on Github; you might find Kurtosis useful.

Cool Devtool Websites?

Dan — Mon, 05 Feb 2024 23:46:55 +0000

We just redid our website for Kurtosis.com!

We're underway with a lot of user discovery so still experimenting with what words makes sense to users. Would love thoughts on how we did:

Does the messaging make sense?
Do you have an idea of what we might be useful for?
Is it kinda cool?

Would love any and all feedback!

If you're interested in chatting on simplifying building your back end for dev and test, come chat with us and/or consider dropping a star on our monorepo <3

https://github.com/kurtosis-tech/kurtosis