So You Want to build a language?

DISCLAIMER: I have never designed a programming language. Neither have most coders. While I do not know the monumental effort that goes into creating a programming language, many of us are familiar with their associated traits, discussed below. This is my first post, and i'm not much of a writer. Forgive my mincing of words.

Creating programming languages seems fun (or so I am told). You get to implement a wide variety of capabilities that are fundamental to computer science and understanding the underlying hardware. Unfortunately, simply creating a compiler/interpreter is far from what it takes a make a language user friendly. I made this post to enumerate these "extra" things, because lets be real, every language should have them. These extra capabilities are not simply "extraneous," they're what makes a language usable on a daily basis (or you know, at all).

I understand the following goals take incredible time to achieve. I don't mean to diminish the work that goes into writing blazing fast runtimes and compilers that output performant output. I just want to highlight that it's not even half the battle. The problem is when the following is provided by third parties, it encourages fragmentation of your ecosystem and bloated projects that require experts just to configure and run (lookin' at you javascript).

IDE Integration

This is one of the biggest points. Imagine the last time you opened a file, and there wasn't syntax highlighting, or any of the following. Certainly one of the biggest mood killers in our field.

• Syntax highlighting
• Code snippets
• Context Aware suggestions & autocomplete
• Refactoring
• Jumping to and/or viewing docs

It used to be that creating the above required an essentially bespoke implementation for each IDE, and sometimes even tooling. Thanks to the Language Server Protocol (from Microsoft) however, that's a thing of the past. Language developers need only write a single server providing all of these features. And best of all, you can write it in your budding language!

Dependency Management

Ah, where to begin. Java began with a system of zip files containing code. This led to namespace collisions and missing depencies galore. In fact, it became good practice to place seperate dependencies in the same package/folder, ACROSS DIFFERENT ZIP FILES. However, for 2020 and beyond please have...

• A User-Facing package registry
  • Links to docs and repository
  • Displays description and README
  • A feature to deprecate libraries
  • Maven repositories have none of the above bells and whistles, and it means projects often seem like abandonware.
• An easy way to publish that doesn't dump the burden of secret keeping on the user
• A way to resolve and declare said dependencies (like cargo.toml rust, build.gradle for the jvm, or package.json for Node)
• Warnings against outdated/insecure deps
• This isn't so much a point as warning: javascript allows one to import arbitrary source files, by url. You can see how well that's worked.
• And last but not least, don't invent some new declarative syntax for your config files, or allow them to be written in a full fledged language (even YOURS). We have toml, yaml, json, etc.
  • Gradle config files are written in groovy/kotlin. I've seen entire build processes defined in a single file. 😰
  • Practices like this make the build process opaque and prevent your project from being ported or even upgraded.
  • Instead, consider a plugin system or addons for your build tool.

Documentation System

Adopt some way of writing in not-quite plain text for your documentation (like markdown or ReDoc), and provide a way to publish these as cool webpages. Javadoc tries to deploys its docs as self-contained websites, to be published by the mainteners of a project. This results in a lot of deployed bloat. Instead, leave the content to the devs, and everything else (navigation, sorting, searching, styling) to your package registry. Example: crates.io.

Debugger

This point ties heavily into IDE Integration, and of course the title says it all. We all know the joys of using logging and print statements.

Linter

Code smells happen all the time. Nobody's perfect, that's what computers are for. I won't discuss the impact of linters in general. When one is provided by the designers of a language however, it usually results in a noticeable increase in code quality. Take for example, clippy, THE rust linter. Beautiful is the only word to describe it. Its saved my a** on several occasions, is fast, and has beautiful output. When a linter is 1st party software, it enables the entire community to contribute even the most obscure lints and save everyone some hassle. And of course, it becomes second nature to hook up to your CI/CD.

Formatter

Admittedly, this is certainly the least pressing of these points. While adopting a certain style can be controversial, thankfully enforcing one is not. If it really is hard to settle on a language-wide style, your formatter should accept the most common sources of ire as configuration options (usually indentation and imports order). A formatter is best when its set and forget as a pre-commit hook. And please, for everyone's sake, please make sure your formatter actually formats, rather than merely pointing out style guide violations. Yes, apparently this is something I must mention.