DEV Community: Manuel Odendahl

Build your own custom AI CLI tools

Manuel Odendahl — Mon, 14 Aug 2023 17:31:11 +0000

Originally posted on: scapegoat.dev

I am notoriously distracted and both my long and short term memories are less reliable than a hobbyist's kubernetes cluster. After 20 years of python, I still can't remember what the arguments to split are.

Thankfully, these days are now over, as I can ask a just as clueless LLM to answer the myriad of questions I would otherwise have to google everyday (please don't use google, use kagi, you'll thank me later). Being part of a secret society of robo-scientists, I have been building some visionary software over the last couple of months, drawing inspiration from many sources, because writing software is about being in touch with the universe.

More prosaically, I have been writing some LLM-powered command line tools, as many people have before me and many people will after my bones are dust and my ideas forgotten. All the tools I present in this blog post have been written in a single day, because that's the world we now live in.

Some preliminary examples

Here is a little taste of the things that are possible with these tools.

All of these examples were built in a couple of hours altogether. By the end of the article, you will be able to build them too, with no code involved.

You might see a a mix of pinocchio commands and llmX commands. The llmX commands are just aliases to pinocchio prompts code X, the llmX-3.5 are aliases to use the gpt-3.5-turbo model (pinocchio prompts code X --openai-engine gpt-3.5-turbo).

Reversing a string in go

Let's start with a simple interview question: reverse a string in go.

Getting some ffmpeg help

Too lazy for manpages? Extremely reliable LLM to the rescue!

Getting some vim help

Some more reliable, trusted help.

Creating a script to help us convert these goddamn videos

Recording these little screencast videos is tedious, let's get some help.

Explaining a terraform file

What the heck did I do 2 months ago?

Writing a bandcamp search client

Too lazy to read API docs? Just paste a curl request in there! Documentation is for chumps!

Code review that genius code

I'm sure this won't miss a single thing. It sure is more annoying than my colleague Janine.

Give the intern some advice

But don't be so blunt.

Create some slides for the weekly review

Tufte said slides shouldn't have bullet points. We disagree.

Installing pinocchio

In order for you, dear reader, to follow along on this adventure, you first need to install pinocchio. Nothing could be easier as we provide binaries for linux and mac (x86 and arm), as well as RPM and deb packages. All the GO GO GADGETS are also published on homebrew and as docker images, if that's your jam.

After configuring your OpenAI API key, you should be all set to go. For more precise instructions, check out the installation guide.

Geppetto - LLM framework

Over the last couple of months, like every other developer out there, I have been working on a LLM framework that will make it trivial to build AMAZING CRAZY STUFF.

The core architecture of this very advanced framework boils down to:

take input values
interpolate them into a big string using the go templating engine
sending the string over HTTP
print out the response

(This is why they call me senior engineer).

The framework's claim to fame, besides having a cool name (geppetto is a client for GPT and pinocchio, the puppet borne of geppetto's woodworking, is the CLI tool used to run applications built with the framework. Of course, LLMs being what they are, pinocchio often lies).

Declarative commands

Jokes aside, general consensus amongst the GO GO GOLEMS has it that geppetto and pinocchio are pretty cool. They are both based on the glazed framework, more precisely based on the Command abstraction.

This groundbreaking abstraction of a "general purpose command" consists of:

a set of input flags and parameters (which can be grouped into layers because having 800 flags tends to get overwhelming), each having a type, a default value and a description
a Run() function that executes the command
some kind of output (either structured tabular data, which is what glazed is built for, or just a sequence of bytes, in our case)

Most of the actual engineering in glazed focuses on:

making it easy to define flags declaratively (usually as YAML)
parsing flags at runtime (usually using the cobra framework)
running the actual Command by calling Run
processing the tabular output in a myriad of ways, none of which matter today

After handling \r\n and \n, converting between strings and integers is the second hardest engineering problem known to man. This is why this framework is senior principal staff engineer grade.

For example, the flags for the go command shown in the first video look like this:



flags:
  - name: additional_system
    type: string
    help: Additional system prompt
    default: ""
  - name: additional
    type: string
    help: Additional prompt
    default: ""
  - name: write_code
    type: bool
    help: Write code instead of answering questions
    default: false
  - name: context
    type: stringFromFiles
    help: Additional context from files
  - name: concise
    type: bool
    help: Give concise answers
    default: false
  - name: use_bullets
    type: bool
    help: Use bullet points in the answer
    default: false
  - name: use_keywords
    type: bool
    help: Use keywords in the answer
    default: false
arguments:
  - name: query
    type: stringList
    help: Question to answer
    required: true

glazed provides a variety of standard types: strings, list of strings, objects, strings from files, booleans, integers, you name it.

These declarative flags as well as layers added by the application all get wired up as CLI flags. Because this is user-first development, the output is colored.

If we were to use something like parka, these flags would be exposed as REST APIs, HTML forms and file download URLs. For example, sqleton can be run as a webserver for easy querying of a database.



flags:
  - name: today
    type: bool
    help: Select only today's orders
    default: true
  - name: from
    type: date
    help: From date
  - name: to
    type: date
    help: To date
  - name: status
    type: stringList
    help: Batch status
  - name: limit
    help: Limit the number of results
    type: int
    default: 0
  - name: offset
    type: int
    help: Offset
    default: 0
  - name: order_by
    type: string
    default: id DESC
    help: Order by

But that is a story for another time.

GeppettoCommand - look ma, no code

Now that we know how to define a command's input parameters, let's get to what a geppetto command is actually made of.

A geppetto command:

uses its input parameters to interpolate a set of messages (system prompt, user messages, assistant messages)
fills out the parameters for a request to OpenAI's completion API
fires off a HTTP request
streams the output back to the standard output.

Geppetto supports specifying functions too, but I've never used them because I try to use boring technology, not the latest fancy shiny new thing.

None of this requires massive amounts of clever computation, which is why we can also fit all of this into the same YAML file. For example, the go command from before looks like this:



name: go
short: Answer questions about the go programming language
factories:
  openai:
    client:
      timeout: 120
    completion:
      engine: gpt-4
      temperature: 0.2
      max_response_tokens: 1024
      stream: true
flags:
# snip snip ...
system-prompt: |
  You are an expert golang programmer. You give concise answers for expert users.
  You give concise answers for expert users.
  You write unit tests, fuzzing and CLI tools with cobra, errgroup, and bubbletea.
  You pass context.Context around for cancellation.
  You use modern golang idioms.
prompt: |
  {{ if .write_code }}Write go code the following task.
  Output only the code, but write a lot of comments as part of the script.{{ else }}
  Answer the following question about go. Use code examples. Give a concise answer.
  {{ end }}
  {{ .query | join " " }}
  {{ .additional }}
  {{ if .context }}
    {{ .context }}
  {{ end }}
  {{ if .concise }}
    Give a concise answer, answer in a single sentence if possible, skip unnecessary explanations.
  {{ end }}
  {{ if .use_bullets }}
    Use bullet points in the answer.
  {{ end }}
  {{ if .use_keywords }}
      Use keywords in the answer.
  {{ end }}

You can see here how we we first specify the OpenAI API parameters:

we are using the gpt-4 model per default (try using gpt-3.5-turbo for programming, I dare you)
with a reasonably low temperature.

Of course, all these settings can be overridden from the command line.

After the (omitted) flags definition from before, we get to the special sauce:

a system prompt template telling the LLM exactly who it is (or, giving it a special SYS token followed by some offputtingly prescriptive tokens with the hope that all this conjuring will cause future tokens to be tokens that are USEFUL TO ME)
a message prompt template that will be passed as a user message to the LLM, again with the hope that this little preamble will cause the model to generate something at least remotely useful. I think of it as having a fortune reader draw some language-shaped cards that tell me what code to write next

While what pinocchio does borders on magic (as all things having to do with LLMs), it is not too complicated to follow:

it interpolates the templates
it collects the system prompt and the prompt as messages
it sends off those messages to the openAI API
it outputs the streaming responses (if streaming is enabled) ### YAML is nice

The nice part (the really nice part) about being able to create "rich" command line applications using glazed and geppetto is that you can now experiment with prompt engineering by using command line flags instead of having to write custom test runners and data tables. This simple example-driven template is usually enough to reproduce most papers from 2021 and 2022, as shown by various datasets used in the Chain Of Thought paper.



name: example-driven
short: Show a chain of thought example
flags:
  - name: question
    type: string
    help: The question to ask
    required: true
  - name: problem
    type: string
    help: The problem to solve
    required: false
  - name: instructions
    type: string
    help: Additional instructions to follow
    required: false
  - name: examples
    type: objectListFromFile
    required: true
prompt: |
  {{ if .problem }}
  Problem: {{ .problem }}
  {{ end }}
  {{ range $i, $example := .examples }}
  Q: {{ $example.question }}
  A: {{ $example.answer }}
  {{ end }}
  {{ if .instructions }}
  Instructions: {{ .instructions }}
  {{ end }}
  Q: {{ .question }}
  A:

And of course, it is entirely possible to generate GeppettoCommands using GeppettoCommands, as shown by this example which can be used to generate SqletonCommands (the same concept, but for SQL).

Using a GeppettoCommand that generates GeppettoCommands to generate itself risks immanentizing the eschaton and bringing singularity into being, so use these new-found powers with caution.



name: quine
short: Generate yourself!
flags:
  - name: example_goal
    short: Example goal
    type: string
    default:  Generate a program to generate itself.
  - name: instructions
    type: string
    help: Additional language specific instructions
    required: false
  - name: example
    type: stringFromFile
    help: Example program
    required: true
  - name: goal
    type: string
    help: The goal to be generated
    default: Generate a program to generate itself.
prompt: |
  Write a program by generating  a YAML describing a command line application with flags and a prompt template using
  go template expansion.
  The flags are used to interpolate the prompt template.

  Here is an example.

  --- GOAL: {{ .example_goal }}
  --- START PROGRAM
  {{ .example | indent 4 }}
  --- END 

  Generate a program to {{ .goal }}.

  {{ if .instructions }}{{ .instructions }}{{ end }}

  --- GOAL: {{ .goal }}
  --- START PROGRAM

SQLETON excursion - generating SQL queries using generated SQL queries

Let's imagine that we want to generate SqletonCommands using the schema from a table in the database, but we are so lazy that we don't want to copy that schema. Wouldn't it be nice if we had a SqletonCommand to introspect a schema?



pinocchio examples code sqleton --topic "Create a command to output the schema of a table in mysql. It should take flags: tables, tables_like, columns, columns_like, type. Output the table name as column as well."

To which the LLM responds:

Here is an example of a command that outputs the schema of a table in MySQL using the provided YAML template:



name: schema
short: Output the schema of a table in MySQL.
flags:
  - name: tables
    type: stringList
    help: List of table names
  - name: tables_like
    type: stringList
    help: List of table name patterns to match
  - name: columns
    type: stringList
    help: List of column names
  - name: columns_like
    type: stringList
    help: List of column name patterns to match
  - name: type
    type: string
    help: Column data type
query: |
  SELECT
    TABLE_NAME AS table_name,
    COLUMN_NAME AS column_name,
    COLUMN_TYPE AS column_type,
    IS_NULLABLE AS is_nullable,
    COLUMN_KEY AS column_key,
    COLUMN_DEFAULT AS column_default,
    EXTRA AS extra
  FROM
    INFORMATION_SCHEMA.COLUMNS
  WHERE 1=1
  {{ if .tables }}
    AND TABLE_NAME IN ({{ .tables | sqlStringIn }})
  {{ end }}
  {{ if .tables_like }}
    AND (
      {{ range $index, $table := .tables_like }}
        TABLE_NAME LIKE '{{ $table }}'
        {{ if not (last $index .tables_like) }}
          OR
        {{ end }}
      {{ end }}
    )
  {{ end }}
  {{ if .columns }}
    AND COLUMN_NAME IN ({{ .columns | sqlStringIn }})
  {{ end }}
  {{ if .columns_like }}
    AND (
      {{ range $index, $column := .columns_like }}
        COLUMN_NAME LIKE '{{ $column }}'
        {{ if not (last $index .columns_like) }}
          OR
        {{ end }}
      {{ end }}
    )
  {{ end }}
  {{ if .type }}
    AND COLUMN_TYPE = '{{ .type }}'
  {{ end }}
  ORDER BY table_name

In this example, the tables flag accepts a list of table names, the tables_like flag accepts a list of table name patterns to match using the LIKE operator, the columns flag accepts a list of column names, the columns_like flag accepts a list of column name patterns to match using the LIKE operator, and the type flag accepts a column data type.

The query uses the INFORMATION_SCHEMA.COLUMNS table to retrieve the schema information. The WHERE clause is dynamically generated based on the provided flags. The ORDER BY clause orders the results by the table name.

Note that the sqlStringIn function is used to generate the IN clause for string values, and the last function is used to determine if it is the last element in a list during iteration.

Storing that command as mysql schema, we can now easily introspect various tables, leveraging all the affordances provided by glazed.



❯ sqleton mysql schema --tables giftcards 
+------------+---------------+-----------------+-------------+------------+----------------+-------+
| table_name | column_name   | column_type     | is_nullable | column_key | column_default | extra |
+------------+---------------+-----------------+-------------+------------+----------------+-------+
| giftcards  | active        | tinyint         | NO          | MUL        | 0              |       |
| giftcards  | amount        | decimal(10,2)   | NO          |            | 0.00           |       |
| giftcards  | balance       | decimal(10,2)   | NO          |            | 0.00           |       |
| giftcards  | code          | varchar(200)    | NO          |            | <nil>          |       |
| giftcards  | created_at    | datetime        | YES         | MUL        | <nil>          |       |
| giftcards  | expired       | int             | YES         | MUL        | <nil>          |       |
| giftcards  | expires_at    | datetime        | YES         | MUL        | <nil>          |       |
| giftcards  | free_shipping | tinyint         | NO          | MUL        | 0              |       |
| giftcards  | id            | int unsigned    | NO          | PRI        | 0              |       |
| giftcards  | note          | mediumtext      | YES         |            | <nil>          |       |
| giftcards  | open_amount   | decimal(10,2)   | NO          |            | 0.00           |       |
| giftcards  | order_date    | date            | YES         | MUL        | <nil>          |       |
| giftcards  | order_id      | bigint unsigned | YES         | MUL        | 0              |       |
| giftcards  | order_number  | longtext        | YES         | MUL        | <nil>          |       |
| giftcards  | redeemed      | decimal(11,2)   | NO          |            | 0.00           |       |
| giftcards  | TYPE          | varchar(9)      | NO          | MUL        |                |       |
+------------+---------------+-----------------+-------------+------------+----------------+-------+

Or slightly smaller:



❯ sqleton mysql schema --tables giftcards --columns-like '%order%' --output yaml --filter extra,column_default
- table_name: giftcards
  column_name: order_date
  column_type: date
  is_nullable: "YES"
  column_key: MUL
- table_name: giftcards
  column_name: order_id
  column_type: bigint unsigned
  is_nullable: "YES"
  column_key: MUL
- table_name: giftcards
  column_name: order_number
  column_type: longtext
  is_nullable: "YES"
  column_key: MUL

We can easily use our new mysql schema command to create new SqletonCommands, say one to query our giftcards table.



❯ sqleton mysql schema \
       --tables giftcards \
       --fields column_name,column_type,is_nullable \
       --output yaml | 
  pinocchio examples code sqleton --ddl - \
     --topic "Giftcards from an ecommerce database, used as giftcards but also warranty and courtesy refunds." \ 
    --instructions "Don't create flags for individual amounts. Create a codes_like flag."

To which the LLM responds:

Here is the updated YAML template for querying the giftcards table:



name: giftcards
short: Get giftcards from the database.
flags:
  - name: id
    type: intList
    help: List of giftcard ids
  - name: code
    type: stringList
    help: List of giftcard codes
  - name: active
    type: bool
    help: Giftcard active status
  - name: expired
    type: bool
    help: Giftcard expired status
  - name: expires_at
    type: date
    help: Giftcard expiration date
  - name: order_id
    type: int
    help: Order ID associated with the giftcard
  - name: order_number
    type: string
    help: Order number associated with the giftcard
  - name: redeemed
    type: float
    help: Giftcard redeemed amount
  - name: type
    type: string
    help: Giftcard type
  - name: codes_like
    type: stringList
    help: List of giftcard codes to filter by using LIKE queries
  - name: limit
    type: int
    help: Limit the number of results
    default: 0
  - name: offset
    type: int
    help: Offset 
    default: 0
  - name: order_by
    type: string
    default: created_at DESC
    help: Order by
query: |
  {{ if .explain }}
    EXPLAIN
  {{ end }}
  SELECT
    id,
    code,
    active,
    expired,
    expires_at,
    order_id,
    order_number,
    redeemed,
    type
  FROM giftcards
  WHERE 1=1
  {{ if .id }}
    AND id IN ({{ .id | sqlIntIn }})
  {{ end }}
  {{ if .code }}
    AND code IN ({{ .code | sqlStringIn }})
  {{ end }}
  {{ if .active }}
    AND active = {{ .active }}
  {{ end }}
  {{ if .expired }}
    AND expired = {{ .expired }}
  {{ end }}
  {{ if .expires_at }}
    AND expires_at = {{ .expires_at | sqlDate }}
  {{ end }}
  {{ if .order_id }}
    AND order_id = {{ .order_id }}
  {{ end }}
  {{ if .order_number }}
    AND order_number = '{{ .order_number }}'
  {{ end }}
  {{ if .redeemed }}
    AND redeemed = {{ .redeemed }}
  {{ end }}
  {{ if .type }}
    AND type = '{{ .type }}'
  {{ end }}
  {{ if .codes_like }}
    AND (
      {{ range $index, $code := .codes_like }}
        code LIKE '%{{ $code }}%'
        {{ if not (last $index .codes_like) }}
          OR
        {{ end }}
      {{ end }}
    )
  {{ end }}
  ORDER BY {{ .order_by }}
  {{ if .limit }}
    LIMIT {{ .limit }}
  {{ if .offset }}
    OFFSET {{ .offset }}
  {{ end }}
  {{ end }}

In this template, I have added flags for the different columns in the giftcards table and modified the WHERE clause to include the new flags. I have also added support for the codes_like flag, which takes a list of giftcard codes and generates a filter statement with multiple LIKE queries joined by OR. The order_by flag is used instead of sort_by to specify the order of the results.

We can now of course run this command and benefit from all the cool glazed bells and whistles.



❯ sqleton run-command /tmp/giftcards.yaml --limit 5
+-------+---------------------+--------+---------+---------------------+----------+--------------+----------+----------+
| id    | code                | active | expired | expires_at          | order_id | order_number | redeemed | type     |
+-------+---------------------+--------+---------+---------------------+----------+--------------+----------+----------+
| 22148 | WRHL-3242-KSKD-SZ4Z | 1      | 0       | 2024-04-29 00:00:00 | <nil>    | <nil>        | 0.00     | coupon   |
| 22147 | NHDC-3YF5-2421-VP25 | 1      | 0       | 2024-04-29 00:00:00 | <nil>    | <nil>        | 259.00   | coupon   |
| 22146 | DAB1-1X6Z-K9AV-XXWF | 1      | 0       | 2024-04-29 00:00:00 | <nil>    | <nil>        | 0.00     | warranty |
| 22145 | WFFT-BXEU-6WGT-E559 | 1      | 0       | 2024-04-29 00:00:00 | <nil>    | <nil>        | 99.50    | warranty |
| 22144 | VCCC-FEBP-KP12-U9P3 | 0      | 0       | 2024-04-29 00:00:00 | <nil>    | <nil>        | 0.00     | coupon   |
+-------+---------------------+--------+---------+---------------------+----------+--------------+----------+----------+

Storing and sharing commands

Once you get ahold of such powerful AI programs^W^W^Wcrude word templates, you can stash them in what is called a "repository". A repository is a directory containing YAML files (plans exist to have repositories backed by a database, but that technology is not within our reach yet). The directory structure is mirrored as verb structure in the CLI app (or URL path when deploying as an API), and the individual YAML represent actual commands. These repositories can be configured in the config file as well, as described in the README.

You can also create aliases using the --create-alias NAME flag. The resulting YAML has to be stored under the same verb path as the original command. So, an alias for prompts code php will have to be stored under prompts/code/php/ in one of your repositories.

Let's say that we want to get a rundown of possible unit tests:



❯ pinocchio prompts code professional --use-bullets "Suggest unit tests for the following code. Don't write any test code, but be exhaustive and consider all possible edge cases." --create-alias suggest-unit-tests
name: suggest-unit-tests
aliasFor: professional
flags:
    use-bullets: "true"
arguments:
    - Suggest unit tests for the following code. Don't write any test code, but be exhaustive and consider all possible edge cases.

By storing the result as suggest-unit-test.yaml in prompts/code/professional, we can now easily suggest unit tests:



❯ pinocchio prompts code professional suggest-unit-tests --context /tmp/reverse.go  --concise
- Test with an empty string to ensure the function handles it correctly.
- Test with a single character string to check if the function returns the same character.
- Test with a two-character string to verify the characters are swapped correctly.
- Test with a multi-character string to ensure the string is reversed correctly.
- Test with a string that contains special characters and numbers to ensure they are reversed correctly.
- Test with a string that contains Unicode characters to verify the function handles them correctly.
- Test with a string that contains spaces to ensure they are preserved in the reversed string.
- Test with a string that contains repeated characters to ensure the order is reversed correctly.
- Test with a very long string to check the function's performance and correctness.

Of course, all these commands can themselves be aliased to shorter commands using standard shell aliases.



for i in aws bash emacs go php python rust typescript sql unix professional; do
        alias llm$i="pinocchio prompts code $i"
        alias llm${i}-3.5="pinocchio --openai-engine gpt-3.5-turbo prompts code $i"
done

What does this all mean?

An earth-shattering consequence of this heavenly design is that you can add repositories for various GO GO GADGETS such as sqleton, escuse-me, geppetto or oak to your projects' codebases, point to them in your config file, and BAM, you now have a rich set of CLI tools that is automatically shared across your team and kept in source control right along the rest of your code.

This is especially useful in order to encode custom refactoring or other rote operations (say, scaffolding the nth API glue to import data into your system). Whereas you would have to spend the effort to build a proper AST, AST transformer, output template, you can now write refactoring tools with basically the same effort as writing "well, actually, wouldn't it be nice if…" on slack.

Remember the words that once echoed through these desolate halls:

WHEN LIFE GIVES YOU STYLE GUIDES, DON'T NITPICK IN CODE REVIEWS. MAKE EXECUTABLE STYLE GUIDES! DEMAND TO SEE TECHNOLOGY'S MANAGER! MAKE IT RUE THE DAY WHERE IT GAVE US EXECUTABLE SPEECH! DO YOU KNOW WHO WE ARE? WE ARE THE GO GO GOLEMS THAT ARE GOING TO BURN YOUR CODEBASE DOWN.

This can look as follows, to convert existing classes to property promotion constructors in PHP8.



name: php-property-promotion
short: Generate a class with constructor property promotion
factories:
  openai:
    client:
      timeout: 120
    completion:
      engine: gpt-4
      temperature: 0.2
      max_response_tokens: 1024
      stop:
        - // End Output
      stream: true
flags:
  - name: instructions
    type: string
    help: Additional language specific instructions
  - name: readonly
    type: bool
    default: true
    help: Make the class readonly
arguments:
  - name: input_file
    type: stringFromFile
    help: Input file containing the attribute definitions
    required: true
prompt: |
  Write a {{ if .readonly }}readonly{{end}} PHP class with constructor property promotion for the following fields.

  {{ if .instructions }}{{ .instructions }}{{ end }}

  For example: 

  // Input
  public ?int $productId; // Internal product ID for reference
  public ?string $itemId; // The ID of the item 
  // End Input

  // Output
  public function __construct(
     /** Internal product ID for reference */
     public ?int $productId = null,
     /** The ID of the item */
     public ?string $itemId = null,
  ) {}
  // End Output

  Now create a constructor for the following fields.

  // Input
  {{ .input_file }}
  // End Input

  // Output

This I think is the most exciting aspect of LLMs. They make it possible to build ad-hoc tooling that is able (even if stochastically hitting or missing) to do rote but complex, ill-defined, time-consuming work that is very specific to the problem and situation at hand. I have been using tools like the above to cut through unknown codebases, add unit tests, clean up READMEs, write CLI tools, generate reports, and much more.

The ones I built today are quite useful and make for cool demos, but the real value comes lies in being able to do very fuzzy, ad-hoc work that needs to be repeated at scale.

Where do we go from here?

I hope to make it easier to share and package these custom prompts and slowly start advertising GO GO GOLEMS and its ecosystem more, in order to get community contributions.

A second project that is underway is building agent tooling. Single prompt applications are useful but often suffer from limited context sizes and lack of iteration. I have been working on a "step" abstraction that should make it possible to run complex agent workflows supporting iteration, error handling, user interaction, caching.

A third project underway is a "prompt context manager" to make it easy to augment the LLM applications with additional information, coming either from external documents, live queries against a codebase, external APIs, saved snippets and summaries from past conversations, etc…

Finally, I have a currently closed-source framework that makes it possible to deploy any glazed command (i.e., a single YAML file) as a web API, a lambda function or a WASM plugin. I hope to be able to port this part to opensource, as it makes the tools exponentially more useful.

LLMs will fundamentally change software engineering

Manuel Odendahl — Thu, 23 Mar 2023 10:30:47 +0000

(X-posted from here

Heard about Large Language Models like ChatGPT4, Bing, GPT3? I'm sure you have.

There is one side of the hype around these technologies that I come across pretty often, which is that these technologies are bad for some $reason ("they are stochastic parrots", "they create bullshit", "they can't reason", "they make up facts", "they might replace junior developers, but they will not replace senior developers"), which, while technically true, is missing a much bigger point: if you are in the business of writing software, these things work.

In fact, they work so well that I think we are on track to see a fundamental shift in how software is built. This is going to have a drastic impact on pretty much everything. The irony of the situation is that programming is probably one of the jobs that can most easily be replaced by these technologies. This article is mostly directed at programmers among my readers: we are at a pivotal moment where we programmers need to understand and appropriate these technologies, on our own terms, before capitalism does it for us.

I have written about how I am convinced that LLMs are a paradigm shift for the pragmatic programmer and started giving more detailed insight into what I have been using these models and the applications around them for. I have also written about my budding ethical stance around this technology, and won't get into that aspect in this post.¹

I think that we are at ground zero of a tremendous revolution in the way we build software. We don't know how to work with these tools yet: we just discovered alien tractor technology. Many critics try to use it as if it was a plain old gardening rake, dismissing it because it plowed through their flowerbed.

I hope to share some of the insights I've gained into what programming with LLMs actually does for me. I have discovered that developing a practice, a methodology, a workflow is key to intellectual work, be it software or writing or music. Because programming is closely intertwined with productive teamwork (especially in our capitalist context), this practice has to be shared. Programming is about coordinating the work of individuals to create common artifacts, and success is determined by how well we coordinate.

I think that programming with LLMs is about to create a radical shift in software architecture, system architecture, programming practices, communication patterns and organizational structures. These are exciting times, because we are the people who are in position to shape what programming is going to be like in the future.

My background is in system programming

I consider myself a programmer first. I've wanted to press buttons to make machines do things since I was 5, and one of my autistic interests has been computers ever since. Even my music, drawing, writing is inextricably bound to these machines of wonder. I have written millions of line² of code and barely a day passes by when I don't push a few commits. All that to say that I'm intensely interested in things that work, and that I have tried many many things. I care enough about coding that if LLMs weren't useful, I wouldn't be using them every day.³

I have been programming heavily with Copilot since it was in beta, and have been using chatgpt for every real-world problem I could think of, beginning the week after it was accessible to the public. I write mostly "boring glue code," and have done so in what I call "systems programming,", i.e. building systems (operating systems, embedded systems, distributed systems, logistics, supply chain). Another definition could be "writing memcpy in many different complicated ways." ⁴

Copilot my uncanny autocomplete

I love "rustic" programming languages: my favourites are PHP and Javascript, Java, Go, (and, for reasons that are beyond the scope of this article: Common Lisp⁵). For me, "rustic" means that what you see is what the author intended, without necessarily a lot of polish: languages that make their context explicit. It's the dinner table built by your grandfather, upgraded by your mother and you now inherit: it is a bit clunky and the paintjob is flaking off, but it has been doing its job for nearly a century without failing. These are languages that Copilot does a superb job with. The patterns it should use are often very "flat," and are often present in the code around it. A symbol's meaning is not influenced by hidden characteristics, abstractions, or module systems. It is not that it is the "complexity" or abstraction-level of the language that makes for inferior results, it's that it is more difficult to infer from the training corpus what the completion of your code should look like.

This is what allows Copilot to do a superb job (while I tried to use it for Haskell and Rust in the summer of 2022, I haven't tried since. Copilot has made some impressive progress since, so my take here might already be completely out of date. I have grown used to Copilot uncannily inferring what I am trying to do after writing the first couple of words. In fact, I have noticed how my physical muscle memory has changed during the programming process. If my internet is down and I—god beware!—have to type in code myself, I have to undergo a mental switch. At first I'll write 3 words and expect 10 lines of code to be scaffolded out. It takes me a few seconds to realize my magical friend Frank the copilot has gone AWOL.

Stochastic parrots, valid code?

One of the common criticisms of Large Language Models is that they often output wrong code. Which is true (ChatGPT4 significantly ups the bar, but it's not too difficult to get it to output wrong code)! Leaving it at that is, however, I think not looking carefully enough. Quickly written wrong code that can be easily corrected is just another name for good code.

Much of my programming consists of writing trivial ideas in longform. I will start with "I need to copy this data from here to here" (which is why I refer to it as "memcpy programming"), and then spell it out in HTTP call this, promise that, SQS event here, batch job there. Just writing the comment call the HTTP api /api/products and send the result to our SNS topic /products is enough for Copilot to basically complete the entire thing.

Because I am not a rocket scientist, the methods are going to be something like (in fake Javascript):



http.GET("/api/products").
   then((products) => if (validateProducts(products)) { 
      this.sns.enqueue() 
    } else { 
      fail("invalid products")
    })

If you autocomplete copilot after writing the comment above, it might very well come up with:



http.GET("/api/products").
   then((products) => if (validator.checkProducts(products)) { 
      const sns = new SNS(this.topic)
      sns.enqueue(products)
    } else { 
      throw Exception("invalid products")
    })

Getting hung up on the fact that it used validateProducts and not validator.checkProducts is I think missing the point. The actual working benefit is that I now usually spend 10 minutes on something that would have taken me 2 h.

I think the implications of that fact stretch much further than "Well, now we have just replaced code monkeys." I think that being able to write tedious code at that speed closes so many feedback loops that it leads to emergent effects that change the way we build software.

Aside #1: Helping Copilot output valid code

Although Copilot is very eager to discover your codebase, it needs to see what style you like. It needs to see what APIs and helper methods you already have at your disposal, and which packages to import.

The solution, if you would like to tab-complete 90ish% of the code you intend to write? Visit a couple of files you want Copilot to "learn", or write one example of what it should generate. If you want Copilot to be fluent at using an obscure library, just go browse its source code, visit a few examples, then go back to frantically tab-complete your way to a working application.

The same technique works for ChatGPT. You want to have a decent output for ChatGPT? Just copy paste a ton of example code upfront. Copy paste your class definitions, the comments to your class definition, maybe some DDL, some example CSV. Paste it often, paste it every time it diverges. Take its code, correct it, paste it back. Paste the documentation page. Paste the entire StackOverflow thread. Paste paste paste, context context context is what is needed here.

Methodology shift #1: writing documentation

This is the first change in methodology we have encountered. More than API documentation for humans, we need to write code (and tools!) that make the API discoverable and "understandable" for LLMs. In most cases, the two come hand in hand. Writing clear and concise comments is what gives the LLM the context it has seen in its training set, thus helping it to infer the correct answer. Clear and concise APIs, with meaningful names, allow us to efficiently (the fewer tokens the better) convey our intent to our tools.

I think we will start seeing practices that will expose documentation as both human readable, and easily "machine parseable" (short, concise, with few-shot examples for concrete tasks). Instead of formal languages such as UML, ODL, SOAP, Swagger, JSON Schema, we will go back to simple, no-nonsense READMEs that give a short overview and a couple of usage examples. This will work not because simple is better (there is a reason we keep reinventing the wheel with these things). It works because the README encodes our "human" intent very well, and the source code embodies what we intend the system to do in excruciating detail, and an LLM can combine the two to generate either more "formal" code for the machine to interpret, or less detailed text for humans to interpret.

I don't think we have realized that the most effective way to communicate with humans is now also an efficient way to communicate with machines.

This of course works both ways. Large Language Models are uncannily effective at transforming crappy comments into well-written, articulate paragraphs. They can generate 5 interesting examples in the time it takes me to say "please." They can update the existing documentation to match the updated API after refactoring in a matter of seconds (and it is only cumbersome because it currently requires copy-pasting to and from ChatGPT). Copilot Labs is experimenting with "brushes", but since I use Intellij and not Copilot, I haven't really used them much).

They can update code to match documentation changes in the same way, or generate valid code out of concise, well-written documentation. Heck, you can literally curl a few endpoints in your terminal, paste the entire thing without any editing into ChatGPT. You can then ask it to create an API library with mocks, unit tests, examples and documentation. This will more often than not output something better than I would have written.

I don't bother manually writing any of this data munching / API wrapping / result validating code anymore. I had to build a server-to-server integration with Google Tag Manager recently. I literally copy pasted the webpage into a simple 3 line prompt and can now generate PHP classes, typescript interfaces, event log parsers, SQL serialization with a simple shell command.

What do we do now that well-written documentation is both fundamental and almost free? Do we become writers? Do we become editors?

Personally, I think so. There is no excuse to not write stellar documentation (or tab-complete your way to it), documentation style and documentation quality will become just as automatable and lintable as how many spaces we put in front of our curly braces, and code comments will never be out of date again (why wouldn't your IDE flag documentation that doesn't match the behaviour of the code?).⁶

And if we become editors, does that mean that learning to work as a programmer is now, from the get go, about learning to read, criticize and correct code? These are the skills that we have painfully turned into practice over the last 30 or 40 years since the explosion of professional software engineering, and are often reserved to the "senior" caste, juniors being busy banging their heads against the compiler or something. But now being a junior means de facto becoming a critical reader. Code reviewing is the new programming. ⁷

Methodology shift #2: Whiteboarding and rubberducking

One thing using LLMs teaches you is that software architecture is about pattern matching, and those patterns are fairly simple. The thing is that by default, ChatGPT for example is akin to a "design interview bullshitter." It will confidently use a lot of clever words, draw the right diagrams on the whiteboard, and be utterly incompetent at making an actually worthwhile point. Asking ChatGPT how to design a certain application will result in eerily similar heaps of teflon-coated, reality-proof platitudes (ChatGPT4 already sets the bar much higher here...).

It is easy to say the right things, it is easy to look up what an effective event-driven architecture looks like, but it is much more difficult to figure out what exactly needs to be done, what is easy, what is difficult, what works well, and what fails in a real-world scenario. But, using the technique shown above, once you start asking ChatGPT how to build out a concrete application by sketching out potential APIs, fleshing out an infrastructure, deciding which protocols to use, you often get a lot of plausible looking, concrete code.

I find generating "plausible" code useful on its own. I don't need to trust the code to be correct—the overall structure and vibe gives me a sense of how this thing will work, what is problematic, and what is clever. It has a tremendous amount of quality code to rely upon to find interesting patterns and well-named classes; I can guide it when I think of something. I basically have a pretty "intelligent" rubber duck at my disposal.⁸ Brainstorming with ChatGPT feels very much like sitting in front of the whiteboard with a colleague and just imagining things, except that you often end up with reasonably close to working code by the end. While I have never done a "three-way" whiteboarding/rubberduck session with a human and ChatGPT, I think that this will become a regular practice for some.

Methodology shift #3: Build more prototypes

ChatGPT is great for generating prototypes, big and small. Ask it about a topic and it will not only answer, but usually provide a fully runnable example in the language of your choice. It might or might not execute, but that's a heck of a lot of really boring nonsense that I don't have to type. Once the LLM has typed out a toy example, I can remodel that simple example into many more things:

A full command-line application
A unit test suite
A fuzzing harness
An example of documentation.
A stress-test tool
A simple web UI ("write HTML for a field that posts to /api/product and display the resulting JSON as a table. Then, write CSS to style it like a geocities page." is how you do it...)
A Docker container to use in CICD
...

LLMs lower the cost of exploration to almost nil.

I recently wanted to write a plugin for OBS that would stop the recording if I didn't close a modal within 1 minute. I had never programmed OBS before, but in three hours I was able to do the following:

Build a Python script that looks very promising
Fight with OBS until I realized that trying to get arm64 python working was too frustrating (1h lost...)
Rewrite the script in LUA and get it working
Realized that what I thought was a weird hallucination was actually the best possible way to do something like a modal in LUA. Decided it sucked.
Write a Go CLI application that controls recording over a websocket.
Write a cross platform UI in Go that shows me the modal and various other buttons to control recording

I was able to try two dead-ends (through no fault of the LLM's own, honestly) and end up with a robust running tool that I will continue expanding. I hate writing UIs, I hate fighting obscure APIs I don't know. I used to only write tools when something was getting so humongously irritating that I just couldn't take it anymore. I have a problematic habit I am actively trying to fight: building personal tools as if they were meant for production at-scale (meaning: after 3 days doing "professional" software engineering, I burn out and the tool ends up in the ditch, full of promise yet unfinished.)

What this means for professional programming is that you can now write code, write a lot of code, write an insane amount of code, and just throw it away. No one will fault you for having a conversation with ChatGPT that generates 5000 lines of code, and then closing the tab. But, the fact is, you wrote 5000 lines of code and decided they were not worth it. When was the last time you did that?

What if it became standard practice to draw up a concise, well-written description of the problem at hand (see the previous section), and then ask the LLM for a microservice architecture sketch in go, a synchronous multithread sketch in rust, a typescript deno version, potentially a lambda. What if you had it generate Terraform for AWS, but also for Azure and GCP? What if an architecture proposal now only gets to the review stage if you try at least A, B, C, D before settling on one, instead of endlessly debating with your colleagues about A vs B? We all know we have biases. We all hold strong opinions that can only be supported by evidence with an n of 1. I am usually more convinced when I see an actual code sketch.

We used to deride "writing code at the speed of mouth," but that is now a reality.

Methodology shift #4: Build more tools

Which brings me to the next practice that I think will be incredibly beneficial. We all know that tools are important, that effective tools are challenging to create, and that management doesn't care or understand the need for tools. LLMs allow us to build tools at the said "speed of mouth." I know that I can now spend 30 to 45 minutes talking to ChatGPT and get a pretty solid tool done. This would probably have taken me 4 to 5 h programming previously, which means that it would have to be split over 2-3 work days (factoring in meetings, code review, lunch breaks, interruptions). Which usually means that the tool won't get built.

A list of tools I built in the last 3 months:

sqleton - a tool to run SQL queries as command line applications
escuse-me - the same for elasticsearch
geppetto - the same for the GPT APIs
biberon - the same for bibtex (very rudimentary, it got one job done)
majuscule - a twitter hashtag segmentation prototype, but with dynamic HTML debug UI
a script to stop OBS recording if I am away from my computer
many utilities for work:
- Convert arbitrary documentation to code (see Google Tag Manager example above).
- Parse and analyze search logs.
- A SQL builder for said search logs.
- A full-featured GTM server-side implementation.
- Tools for managing banners and their assets.
- An application that runs invoices through OCR, cleans it up with GPT, searches ElasticSearch for SKUs, and renders out a cleaned up invoice matching our inventory
- Report generators for every single datalake debugging I had to do
- Documentation generator for Google Tag Manager JSON exports

I can't express how fundamentally different programming feels now that I can build 2 quality tools per day, for every single itch I want to scratch.

Aside #2: Fundamentals will probably be just as critical

I believe that abstraction is primarily learnt by having seen and played with enough concrete use cases that higher structures get formed. Abstractions are a two-edged sword, as an ill-fitting abstraction will cause constant friction. While LLMs allow us to work at a "fuzzy abstraction" level (the abstraction is not ready to be formalized, but it is forming and shaping the words used to describe a problem) as well as quickly explore concrete implementations, "controlling" an LLM is best done by having a solid understanding of the problem in mind.

One very real downside I have experienced with conversational LLMs is the temptation to keep chatting and hacking at a problem in the hopes that the model will "get it" at some point. This is exacerbated when you don't have a proper grasp of the problem you are trying to solve. Your focus shifts from having a productive conversation to stumbling over hallucinations that lead you on wild goose chases. In those cases, I found myself closing my internet connection altogether. Instead, I stuck with offline documentation and a book until I got a better sense of what I was dealing with.

With "real" knowledge in hand, asking the right questions of the LLM leads to a much more productive session. Anybody deriding "prompt engineering" as a ridiculous discipline hasn't spent enough time trying to write effective prompts.

Aside #3: Avoid unproductive chat sessions

I hope that future conversational agents will be able to flag when these unproductive spirals happen. Currently, ChatGPT will just continue to engage, but I could see it at some point stopping, and pointing to proper tutorials and resources. Another option would be to ask for more details. The progress that ChatGPT4 seems to have made (I have only used it seriously for 2 days, so I can't really form a proper opinion) is quite impressive. One thing this field has taught me is not to make assumptions about the capabilities of the next generation. It might have the same fundamental issues the current models have, but it might just become "good enough" that for all intents and purposes, it doesn't matter.

Methodology shift #5: Continuous code review

I think that a major focus of tooling is going to be on "continuous code review." A model can watch you build your software, infer your intents and the structure of your thinking, and provide feedback on your approach. It can flag typos, security mistakes, and non-idiomatic uses of APIs. I was impressed by ChatGPT4 correcting my code and doing some pretty effective factoring of the problem into interfaces and functions by itself. In fact, I would say that in the small, ChatGPT4 is a much better programmer than I am. It knows many more idioms, it doesn't forget security issues, and it can spit out unit tests at the speed of token sampling.

People who think that these models will lead to a proliferation of bottom of the barrel stackoverflow code riddled with security mistakes are missing how quickly these models have become better at what they do. I think it is because it is easy to forget just how much good code can now be found online. Great code is bound to be more widely disseminated in its training corpus, and said corpus is most certainly closely scrutinized and tweaked. The jump from ChatGPT3.5 to ChatGPT4 in terms of software architecture "rhetoric" makes that abundantly clear.

I use a series of prompts that ask the model to give its feedback regarding security issues, edge cases I've missed, documentation that is unclear. As of now, I have to manually copy and paste this into chatgpt, provide missing context, and refine its answers. This is however a matter of engineering. The model itself already does an impressive job. This is infinitely better than most code reviews I've received over my career, and it is instantaneous, along with examples to reproduce the issues found and fix suggestions.

It is now basically easier to write complex, fuzzy linters that can check for domain-specific, codebase-specific patterns, because such prompts consist of a couple of (well-informed) human language prompts. It is probably faster to instruct the LLM-linter to "check that singletons are only used when dealing with customer data" than to properly configure curly-brace behaviour in editorconfig. It won't catch every usecase, but it will catch enough to be worth its while.

Methodology shift #6: Cognitive impact of using LLMs

This might be the most subtle shift, but I believe it's also the most profound change that using LLMs has brought to me. After a day of being focused on getting "the tedious stuff" right (using the right API, checking the right errors, properly calling API X, implementing the unit tests and mocks for Y, writing an API wrapper for Z), my brain would be entirely gone, consumed by the minutiae and intensity required to not slip up. I would spend the evening playing video games or watching Netflix to recover.

Since extensively using Copilot and ChatGPT, this cognitive exhaustion is pretty much gone. 6pm strikes and I feel like I spent the day chatting with a buddy, yet 5 PRs have been merged, the unit tests have been written, two tools have been improved and the code has shipped.

This allows me to make significant progress on my open-source projects. I know that I'll be able to get one non-trivial thing done before dinner, and maybe one or two more after. Where I would previously spend 3 h on Saturday trying to get an AWS lambda running, wondering "why the hell am I spending my Saturday this way", I will now happily close one or two tickets, and spend the rest of the day with my family and doing chores.

Doing more "high-level" thinking

I am a firm believer in letting things rest, and doing "shower-thought-driven" software engineering. I believe it is necessary to think deeply about something, try to build a few prototypes, and then let it settle. The real insights (and bug solving) happen when taking a walk, when my brain is well-rested, after exercising or in the proverbial shower. Whereas I would previously maybe be able to 30 minutes to 1h of this "free" time per day, if at all, by being able to fold the tedious stuff that used to take 4h into 1 or 2h (I am averaging here, but writing something like an API wrapper legitimately costs me 10 minutes instead of 2 days nowadays), I now have 3 to 4h of "free" thinking time per day.

This means that I can think about what needs to be done and what doesn't. It means I can spend some time trying out alternative approaches (build more prototypes, as detailed above). I can spend more time talking to stakeholders and figuring out exactly what we need. I can think about the tools that the team itself needs. I can work on pedagogical material. I can make reports pretty. It is not that these things couldn't be done before, but they have become so insanely cheap that there is no good reason not to do them.

Conclusion

I wish more senior developers had spent the time to properly assess these technologies, with an open mind instead of reacting defensively with knee-jerk reactions. These technologies are scary, because they make it clear that programmers are going to be the first in-line to be replaced by machines.

All the "improvements" listed above are going to be exploited by the way our industry is structured (corporations are already AIs, in some way) to squeeze even more soul-crushing productivity to the benefit of a very few individuals. We have an opportunity right now to reckon with how we deal with these emerging powers. I am trying to work out which direction I want to take, besides writing about what I think is valuable engineering insight: taking these technologies out of the moat of Silicon Valley companies; using the "improved" productivity before companies catch up to figure out how to organize; leveraging LLMs to build better open-source; using these technologies to build tools for people, not business.

Furthermore, while I might slip and anthromorphize LLMs, I do so in the way that I would say that "the compiler thinks." As far as I am concerned, LLMs are relatively straightforward pieces of code trained on a shitton of data, and ChatGPT is not more alive or reasoning or feeling than /bin/ls and my S3 bucket of production logs. It doesn't know, it doesn't remember, it has no intent, it has no interactions with the world. I want us to talk about LLMs as what they are: probabilistic models of language that predict the next token based on a given context, after having been trained on exactly that. The fact that this formulation has led to such things as ChatGPT blows my mind, but I'm not interested in discussing irrefutable speculation. I just want to write code, man... ↩
There's a whole thing about programming vs software engineering that I never fully understood. Surely software engineering is about building good software, and programming is, well, building good software too. When I draw diagrams I am programming. When I write documents I am programming. When I talk with colleagues in front of a whiteboard we are programming. When I read and write a lot of software, mindfully, I do a lot of software engineering, because I want my software to work today, tomorrow, in ten years (which means measuring, testing, benchmarking, breaking, stress-testing, documenting). I want it to work even when 30 people work on it under the pressure of deadlines (which means designing, refactoring, testing, building development workflows, writing documentation, communicating, understanding team structures and business goals and legacy and individual cognitive styles). I call it programming because the only tangible result that we can actually all agree on is the resulting artifacts (source code and documentation). I love working on and reading legacy codebases, and you can read most of the "more abstract" things in the source code itself (on the easier side: team dysfunction, badly aligned goals, insufficient communication; on the harder side: good onboarding, individual contributor growth, great business alignment). That's why I tend to bring up coding so much. You can talk beautifully about all kinds of concepts, but what really matters is the code that comes out. To me, good code is synonymous with good engineering (good code can be: code that solves a business problem, where often no code is actually the best code; code that is elegant (some programmers like making elegant things, most like using elegant things); code that is fun; etc...). ↩
If this comes across as braggy, one thing I learned about the last year after discovering I'm autistic is to not care too much about how I come across. I think I am a pretty terrible, careless coder; I love sharing everything I know: I want everybody to find the love I do when I use computers. I am disappointed that I will never get to be a jack of all trades, because there is just too much cool stuff out there. I will leave mastery to people who have a more focused curiosity. ↩
This seems to be another loaded term where people fight over definitions. I am grouping embedded, some web, distributed, operating systems and some database development under the systems programming umbrella. It seems like a very broad umbrella, but from a programming perspective, I see it as the programming of queues, resource ownership, resource initialization and teardown, concurrency, locking; the programming of protocols, data serialization, storage, bandwidth, throughput, latency; the programming of state machines, coroutines, schedulers, threads, drivers and APIs. When we define "systems programming" from the programming perspective (what code we write), we discover many parallels which are "generative," i.e. they generate insight and ideas through the use of code. This is in contrast to defining "systems programming" as say, the programming of operating systems. This shuts down insight. It causes generations of web developers to reinvent the wheel each time they want to manage minified assets instead of using a resource pool. It leads generations of embedded developers to reject better tooling and effective testing and deployment practices. It leads to a smorgasbord of poorly designed embedded UIs and slow web applications and unpleasant concurrency abstractions and operating systems that think everything is an integer. ↩
You might wonder why I group Common Lisp along with PHP. What makes Common Lisp special as a language is that it allows you to build your own language whenever you need to, however you need to. You can refine it interactively. If your goal is to write code with a clear structure and explicit context, Common Lisp is an excellent tool. Instead of shoe-horning your communication into a ready-made, rigid framework, you build the structure to fit your communication. As we will see, this is what allows LLMs to do a good job. Also, take a look at the Common Lisp spec and tell me it's not... clunky with a flaky paintjob. ↩
You might realize by browsing geppetto's repository that it's just as easy to... not write documentation. Entirely free it is not, that is for sure. Another reason here is that I don't want people using this tool just yet. I have put significantly more effort into glazed's documentation, including building an entire HelpSystem inspired by Mathematica's stellar documentation. ↩
I think people worried about junior programmers becoming obsolete should rather be worried about junior programmers replacing seniors. It's easy to believe that your mind stays younger as you grow older. However, hanging around discord with passionate 15 year olds quickly shows me how calcified I have become. I might think that my elegant take on frontend frameworks and component-based CSS, my ideally crafted git workflow and observability practices are the embodiment of craft. In the meantime, the kids think I could just as well write COBOL while they merge their 30th pull request of the day. This will enable their fully automated CICD to deploy the newly created artifacts. Every random repository that gets shared on Tiktok gets 300 (3000? 30000?) stars within a few days. While me writing opensource back in 1998 meant maybe getting a single patch merged because I knew someone on IRC who cared, today, it means growing up fast AF and becoming an agile tech lead quickly. ↩
As someone who needs to ramble and ramble and ramble at somebody in order to clarify my thoughts, this is genuine magic. One downside is that the tool's "personality" changes over time. It latches onto novel concepts, and heavily redacts or tones down more opinionated statements, often in response to prompting attacks and in order to reduce the amount of hallucinations and other side-effects. This is the inverse of what you want when brainstorming, where far-out ideas are what spark joy. As a work-around, I often prompt the worst models and paste their confabulations into the more reasonable big brother, in order to disrupt the process. ↩

Embedded development is not like web development

Manuel Odendahl — Sat, 05 Nov 2022 15:58:27 +0000

When I wrote Embedded programming is like web development, I thought it would be a pretty uncontroversial article—I expected the response to be along the lines of "geez manuel, that's pretty obvious."

It turns out that that was far from the case—most feedback I got was that both were significantly different. This puzzles me deeply: I switch back and forth between both domains weekly, yet I feel like I am using the same techniques and writing almost identical code.

To gain a better perspective, I wrote up all the reasons I could think of why embedded development is not like web development. I hope that this gives some context to the previous article and helps me provide more clarity moving forward.

Skills of a bare-metal embedded developer

I will use bare-metal microcontroller-oriented work as a stand-in for "embedded development." While a fair amount of embedded development these days involves Linux system programming, cloud computing, graphical UIs, and data processing, I don't want to blur the boundaries too much and muddy my examples.

If you do bare-metal programming, you need to know a certain set of tools and practical skills:

It helps to learn how your CPU works. It is frequent to change architectures on different projects, and there is a wide range of peripheral buses and architectures that need to be relearned. While programming languages stay the same, there is a lot of datasheet reading involved.
You must be comfortable with various physical tools: oscilloscope, logic analyzer, soldering iron, and breadboards.
You need to know at least rudimentary electronics to pinpoint electrical bugs and know how to use the oscilloscope. More advanced electronics knowledge is useful for building peripheral circuits.
Running and debugging your program requires a debugging probe.
You have to be able to read and step through assembly language. The compiler output often demands special scrutiny.
You will have to deal with intermediate board versions that are often broken in subtle ways (wrong traces, signal noise, bad power supply).
You usually work with real-world prototypes (for example, a set of motors on your desk instead of the real machine). These not only exhibit their own quirky bugs but often are incomplete.

The code and runtime requirements are very different:

Codebases are usually small and self-contained.
There is low code reuse, and open-source libraries are fairly uncommon (although that is changing).
Much of the code is interfacing with peripherals and bus protocols, requiring a lot of reading and comprehending datasheets.
Memory management is typically manual, and memory is scarce: dynamic allocation and fancy data structures are usually out.
Parts of the system have real-time constraints, often hard real-time.
Doing IO usually involves dealing with interrupts, having to program peripherals by reading and writing memory-mapped IO, and often using DMA and timers. This is asynchronous by nature; there is often no OS to provide blocking semantics.
There is no or limited persistent storage.
Debugging usually perturbs the system badly enough that elaborate debugging strategies have to be devised.
There is not that much material out there to learn from. Communities are fragmented and scarce (this has changed over the years, with the influx of developers growing up on Arduino and now with the Rust embedded movement).

Skills of a web full-stack developer

Similarly, web development has its own tools, technologies, workflows, and constraints.

A lot of different technologies are involved in a single project. You will need to know some CSS, HTML, SQL, and Linux skills in addition to "pure" programming. You will often interface with a plethora of external APIs.
There is a lot of code reuse. You often work against big frameworks like Django, React, and Vue. These frameworks move quickly, and there is always some new hype around the corner.
There are a lot of tools: you manage packages, minify, lint, type check, bundle, transpile your front end, migrate your database, scaffold new components, run development containers, do remote debugging, and deploy and monitor your services in production. These tools evolve at the speed of the modern open-source community, which can be overwhelming.
Programming trends and techniques evolve often; there is a lot of hype-driven iteration. Conversely, there are a lot of learning materials, conferences, and communities out there to help you stay up to date.
Your code interfaces with a lot of third-party systems and services. Even a simple web application usually deals with a web server, a cache, a database, a load balancer, cloud storage, a logging service, and more. More complex systems deal with container orchestration, microservices, message buses, secrets management, and edge computing.
Much time is spent working with UX, graphic design, and marketing, so much so that some of these functions are sometimes part of engineering teams.
Your system runs on pretty beefy machines (browser, cloud servers). You can be fairly loose with runtime and memory requirements and use a lot of "layered" frameworks.
While asynchronous code is fairly common, you are offered a lot of synchronous APIs to simplify your code, especially in backend work.
Products tend to be significantly more complicated, with complex UX workflows, large-scale deployments, and the integration of many services.
The actual runtime environment of your system is often significantly different from the development or staging environment. There will be emergent behavior at scale that can't easily be replicated on a developer's machine.
Products are continuously observed and maintained while they are running. There usually is a dedicated ops team responsible for maintaining the application's runtime.

So, why did I say that embedded development is just like web development?

I think that the day-to-day of a programmer differs significantly depending on which domain they work with. The set of skills, the onboarding, and the learning experience are tremendously different. You can't just put a junior embedded developer before a web application and have them contribute meaningfully.

However, once you become a seasoned developer (at a senior+/staff level), your work becomes much more abstract: you think in systems more than in the minutiae of actual code.

In fact, with the decreasing amount of coding you do, you can quickly become rusty. It's been years since I have written a kernel driver or a bare-metal firmware, and I have to fight with webpack every time I get back into front-end development. My CSS knowledge—poor in the best of times—evaporates every few weeks. However, I know that I can pick up whatever current knowledge is required to build them, and I know where they fit in the system and what tradeoffs can be made. I can also provide guidance and advice to more junior developers on the team and help them grow.

Overall, higher-level patterns transpose very effectively from one domain to the other. Someone knowledgeable about "real-time" WebRTC, WebSockets, HTTP, and gRPC will have no trouble understanding the nuances of USB or how a network protocol might fit the problem at hand. Similarly, a frontend developer that is knowledgeable in building complex asynchronous applications using state reducers, promises, and state machines will have no problem designing and code reviewing some more intricate firmware—even writing one after having been shown the nitty-gritty of flashing and debugging by an experience embedded develop.

I hope to give concrete code examples to make this admittedly abstract statement much more tangible in my next few posts.

Web development is just like embedded development

Manuel Odendahl — Thu, 03 Nov 2022 13:21:10 +0000

I often see people discuss embedded development and web development as being two opposite sides of the spectrum of software engineering: one is low-level and hardcore, and the other one is high-level and almost trivial. After more than 20 years working in both domains, I have come to view web and embedded as highly similar—I can use the same approaches in both worlds.

The usual discourse goes as follows:

Embedded development is complex and open to a select few experienced developers. It requires precise thinking and strict software engineering.
Web development, on the other hand is easy, relaxed, mainly concerned with making things pretty and throwing CSS, javascript frameworks, and SQL at a CRUD application

Web applications and embedded systems are both distributed systems

In an embedded system, you have sensors, actuators, multiple microcontrollers, often a Linux system, increasingly standard cloud connectivity, and over-the-air updates. In a web application, you have client browsers, servers, storage services, databases, caches, logging, and monitoring. These components need to communicate to present the user with the desired functionality.

Furthermore, these components need to be monitored and maintained. The system is usually permanently online. Building software updates is intricate; distributed and update processes often require orchestration. Debugging at scale usually requires monitoring and log collection, as interacting with any individual component is often impossible.

Web and embedded software engineering is remarkably similar

At a more technical level, the software and architecture of web and embedded systems are built on the same underlying abstractions and must fulfill similar constraints.

At a higher abstraction level:

The user application is implemented by a distributed system
Actors communicate over a network, using a variety of protocols
The application is long-lived, requiring special attention to error handling
Most actions require a sequence of asynchronous steps
Deployment at scale requires synchronization because of interdependent components
Because of their complexity, systems often exhibit emergent behavior
The execution environment is remote (local web development usually requires some docker setup; embedded development requires some playground for the actual sensors and actors)
Performance is an essential component of system design due to power, real-time, response time, or cost constraints

The similarities are even more striking once we zoom in on the code level:

User and hardware interaction often require complex behavior due to UX design,
Most of the code is sequencing asynchronous steps
There is no happy path. Because at least part of the system runs without direct user supervision, we need at least restarts and error logging
The same software patterns prove very effective:
- state machines, promises, or coroutines
- event-driven development
- structured logging and observability metrics
- structured RPC IDLs
- monoidal state handling (state reducers like redux store)
Instrumentation for debugging is extremely useful, yet can quickly impact runtime behavior
Many issues can only be caught in production (or in-field testing). Mocking only goes so far due to the emergent behaviors mentioned above

There is a lot to learn from both fields

I have learned a lot from working in both domains and have carried over not just ideas but actual code and tooling. Framing embedded development as low-level systems requiring advanced skills versus web development as a free-for-all, low-hanging fruit obscures how similar they are.

Both are hard to get right, both benefit from a holistic systems approach, and actual code and tooling carry over surprisingly easily.

I plan this to become a longer series that explores what I learned in my career in web and embedded, so stay tuned!

(cover image: ranzino flickr)

Don’t be arrogantly ignorant: don’t flame

Manuel Odendahl — Sun, 16 Oct 2022 17:27:22 +0000

There is a widespread tendency in technology to discuss specific programming languages (and their frameworks) with open contempt. Who hasn’t heard people scoff at how stupid PHP is, how backward Java programmers are, and how criminally short-sighted C++ users are?

Nothing infuriates me more than this open gatekeeping, this active bullying. Putting people down is not valid technical criticism: its only goal is propping oneself up as part of an elite in-group. It is blatantly exclusionary.

Discussing technology should be a dialogue driven by compassion and respect. Not putting in the effort to understand why people made a particular choice, what constraints drove them to it, and most importantly, what lessons you can learn from them. Borrowing the expression from a friend, you are being arrogantly ignorant.

Knee-jerk criticism considered harmful

Gatekeeping is the act of denying people access to resources that you have access to. When you derisively denounce a programming language as wrong or harmful, you are framing its users as stupid and incapable; you are implying they are not worthy of working in technology and should leave.

Such statements show a screaming lack of empathy: everybody starts as a beginner and tries to make the best of what they know — you certainly did, not that long ago. A person learning to program at a young age on their personal computer, with parents in engineering, and then attending a prestigious school is just as worthy of being a programmer as someone attending a boot camp at age 35 and then hustling WordPress themes. Expecting the latter to know about dependent types because the first had the opportunity to work at that level of abstraction is shortsighted at best; privileged, and entitled at worst.

PHP is one of my favorite languages, not just because I got into web programming in the wayward days of PHP2, but because of how empowering it is to beginners. When you write PHP, you always have a running website — the language feeds and sustains your motivation. Fighting pip, setting up a remote environment in WSL, and having to learn vim is why many people drop out or feel they don’t belong.

With the widespread adoption of WordPress, you can, in a very short amount of time, become a professional developer: building websites for local businesses, selling themes and plugins on its marketplace, or joining one of the many consultancies and freelance marketplaces. It is hard to stress how transformative this can be for marginalized communities and people in countries outside of western Europe and the US. PHP and freelancer.com allowed me to go from making sub-minimum wage to being able to make rent in an afternoon.

PHP changed orders of magnitude more lives than Haskell.

Gatekeeping as the actual technological sin

Off-handedly rejecting PHP, Java, WordPress, Javascript, and other oft-criticized languages and frameworks is a sign of a closed mind; of willing ignorance.

People write code trying to operate to the best of their capabilities. They make choices according to what they know, the industry's current state, the constraints of the problem they are trying to solve, the pressures of the market, and their personal lives. Viewed through these lenses and given how wildly successful these technologies are shows how capable and driven their users are.

The only valid reason to publicly criticize a decision is when it purposefully harms people or was made by someone who fully knew that there was a better choice but chose not to for personal gain. I often went for architecture choices that I thought were not perfect: out of time pressure; to help a colleague grow by having them make their own choices; so I could hire from a broader, more diverse community.

Rejecting a programming language outright makes you a poorer technologist. It closes you off to huge, vibrant, diverse communities. The lack of humility causes you to miss the creativity of engaged, practically-minded users. You might never experience the quality of their tools and how good their documentation and onboarding can be. A tool with 200,000 users and 500 contributors will quickly become better than something maintained by 3 people in academia. Beginners who get pull requests on their project in the first week promptly improve their software engineering game.

Being dismissive shows that you are missing the larger context of being a programmer: building products, turning concepts into reality, and working and interacting with others. A programmer is much more than a code monkey narrowly defined by their choice of technology.

By poo-poo-ing these popular languages and frameworks, not only are you keeping people from becoming better professionals; from having access to the knowledge and resources that you have—you are actively stunting your personal growth.

You are staying ignorant out of arrogance.

Instead of criticizing, engage and share

Of course, valid criticism is essential and healthy. While everything is a matter of choice and compromise, some options are better than others; some decisions are obvious mistakes (in hindsight). Learning new languages and approaches is an essential path to growth.

Criticism is valuable when everybody participates in good faith, with mutual respect. One should criticize concrete, technical points—critique shouldn’t be aimed at people and why they made a particular choice. Core contributors to languages such as C, Haskell, Typescript, and PHP have candid, deep, and respectful conversations. They know they each come from a different background, embrace each others’ choices, and know they can learn from them.

Instead of putting a language and its users down, consider how you and your interlocutors can enrich each other's lives. We all love programming, working in tech, and learning. If we understand that we each have our journey, skills, motivations, successes, and struggles, we can all make better choices in the future.

I will forever remember the day someone on IRC praised me for my janky PHP website and told me that I would enjoy “Structure and Interpretation of Computer Programs” by Abelson and Sussman, which transformed my career. Had they told me I was an idiot, that PHP sucked, and that I should be ashamed for having written such blatant SQL injections, I might never have gotten to where I am today.

Conclusion

The next time you feel the urge to criticize something or someone, take a breath and reflect on where that urge comes from: do you want to feel superior, or do you genuinely want to help someone?

We should all aspire to make technology more welcoming — the best way to do so is to approach every situation with humility, compassion, empathy, and a love of technology and its communities.

If you think you know better, then act like it.

Writing made me a better engineer

Manuel Odendahl — Thu, 04 Aug 2022 14:04:00 +0000

Last year, I did one of the best things I could have done: I started writing regularly to become a better software engineer. Not only did I become a better communicator, but it also taught me how to think strategically.

Learning to write

In early 2021, I started writing for myself.

I wrote down problems I was working on, sketched out solutions, and documented what I was learning. I drafted meeting notes, conversations with my manager, and ideas I wanted to share with my team. Putting words to the editor allowed me to shape and manipulate thoughts as discrete entities; before, I was operating on gut instinct and fuzzy mental pictures.

I realized that writing about engineering is hard because good writing requires good thinking.

Now, I was able to write software just fine before! I achieved a lot by following my instinct, experience, and inner sense of order. However, I couldn't easily explain my goal or why I was pursuing it.

Learning to edit

I wrote a lot. I took detailed notes while programming. I recorded thoughts I had while on walks. I scoped out entire projects in one session. But I would often return to a pile of half-baked ramblings the next day. Half of the time, I couldn't remember what I was writing about. I couldn't just write down my ideas and rely on them later.

This taught me I was my primary audience.

I had to edit what I wrote, tighten it, and clean it up. I had to convince my future self that my ideas were sound or at the very least understandable. I added more structure to my project documents. I started naming the patterns I used, describing the pros and cons of specific approaches, and exploring alternatives.

Would an event-driven design work? Would our data lake scale? How much would storage need to grow?

Through writing, I learned to distill my intuitions into coherent concepts. I was able to articulate long-term goals and how to achieve them. I read https://www.amazon.com/Good-Strategy-Bad-Difference-Matters/dp/0307886239 and discovered that this is called "strategy."

Learning strategic thinking

Writing about strategy was even more challenging.

A good strategy needs a clear goal vision, a comprehensive view of opportunities and obstacles, and a solid understanding of the technical landscape. You can leave many things unsaid when only writing code and rudimentary project documents. Nobody reads pull requests, JIRA tickets, incomplete meeting notes, and reasons about the big picture. Even an exhaustive, thorough code review won't expose strategic flaws.

Yet strategy is precisely what needs to be discussed and revised most carefully. Maybe that code doesn't need to be written at all!

A lousy strategy impeccably executed is useless, while a good strategy with a lackluster implementation does wonders.

Communicating strategy

I have always been effective at steering legacy codebases towards impactful business goals. But I could never explain how I did it, not to myself, not to stakeholders, not to teammates. This meant that reviewing strategy was impossible.

Instead, we had meandering whiteboard sessions and slack threads; architecture and implementation efforts were pulling in different directions; a lot of glue code was required.

Once I started to focus on writing, things fell into place. After a few months of practice, I wrote a concise three-page strategy document outlining our data engineering strategy. A few critical sentences laid out how to align incoming requirements with long-term goals, acting as a lighthouse for more detailed work.

I started writing similarly short RFCs (see https://dev.to/wesen/quick-tip-tuesday-writing-rfcs-for-fun-and-profit-3bo). I had to persevere for months, writing dozens of documents before the process was adopted for all new development.

Writing for better engineering

Today, I use writing for every engineering task I work on. It allows me to think clearly over long periods. It allows me to make and document decisions and give them the depth and long-term vision they deserve.

(Cover: Battle of Borodino - Baron Lejeune - Wikimedia)

20+ years into my career, I still love PHP and Javascript

Manuel Odendahl — Mon, 01 Aug 2022 13:17:00 +0000

Over the last twenty years, I have used over a dozen languages professionally, from C to Common Lisp, from Java to Python, from C++ to Typescript.

Yet, I love janky programming languages. In particular, I really enjoy PHP and Javascript.

Here's why.

They are used by people who get shit done.

This makes it easy to find people who:

understand business needs,
can iterate quickly
have shipped and maintained many projects in the past.

There is a lot of FUD that has been cargo-culted throughout the years.

PHP8 is not your grandmother's PHP3, ECMAScript 12 is not your grandfather's Javascript.

Their ecosystems haven't stood still either. Tools like CPAN, PECL, npm have put significant evolutionary pressure on other languages. With its daily new paradigms and frameworks, Javascript has been a hotbed of innovation for years.

Their quality is offset by other factors.

These languages are not popular for their elegance, but for other reasons.

These can be:

speed
ease of deployment
malleability
tooling
community
frameworks
platforms

PHP is a favorite of mine because of its cgi-bin model:

legacy codebases can be refactored in pieces
resource leaks are non-existent
caching and scaling are simpler

They are fun.

Because basic areas of the language are problematic areas means that programming comes with a sense of experimentation. There is pleasure in finding ways to write elegant code in a language that doesn't prescribe any.

Janky programming languages are great equalizers.

Because they are quite permissive, they allow people of all levels and backgrounds to get started with programming. You can be a complete PHP beginner and get a real website going within days.

Having started by copy-pasting BASIC snippets and later on making my own websites with PHP2 and PHP3, I feel pride in being able to pay back by helping newcomers where they are.

Poo-poo-ing a 12-year-old or a recent BootCamp graduate because they are writing actual code is missing the point.

In fact, because users are builders and learners because they have diverse backgrounds, janky languages tend to have very active and welcome communities.

I never had issues writing good code.

PHP, Javascript, Lua, Perl all provide effective tools:

functions
variables
classes
first-class functions
sometimes traits and prototypal inheritance.

Good software is:

product alignment
software design
naming and design patterns
enforcing invariants through static analysis or unit testing
providing fast and effective feedback loops to developers
good team communication

None of which depend on the language.

Javascript and PHP in particular come with:

solid libraries
pragmatic static analysis tools
great IDE support

In fact, they have better tooling than many a "serious" programming language.

I love legacy codebases.

A legacy codebase means that the product is performing well. It means that I can often make immediate and impactful improvements.

For me, nothing comes close to the pleasure of improving a product with many users.

(Cover Image: CC BY-SA 4.0 - Jcak - Wikimedia)

Diagrams are the ultimate engineering tool

Manuel Odendahl — Sat, 30 Jul 2022 20:13:35 +0000

Since I was a little kid, diagrams have fascinated me.
As I grew as a software developer, I discovered how much more there is to them.

Diagrams are engineering tools

Diagrams are visually beautiful. They also embody the promise of a hidden, complex yet elegant mathematical underworld. An image can replace 1000 words – a diagram can replace 1000 ideas.

Diagrams are everywhere:

business strategy
chemical processes
control systems
project planning
fantasy football
category theory
database schemas
electrical circuits
...

Diagrams are not only omnipresent across disciplines. They are pervasive within disciplines. When we collaborate, ideate, and write documentation, we create diagrams. They are immediate and malleable.
Changing an arrow to point from A to B instead of C; drawing a box around E and E – a user can make drastic changes instantly.

That diagrams are ubiquitous is not a coincidence.

Diagrams are the way humans model the world

Diagrams are usually about the same size: five to ten objects connected by at most a dozen arrows. More complex relationships require nesting.
Us humans need abstraction to break things down into manageable amounts of "things."

Arrows compose: an arrow from A to B, followed by an arrow from B to C means that changing A will impact C.
We understand the world causally: if arrows didn't compose, the diagram wouldn't make sense.

Feedback loops are immediately visible. Participants with no formal background understand that something special is happening.

Research in category theory shows that many diagrams correspond to categories. Often, these are symmetric monoidal categories.

Diagrams allow us to work across disciplines

The implications are enormous. Engineers in one field can make significant contributions to other fields. They are experienced working with similar diagrams. Questioning what an arrow means in a state machine diagram, pointing out that a connection is missing: these are incredibly subtle and valuable insights. Yet people with no software background routinely make these observations.

Diagrams are powerful

Take diagrams seriously. They are the most powerful software engineering technique I know.

How do you use diagrams in your work?

(Cover image by Jazz Di Mauro)

Asking stupid questions

Manuel Odendahl — Sat, 02 Jul 2022 14:13:47 +0000

Learning to ask what you think are stupid questions is not easy.
Asking "stupid" questions requires a lot of confidence.
We have been conditioned to not admit weakness and ignorance; deconstructing the ingrained feelings of shame is not an easy task.

Over time, I realized that questions people think are stupid usually fall into three categories.

1. Questions about random tidbits of knowledge

Some people feel bad about asking questions like:

what's the method that does X in language Y?
what's the command to run the unit tests?
how do I install postgresql on ubuntu?

Well, no one is able to recall every random fact about computers.

Knowing how to do something is transient: it goes in and out of working memory;
if an answer can't be found with 10 minutes of searching, it makes sense to ask someone else.
If you encounter a question the first time, you might not know what to search for anyway.

Of course, If you find yourself asking people the same question repeatedly, it might be time to write a documentation page or put some post-its on your screen.

A question is only stupid if you:

don't put in an appropriate amount of effort before asking
don't try to learn from the response

2. Questions that require mental chunks

There are questions where the answer is kind of obvious, but only if you have acquired the right mental representations.
This takes practice, effort and experience.

Once you have these mental representations, it is easy to forget the hard work that was required to acquire them.
All that remains is a sharp "uh, obvious" feeling.

When asking questions that rely on built-up insight, you will often not be aware of the concepts you are lacking; thus, miss how valid and valuable your question actually is.
Googling will not serve you well – the knowledge you are missing is not just a 3-step instruction list, but requires deep learning.

I will try to illustrate this with an example from web development.

Imagine someone starting out as a frontend developer, asking "how do i make the title text red?".
That's an extremely not stupid question.
While one could go "duh just use color: red; in your CSS", this presupposes knowing:

what HTML is
what CSS is
what a CSS rule is
what a CSS selector is
an understanding of how these rules get applied when loading a page

People who come up with these concepts on their own usually end up writing a PhD thesis.

Answering these questions well requires a tremendous amount of pedagogical sensibility.
You need to recognize where the person asking the question is at, and get them started on the right path forward.
This can be both unblocking them in the immediate, while giving them pointers about the concepts to look up and study in their learning time.

Asking these questions in a public setting is a mixed bag, as people might not recognize what is going on.
However, these questions are great when pairing (pair programming).
If anything is unclear while you are pairing, and you feel the person you are pairing with is a good partner, ask every single stupid question that comes to your mind.
That is what pairing is all about.

3. Questions with no good answer

There are many questions that just have no clear-cut answer at all.
If you were to ask an expert in the field, they'd probably pause for a few seconds and go "it depends".
For example, a question like "where should i save the user state in my react app?" not only elicits many strong responses, but the community consensus changes every couple of years.
Anybody with a hard and fast answer doesn't have a full understanding of the depth ot the topic.

Every answer is going to make a series of tradeoffs.
Choosing the "right" answer often transcends the technical, and goes into a more general "software engineering / architecture"; it often involves team politics as well.

"Stupid" questions like 3. are questions you should never shy away from asking in an supportive setting (pairing again is a perfect opportunity).
Either a compromise has been reached, and learning about said compromise is extremely important to be able to navigate the project well.
Or a compromise hasn't been reached, and that often points to a bigger issue at play.

Over time, you will slowly get a sense of which questions fall into that category.

Asking stupid questions confidently, keeping a beginner's mind, is a skill to be practiced and strengthened.
The best way I know of to develop this muscle is to continuously learn new things. It is the essence of learning at the edge of your capabilities – being "stupid" and asking the questions to become less "stupid".

Asking stupid questions also requires a safe environment; you need to feel supported and know that showing vulnerability is not going to harm you. This is sadly especially true in a professional setting, and even more so if you are part of a section of the population that is routinely discriminated against. I have written a bit more about this in Imposter syndrome is about systemic discrimination, and it is a topic I plan to write more about in the future.

(first published at the scapegoat dev - asking stupid questions

Two things I find difficult

Manuel Odendahl — Thu, 30 Jun 2022 04:39:24 +0000

In web development:

synchronizing state between frontend and backend
distributing state inside a frontend application

In embedded software:

getting the LED to blink at the right speed
I2C

In software architecture:

communicating with people who don't think in systems
getting team buy-in while keeping a coherent vision

In data science:

cleaning and indexing CSV files
recognizing and explaining the 800 ways data can lie

When learning:

finding the right balance between practice and theory
making up exercises when none exist

(titile image: wolfgangfoto - CC)

What I learned about writing and workflow during the pandemic

Manuel Odendahl — Mon, 06 Jun 2022 12:26:42 +0000

Over the last 1.5 years, I have been working at getting better at writing and communication. I started using the gorgeous Bear application on my iPad. Using beautiful tools has always inspired me (so much so that I will often go for looks over functionality). Bear is not only gorgeous, it also comes with a big bag of features: markdown, tags, wiki-style linking, apple watch app, beautiful exports, automatic syncing.

As I was getting to know Bear, I experimented with different kinds of writing. Having found a productive workflow as a musician over the pandemic, I tried to apply the concepts that worked for me in the studio to my new endeavor.

Daily process

The most important part of being productive making music was to sit down and do some work everyday, even if just for one hour. Over time, the consistent work accrues and you reap seemingly enormous benefits, while never really having felt the need for inspiration to strike or having to do long grueling sessions.

The advantage of doing consistent short sessions is that it never feels bad to stop when things get frustrating - you did your work for the day, tomorrow will bring new joy.

I divided this "creative workflow" in roughly four big activities.

Collecting and janitorial work

The first area is what I call process the "janitorial" part. It is learning new techniques, searching for samples, recording field recordings, making patch collections (preset sounds), sorting files, wiring things up with cables, installing software updates. This is stuff that you absolutely don't want to do when in the middle of a more creative session, as it is a sure way to bring you out of the flow.

For writing, it means note-taking, gathering reference material, picking up and jotting down random ideas, collecting quotes. I started taking notes for articles I came across the internet, making little entries for topics I knew things about, all loosely connected through wiki-links.

While researching Bear and note-taking, I came across Andy Matuschak's work and the ide aof keeping a Zettelkasten, to which I will come back to in a future post. I started keeping smaller "atomic notes" and cross-linking them. I used tags to keep my notes organized. I ultimately migrated from Bear to Obsidian for managing my notes (more about this in a future article), and you can see a "public" part of my vault at mnml's vault.

Creating raw material

The next activity is creating raw material.

Musically, this would be coming up with beats, song ideas, recording synth progressions. It is not about making a song, it is about having fun making music in a very restricted context. In fact, the more restrictions are placed on the activity, the easier it is to get started. A prompt would for example be "make 10 beats with this one set of drum sounds". Often, a seemingly banal restriction would spark inspiration and I would go on for hours.

Freeform writing

Creating raw written material means finding ways to deal with the fear of the blank page. In order to turn off the internal critic, I practiced writing in a stream of consciousness style. without worries about the outcome. I wrote free-form about software architecture, about bugs I was encountering, about project planning ideas. Over time, getting into the writing zone became effortless. The times I start feeling paralyzed, I usually am anxious about the final form of what I am trying to write.

Keep a collection of prompts

I keep a long list of prompts (for example Blog ideas for blog articles) for different areas. I have a long list of prompts for work related topics (different technologies that could be interesting, marketing ideas, broader thoughts about product and workflow, links to articles I found interesting). If I feel stuck or can't seem to find a way to create a seedling, I will choose a prompt in one of my ideas documents and try to write down a stream of consciousness about it.

Using daily notes

In order to separate writing from an outcome, I do most of my writing in a daily note. It is completely free-form, and I will often paste images, drawings, website captures, conversations along with longer segments of writing. I keep it messy for a reason: purposely not try to make things tidy or think about a potential reader.

Copy pasting chat threads

Interestingly, another technique for creating raw material that works for me is just copy-pasting the long ramblings I post on various chat apps and social media pages. It is a variant of the rubberduck technique, in that here thinking about a reader actually stimulates creativity. I need to feel like I am having a conversation in order to refine my ideas. I would often paste slack screenshots or twitter threads into my notes.

Taking notes on the go

Finally, the apple watch of Bear.app was invaluable. I usually have a lot of ideas while riding my bike or running, and with the watch I could easily record these straight into my daily note. The speech recognition was always a bit of a mess, and I learnt over time to take much longer notes than I thought would be necessary at the time. Nothing worse than coming back to a note that says "bathroom architecture tomato pixie" and have no idea what that was supposed to mean.

Properly file the created material!

An absolutely vital part of creating raw material is to make that raw material easy to reuse in the future. This means naming files correctly, filing them in ways where they can be found later on, adding proper metadata, saving related resources nearby. This is more important for music than for writing, but even for writing, I make sure to cleanly label my files, link them in a way that makes it easy to recall them, link additional documents such as project files for graphics and diagrams, correctly quote referenced material.

I put everything on dropbox so that I can access it quickly from wherever I am, even on other people's computers. It also makes it easy to share work in progress if I feel like collaborating.

Assembling

In the studio, the next part of the process is finding different musical ideas and sounds and making them work together. This is when songs start to appear. The key here is to have quick recall of all the raw material created in the previous step. This is where the janitorial work comes from. If everything is nicely labeled, well sorted, backed up, documented, it becomes very easy to stay in the flow while assembling. Having a lot of raw material also helps.

A fascinating thing is that musically, the best ideas often come from the least inspired sessions. If I was feeling very inspired and excited making beats or certain synthesizer sounds, this usually meant that after a night of sleep, the result sounded horribly cliche. Sessions where I really had to labor to come up with something interesting provided much richer material in the long run.

Assembling writing

I feel that writing is similar, although I have much less experience in that domain. Notes and raw writings need to be easily retrievable. They also need to "age". This means that the value of an idea or a piece of writing becomes more apparent over time. If something is still interesting after 6 months, or you can find it expressed in 5 different ways in multiple notes, it probably touches upon some deeper meaning.

Both musically and in writing, I think of assembling as working with scissors and glue. It is not creating as much as it is just moving things left and right and trying out different juxtapositions. This is where using digital media helps tremendously. Copy, paste, duplicate, erase, undo, redo, save copies. All these things are tremendously tedious with real tape or pen and paper.

Furthermore, using tags and wiki-links and the search functionality in Bear.app, I could quickly retrieve my notes and turn them into bigger documents. This easy recall of previous notes and references gave my writing a new depth.

Editing

The final part of the process is editing. After heated creation and assembling, a fair amount of semi-finished ideas have accumulated. They are raw songs or articles, unfinished, but roughly of the right length (or longer), with the main idea clearly present. At this point, I choose one or two of the best ideas present, and I set down to work to actually finishing them.

This is almost mechanical work. It is about editing sentences, shortening paragraphs, looking up references, reading for flow. In music, it is cleaning up recordings, adding missing elements, mixing the sounds spatially, making sure the flow and structure is correct. The work is technical and requires an alert mind, but it doesn't rely on inspiration. You can just turn on the computer and do it, a bit like doing the dishes.

Interestingly, for outsiders, this is where "the magic" happens. It seems that all the rubbish writing, the wonky sounds and ideas suddenly fall into place and turn into something magical, sometimes within minutes. But really, the magic is in the workflow. It is present in labeling notes, in collecting web links, in writing free-flowing rubbish, in assembling individual paragraphs. The editing pass is akin to putting a beautiful cake into the oven, and then waiting. You know it will come out great if you are just careful enough to take it out at the right time.

Quantity over Quality

In both creating raw material and in assembling, I always try to go for quantity over quality. The quality of the output is not directly related to the quality of what the work that gets done at each step. The quality of the output is only determined by the selection that gets done before editing, and by the experience and ruthlessness of the editing itself.

I would say that for my music, around 2% of the material I create ultimately gets released. But the 98% are what made that 2% worth releasing.

What are your techniques for writing?

What about you? Do you have a workflow for writing? Did the pandemic help you become a better writer? What do you think you could improve?

How I Got Into Computers - Thomson MO5, My First Love

Manuel Odendahl — Tue, 31 May 2022 15:03:26 +0000

1987, Geneva, Switzerland.

My grandmother and I are walking in the city center and pass an underground garage entry. We pass a large ticket vending machine. Right in the middle of the front panel, ominously red, there is a button. Five year old me evidently presses that button; not even a second later, a light goes on and a distorted voice comes out of a speaker, yelling something I don't understand. I panic, my grandmother quickly yanks me to the side and tells me harshly not to press random buttons again.

I know better: when I am a grown-up, I will press random buttons and make things happen, and noone will be able to stop me.

1988, Aumale, a small town in rural Normandie, France.

My first grade class (in fact, the town was so small that first grade and fifth grade were combined into a single classroom of 20) go to the town's community center. It is a small room lined with TVs and the French answer to the ZX80: the Thomson MO5. France in the 80ies had a strong penchant for Digital Utopianism, as evidenced by the Minitel, and pervasive computer education in schools.

When turned on, the machine instantaneously boots into a BASIC prompt. You can either write your program cold, or load it from a ROM cartridge or a standard tape cassette. Our teacher plugged in the LOGO cartridge and the magic turtle appeared on screen. AVANCE 50 DROITE 50 RECULE 50 GAUCHE 50 was my first program. The software I liked the most was the "story engine", where you could could fill in different words into a skeletal tale.

While we never really learned how to program, this was a clear indication that pressing random buttons was a valid career path.

1991, Strasbourg, France.

Around 1989, my parents saved up and bought a used 8086 without a graphics card; while it had no hard drive, it had two floppy drives, immensely better than a slow tape reader. It came with DOS and Wordperfect on 5 1/4" floppies; along with DOS came a bunch of other tools: GWBASIC and DEBUG.COM were to have a lasting influence. While I didn't get into programming until later in 1991, I did read the Wordperfect manual back to front (it came in beautiful blue binders), and learned to program macros. I can remember the tutorial contents about the Versailles castle to this day.

In the early 1990ies, with a 8086 and no graphics card, there weren't many video games to play with. We had Tetris; my father also got ahold of a copy of Leisure Suit Larry (which interestingly enough is rendered as text with the ASCII extension character set). That was it.

Leisure Suit Larry allowed me to deeply hone my English language skills. I also learned a lot about American presidents - knowledge I needed to pass the age test at game start (I later learned you could also just press F11). I did make significant progress in the game, but let's say that it only had a few pulsating, blocky characters to offer in lieu of titillation.

Due to the invasion of home computers in the 80ies, the ZX80, the C64, the Thomsons, the Ataris, the Amigas, public libraries were flooded with introduction to programming books for kids (well... clearly targeted at boys. For an illustration of sexism in french society, here is a quite entertaining look at TV ads). These books were filled with little robots explaining what a variable is, and beautifully illustrated program listings.

Gone were the days of solving riddles in seedy Vegas hooker bars. I could now copy the following code from Usborne's Space Games and be escape from aliens in my spaceship.

I still can't convey how exhilarating this was.

I graduated to writing a full adventure game following the instructions in this book: Write your own adventure programs (Usborne). The BASIC didn't line up, the computer architecture neither, and I didn't really understand that at the time. It took me many months to get it to run, but I ultimately. I have since carried that book with me everywhere as a talisman, a reminder of the dreams that got me into programming.

2022, Boston MA, USA.

30 years and a lot of button pressing to make things happen later: Here I am, about to turn 40, in a big city on a far away continent, directly across the ocean from that small town in Normandie, flush with computerbucks, facing mortality, searching for the meaning of life, circling the drain of my midlife crisis. And I say to myself, "Oh heck Manuel, YOLO", rewind the first tape of MANDRAGORE, and type RUN"".

15 excruciating minutes of tape loading later, I know why writing your own games was so popular. It wasn't hard to do better than this: