Database Functions in Elm-Time - Easy Database Updates in Production

Originally published at https://michaelrätzel.com/blog/database-functions-in-elm-time-easy-database-updates-in-production

Last week, I added this new feature in Elm-Time, helping you manage and update production databases using custom Elm functions.

It's called 'Database Functions' and allows you to apply Elm functions on your database.

You can see it in action in this demo video:

https://youtu.be/9mFjdf_ABNM

The Problem of Updating a Database Online in Production

Elm-Time is the runtime for the Elm programming language that radically simplifies the development and operation of web services and full-stack web apps.

Although running a backend on Elm-Time is generally a smooth experience, I was not satisfied with the options for manually editing the Elm application state.

For example, I sometimes add items to user accounts as special rewards or delete an account. This means changing the database that is running online in production. The database transactions to make these changes should be quick, as I want to avoid service interruptions. And I also want to avoid a complex preparation: The smaller the project, the more likely I prefer manually entering these changes instead of wiring it up in the application program code.

What had been our options to update a database online in production?

In general, the admin interface is the way to make changes without involving the app's main update function. In earlier versions, the options here were the backup/restore APIs or a migration.

Using the backup and restore APIs for such an update is impractical: If we don't pause the processing of application events, we would effectively erase updates between reading the old state and setting the new state.

Updates via migrations, on the other hand, guarantee isolation and consistency. As we discovered earlier, we support applying a migration function on any deployment, not only when the state type changes.

So yes, we can use a migration function to update the database correctly. But this is unwieldy as it requires running a deployment. The idea for today's solution has been around for some time. And this month, I felt frustrated enough to prioritize it.

So how does the new feature make this easier?

It enables us to apply update functions with custom arguments using the admin interface.

Using Database Functions in Elm-Time

To make a function available on the admin interface, we place it in an Elm module named Backend.ExposeFunctionsToAdmin.

Let's look at an example that exposes three function declarations this way. Below is an excerpt from an example application at https://github.com/elm-time/elm-time/blob/b0062e0c43250b5b49c82fbc4c740ccccffe9bb2/implement/example-apps/database-demo/src/Backend/ExposeFunctionsToAdmin.elm

module Backend.ExposeFunctionsToAdmin exposing (..)

import Backend.State
import Dict
import UserAccount


listNewUserAccounts :
    { accountMaximumAgeInDays : Int }
    -> Backend.State.State
    -> List { accountId : String, accountAgeInDays : Int }
listNewUserAccounts { accountMaximumAgeInDays } state =
[...]


deleteUserAccountById : String -> Backend.State.State -> Backend.State.State
deleteUserAccountById userAccountId stateBefore =
    { stateBefore
        | usersAccounts = stateBefore.usersAccounts |> Dict.remove userAccountId
    }


userAccountAddCreditBalanceEvent :
    { userAccountId : String, addedCredits : Int, reasonText : String }
    -> Backend.State.State
    -> Backend.State.State
userAccountAddCreditBalanceEvent { userAccountId, addedCredits, reasonText } stateBefore =
[...]

Since we have placed them into the Elm module with that magic name, Elm-Time makes these functions available via the admin interface. That means whether you use the command-line or graphical interface, the authorization works the same as for deployments.

In the command-line interface, we can use the new commands list-functions and apply-function.

In the graphical admin interface, we find the exposed functions in the new 'Database Functions' section.

Update the Database - Apply a Function

If you want to follow along, you can use this command to run the same demo app:

elm-time  run-server  --admin-password=test  --deploy=https://github.com/elm-time/elm-time/tree/b0062e0c43250b5b49c82fbc4c740ccccffe9bb2/implement/example-apps/database-demo

Let's take a closer look and apply the simplest function in our example, deleteUserAccountById. This function has two parameters, the userAccountId and the stateBefore of the database.

The second parameter bound to stateBefore is special because its type equals our application state type. The runtime understands this and automatically fills in the current application state as the argument value. That leaves the first parameter, userAccountId, for us to supply manually.

As we can see in the type annotation, the return type also equals the app state type. That means applying this function has the potential to change the application state. Sometimes, we want to do a dry run before changing our database's main branch. For that reason, the interface lets us discard the resulting value instead of committing it to the database.

To make the change go live, we enable the checkbox 'Commit resulting state to database'

In the CLI, we set the corresponding flag using the --commit-resulting-state on the apply-function command.

After using the 'Apply Function' button in the GUI or the apply-function command in the CLI, we get a report about the results of the function application.

Conclusion

In summary, we can now quickly make a manual update to an online database without the need to go through the main update function of the Elm app.

Since this feature is brand-new, I assume the interfaces will evolve and look different soon. We will see. The functionality I showed here has been implemented and is available since version 2023-05-07. For me, this solution made operating backends much more relaxed.
I am looking forward to hearing your thoughts!

Comments

[druchan]

hey, just stumbled on elm-time. seems like some phenomenal work getting a frontend-specific language play with backend APIs and databases.

i was trying to find some sort of an introductory stuff to elm-time but couldnt. can you point me to something that I can use to make myself familiar with the background/philosophy of elm-time?