DEV Community: Ayron Wohletz

Running Prisma migrate in an Electron app

Ayron Wohletz — Sun, 13 Feb 2022 15:19:52 +0000

See https://github.com/awohletz/electron-prisma-trpc-example for a template that demonstrates this approach.

I use Prisma Migrate in my Electron app. Here's a mechanism to detect when I need to run migrations on app start.

In main.ts, I check if the DB needs creation (first app run) or if it needs a migration:

let needsMigration;
const dbExists = fs.existsSync(dbPath);
if (!dbExists) {
    needsMigration = true;
    // prisma for whatever reason has trouble if the database file does not exist yet.
    // So just touch it here
    fs.closeSync(fs.openSync(dbPath, 'w'));
} else {
    try {
        const latest: Migration[] = await prisma.$queryRaw`select * from _prisma_migrations order by finished_at`;
        needsMigration = last(latest)?.migration_name !== latestMigration;
    } catch (e) {
        log.error(e);
        needsMigration = true;
    }
}

Here, latestMigration is a hardcoded constant:

// This needs to be updated every time you create a migration!
export const latestMigration = "20220213142643_added_col_index";

Note that running migrate on an empty, merely-touched DB file will create the DB tables. There's no need to package a hardcoded template database to start with.

Then I run the migration if needed:

if (needsMigration) {
    try {
        const schemaPath = path.join(
            app.getAppPath().replace('app.asar', 'app.asar.unpacked'),
            'prisma',
            "schema.prisma"
        );
        log.info(`Needs a migration. Running prisma migrate with schema path ${schemaPath}`);

        // first create or migrate the database! If you were deploying prisma to a cloud service, this migrate deploy
        // command you would run as part of your CI/CD deployment. Since this is an electron app, it just needs
        // to run when the production app is started. That way if the user updates AriNote and the schema has
        // changed, it will transparently migrate their DB.
        await runPrismaCommand({
            command: ["migrate", "deploy", "--schema", schemaPath],
            dbUrl
        });

        // seed
        log.info("Seeding...");
        await seed(prisma);

    } catch (e) {
        log.error(e);
        process.exit(1);
    }
} else {
    log.info("Does not need migration");
}

seed is just a function that inserts initial data relevant to my app:

export async function seed(prisma: PrismaClient) {
    await prisma.workspace.upsert({
        create: {
            id: "default",
            name: "Default",
        },
        update: {},
        where: {
            id: "default"
        }
    })
}

Workflow

With this, to make a schema modification I follow these steps:

Edit the schema.prisma with the changes to the schema.
Create the migration with command: prisma migrate dev --name added_col_index. I'm naming it added_col_index in this particular example since I added a column.
Update latestMigration constant in the code with the generated migration folder name: export const latestMigration = "20220213142643_added_col_index"
Build, pack, and distribute the app as usual.
When users update the app, the updated code will contain a new latestMigration, which will make the app run Prisma Migrate on start, because the user's database does not contain that migration (in _prisma_migrations table). Prisma Migrate compares the migrations available in the prisma/migrations folder to the migrations run in the user's DB. It runs the new migration.
App opens the browser window as usual. The user only notices a little slower start time.

This has the manual step of updating latestMigration, but since I rarely create migrations, it works well enough.

To automate it, I could probably write some code that looks at the migrations folder (in the packed app) to see the latest migration file and compare that to the latest migration run in the DB.

It's actually not necessary to detect whether a migration is needed. I could just run migrate on every app start. However, that adds noticeably to the app startup time.

How I sign and notarize my Electron app on MacOS

Ayron Wohletz — Thu, 16 Dec 2021 20:13:04 +0000

Distributing a desktop application for MacOS requires signing it and getting it notarized by Apple. MacOS won’t let the app run otherwise.

I distribute my Electron app outside of the App Store, via download on my website. If you’re distributing via the App Store, the steps may be different than what I have here. Note I am based in the United States, and I did all this on a new (as of 2021) Mac Mini with M1 chip.

Here are the steps I took to get my Electron app signed and notarized:

Sign up for the Apple Developer Program and get a Developer ID
Create Apple Developer ID and Apple Developer Application certificates, download them, and install them in your keychain
Once those certs are in your keychain, electron-builder will sign automatically while building
Use a script to notarize the app after signing

Let’s go over each step.

1. Sign up for the Apple Developer Program

Go to the Apple website to enroll in the Developer Program: https://developer.apple.com/programs/

This comes with a cost. At the time of this writing, it was 99 USD per membership per year.

Enrollment gives you two options: Individual or organization. I enrolled as an organization (using my registered LLC). This is because I like to conduct business under my business name, to benefit from the legal structure that an LLC gives me. Enrolling as an organization has more requirements for verification than an individual. I needed a DUNS number, a website, and a custom domain email. Apple called me to verify my business.

2. Create and install certificates

After completing enrollment and verification, I got a developer account on developer.apple.com. I signed in on there, went to Certificates, IDs, & Profiles.

I created two certificates, one of Developer ID Application type and one of Developer ID Installer type. Once created and download, a double-click on the file should install them in your Keychain.

3. Sign while building

With those certificates in my keychain, here’s relevant config from my electron-builder.yml:

afterSign: notarize.js
mac:
  category: public.app-category.productivity
#  gatekeeperAssess: true
  target: [dmg, zip] # zip is required for auto-updating because of electron-userland/electron-builder#2199
  entitlements: ./entitlements.mac.plist
  entitlementsInherit: ./entitlements.mac.plist
  hardenedRuntime: true

Apparently electron-builder automatically uses the certificates in my keychain, because I don’t have to mention them in the config.

I did have to create an entitlements.mac.plist file. Otherwise my app, when run, would request random weird permissions that it didn’t need. I looked at the docs and some open source examples (e.g. Athens) and came up with this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <!-- https://github.com/electron/electron-notarize#prerequisites -->
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <!-- https://github.com/electron-userland/electron-builder/issues/3940 -->
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
  </dict>
</plist>

This may or may not be appropriate for your app. One requirement of my app is to run third-party executables (Prisma) that I’ve bundled with the app.

4. Notarize

The final step to getting the app to run on Mac is notarization. There are some good tutorials online on how to do this with electron-builder that I referred to (see https://philo.dev/notarizing-your-electron-application/ and https://kilianvalkhof.com/2019/electron/notarizing-your-electron-application/ ).

In the electron-builder config above, see the line afterSign: notarize.js. The notarize.js script I took from here:

require('dotenv').config();
const fs = require('fs')
const path = require('path')
const electron_notarize = require('electron-notarize');

module.exports = async function (params) {
    if (process.platform !== 'darwin') {
        return
    }

    console.log('afterSign hook triggered', params)

    let appId = 'com.funtoimagine.arinote'

    let appPath = path.join(
        params.appOutDir,
        `${params.packager.appInfo.productFilename}.app`
    )
    if (!fs.existsSync(appPath)) {
        console.log('skip')
        return
    }

    console.log(
        `Notarizing ${appId} found at ${appPath} with Apple ID ${process.env.APPLE_ID}`
    )

    try {
        await electron_notarize.notarize({
            appBundleId: appId,
            appPath: appPath,
            appleId: process.env.APPLE_ID,
            appleIdPassword: process.env.APPLE_ID_PASSWORD,
        })
    } catch (error) {
        console.error(error)
    }

    console.log(`Done notarizing ${appId}`)
}

With this, electron-builder signs and notarizes my app. The app is then ready for distribution.

Here’s my package.json script to do a production build on Mac: cross-env NODE_ENV=production npm run test && npm run prod-build && electron-builder --mac

And this does a build and publish to my Github releases: dotenv -- electron-builder -p always.

The notarization step takes 5-10 minutes for my app.

See How I code signed an Electron app on Windows for the Windows equivalent of this process.

How I code signed an Electron app on Windows

Ayron Wohletz — Tue, 14 Dec 2021 17:33:17 +0000

So you want to distribute your Electron app to Windows users, but when people run it they get a nasty warning “Microsoft Defender SmartScreen prevented an unrecognized app from running” and it shows “Unknown publisher.”

One way to immediately get rid of this warning is to purchase an Extended Validation (EV) Code Signing certificate. I opted instead for a normal, non-EV Code Signing certificate. A normal Code Signing cert will show the publisher name in the warning instead of “unknown publisher.” However, SmartScreen will still need time to begin trusting my app before it will stop showing the warning altogether. I made that tradeoff to reduce the cost and validation time for my small app.

So here’s how I used electron-builder to sign my Electron app on Windows with a Code Signing certificate. It took three steps:

Obtain the certificate
Download the certificate
Configure electron-builder

1. Obtain the certificate

I purchased a Microsoft Authenticode Code Signing Certificate from Sectigo on this website: https://codesigncert.com/codesigning/authenti-codesigning.

To issue the certificate, the certificate vendor needs to either validate you as an individual or validate your business. I opted for business validation. For business validation, your business must be listed in public records that the vendor can look up. Your business phone number has to be listed as well. If the info is not listed, the validation process gets longer and more complicated.

After I purchased the certificate, codesigncert.com gave me some instructions to kick off the validation process.

It had me create a Certificate Signing Request (CSR) with DigiCert Certificate Utility.
I uploaded that CSR (which is just a bunch of text) to the codesigncert.com website. The info entered into the form on the website has to match what you inputted in the DigiCert tool.
Now the certificate vendor will begin the validation process. In my case, it was Sectigo.

Codesigncert.com provides some info on the validation requirements — address, phone number, public listing, etc. To make my business info plus phone number visible before starting validation, I updated my business profile on dnb.com. Their customer support crew sent me a couple messages during validation that they needed to validate my business phone number. I called them and directed the operator to my dnb.com listing. The operator was then able to validate and issue the certificate. All-in-all the purchase and validation process took a couple weeks.

2. Download the certificate

After the certificate is issued, I had to download it with Internet Explorer (IE11). Yes, it’s strange, but the certificate only downloads in the correct form with IE. At first I downloaded it with Chrome (my usual browser) but the downloaded file didn’t contain the private key. That tripped me up for a couple hours. Eventually I found that IE is a requirement. Then I followed the Sectigo tutorial to export the certificate to pfx, which electron-builder needs.

3. Configure electron-builder

First set the environment variables CSC_LINK and CSC_KEY_PASSWORD.

CSC_LINK — This should contain the path of the .pfx file you downloaded.
CSC_PASSWORD — This should have the .pfx password you configured while downloading from IE.

Electron builder will pick those env vars up and sign all the executables in your bundle (including third-part executables in asar.unpacked). I set those in my dot-env file (.env) and used the package dotenv-cli to load them in package.json scripts. Here is the package.json script that builds and signs: dotenv -- cross-env NODE_ENV=production npm run test && npm run prod-build && electron-builder

And this script builds, signs, and publishes to my Github releases: dotenv -- electron-builder -p always

Here’s the relevant electron-builder config that I used:

win:
  target:
    - nsis
  signingHashAlgorithms: [ 'sha256' ]
  publisherName: (YOUR BUSINESS NAME HERE)
  signAndEditExecutable: true
  verifyUpdateCodeSignature: true

After electron-builder has built and signed your executable, you can verify the signing with DigiCert Util if you want (press the “Check Signature” button).

Summary

Windows now shows my business name as the publisher in the SmartScreen warning when users run the app. It’s nice to know that my app cannot be tampered with and faulty copies sent to users. According to online materials I’ve seen, the warning should go away altogether once enough people download and run it.

I also code signed the app for MacOS. That was a separate process, and I couldn’t use the cert I bought for Windows. See How I Sign and Notarize my Electron App on MacOS for details.

How to separate concerns in code

Ayron Wohletz — Mon, 08 Nov 2021 17:00:06 +0000

We developers often praise separation of concerns. Neatly separated systems are easier to work with, because you can change one part without breaking anything else. Separated logic has wider applicability, because it's not tied to irrelevant context. However, it's not always obvious how to achieve this separation. Here I will share a heuristic for separating concerns.

I call that heuristic "varying the world." To vary the world means to consider what may change and then isolate the changing parts in their own modules. I picked up the idea years ago from the C2 wiki, where a certain Carl R. Castro wrote about "The Principle of Essential Representation". Carl wrote in the context of Object-Oriented design, but the heuristic could apply to many types of systems. Here I apply it to JS functions.

Example #1

Let's start with a typical NodeJS procedure. Here's one that reads user data from a CSV file, calculates some statistics, and then writes the result to another file.

import * as fs from "fs";
import {maxBy, map} from "lodash";

interface User {
    name: string;
    followers: number;
    posts: number;
    joinedOn: Date;
}

interface UserStats {
    numUsers: number;
    maxNumFollowers: number;
    mostRecentlyJoined: string;
}

function calculateUserStats() {
    try {
        const userData = fs.readFileSync('users.csv', 'utf8');
        const userRows = userData.split("\n").slice(1, -1); //assuming header row and blank last row
        const users: User[] = map(userRows, row => {
            const [name, followers, posts, joinedOn] = row.split(",");
            return {
                name,
                followers: parseInt(followers),
                posts: parseInt(posts),
                joinedOn: new Date(joinedOn)
            };
        });

        const stats: UserStats = {
            numUsers: users.length,
            maxNumFollowers: maxBy(users, u => u.followers).followers,
            mostRecentlyJoined: maxBy(users, u => u.joinedOn.getTime()).name
        }

        fs.writeFileSync("./stats.json", JSON.stringify(stats, null, 4));
        console.log("Done writing stats.json");

    } catch (err) {
        console.error(err)
        process.exit(1);
    }
}

calculateUserStats();

So with this users.csv:

name,followers,posts,joinedOn
Priya,2503,100,11/7/2020
John,300,5,10/1/2019
Georgette,2503,100,5/2/2018
Ayron,9000,1000,1/1/2021

It generates this stats.json:

{
    "numUsers": 4,
    "maxNumFollowers": 9000,
    "mostRecentlyJoined": "Ayron"
}

Now with this calculateUserStats, think about alternate realities/worlds/environments/contexts that it might be used in. In other words, think about what might change. There is an alternate reality where we read user data from a JSON file instead of CSV file. We might run this function in an AWS Lambda or on a local desktop computer. It could be called synchronously or asynchronously. The statistics we might want to write to a file, return to the caller, or do something else.

This varying of the world reveals the changing parts of calculateUserStats. All these changing parts are bundled together in the same function. This narrows the function's range of applicability. You can only use it in environments that have a local file named "users.csv". It can only be called synchronously, blocking the caller. It restricts the result to be written to a file.

Varying the world sometimes also reveals parts of a function that don't change among the different contexts. We can call this the "essence" of the function. This is usually domain logic that our application has to perform to solve the user's problem.

I would say the essence of calculateUserStats is taking some user data and producing some statistics. I.e. represented by this code:

const stats = {
    numUsers: users.length,
    maxNumFollowers: maxBy(users, u => u.followers).followers,
    avgNumPosts: avg(map(users, u => u.posts)),
    mostRecentlyJoined: maxBy(users, u => u.joinedOn.getTime()).name
}

No matter where or how or when we run it, this is the essential task. This is the domain logic. Our user wants these statistics, and they don't care how we do it. We could do it with this code, by a human using pen and paper, or by a clever arrangement of gears and levers.

With the results of our varying the world thought experiment, we can start separating parts of this function one-by-one. Let's start with the method of obtaining the user data. Clearly that can vary. So let's parameterize it:

function getUsers(): User[] {
    try {
        const userData = fs.readFileSync('users.csv', 'utf8');
        const userRows = userData.split("\n").slice(1, -1); //assuming header row and blank last row
        return map(userRows, row => {
            const [name, followers, posts, joinedOn] = row.split(",");
            return {
                name,
                followers: parseInt(followers),
                posts: parseInt(posts),
                joinedOn: new Date(joinedOn)
            };
        });
    } catch (err) {
        console.error(err)
        process.exit(1);
    }
}

function calculateUserStats(users: User[]) {
    const stats = {
        numUsers: users.length,
        maxNumFollowers: maxBy(users, u => u.followers).followers,
        mostRecentlyJoined: maxBy(users, u => u.joinedOn.getTime()).name
    }

    fs.writeFileSync("./stats.json", JSON.stringify(stats, null, 4));
    console.log("Done writing stats.json");
}

calculateUserStats(getUsers());

The "decoupling" and "separation of concerns" is starting to emerge – calculateUserStats no longer has to know or care where users comes from.

Ok, how about the output? What we do with the stats can vary. So let's extract that part:

function getUsers(): User[] {
    ...
}

function writeStats(stats: UserStats) {
    fs.writeFileSync("./stats.json", JSON.stringify(stats, null, 4));
    console.log("Done writing stats.json");
}

function calculateUserStats(users: User[]): UserStats {
    return {
        numUsers: users.length,
        maxNumFollowers: maxBy(users, u => u.followers).followers,
        avgNumPosts: avg(map(users, u => u.posts)),
        mostRecentlyJoined: maxBy(users, u => u.joinedOn.getTime()).name
    }
}

writeStats(calculateUserStats(getUsers()));

calculateUserStats is now a pure function. It has no side effects – no connections or entanglements with the outside world that we need worry about. Give it the same users and you'll always get the same result. It doesn't know where you get the users or what you do with the user stats. This makes it a portable, testable, decoupled unit of domain logic.

We could bring this further, for example, by considering that the users array may not fit in memory. calculateUserStats assumes a world in which it does. So we could make it accept an abstract stream interface. Also, we could separate writeStats and getUsers further.

Example #2

Let's take getUsers from above. I can see at least four things that may vary:

Reading from a file with a specific name (users.csv) – where do we locate the file?
Parsing the CSV string – what is the format of rows and cols?
Error handling – what do we want to do in case of an error?
Sync or async – do we want to block the caller or no?

Here's my first pass at splitting it up:

// 1. Reading from a file with a specific name (users.csv)
function readFile(filename: string): Promise<string> {
    return new Promise((resolve, reject) =>
        fs.readFile(filename, 'utf8', (err, data) => {
            if (err) {
                reject(err);
            }
            resolve(data);
        }));
}

// 2. Parsing the CSV string
function parseUserCsv(userCsv: string): User[] {
    const userRows = userCsv.split("\n").slice(1, -1); //assuming header row and blank last row
    return map(userRows, row => {
        const [name, followers, posts, joinedOn] = row.split(",");
        return {
            name,
            followers: parseInt(followers),
            posts: parseInt(posts),
            joinedOn: new Date(joinedOn)
        };
    });
}

async function readAndParseUserCsv(filename: string): Promise<User[]> {
    const userCsv = await readFile(filename);
    return parseUserCsv(userCsv);
}

// 3. Error handling
async function getUsersOrExit(filename: string): Promise<User[]> {
    try {
        return await readAndParseUserCsv(filename);
    } catch (err) {
        console.error(err)
        process.exit(1);
    }
}

// Tie it together in a high-level interface if desired
const getUsers = () => getUsersOrExit("users.csv");

This splitting (or "slicing into layers") has opened up combinatorial possibilities for the caller. Now the caller, if they already have a CSV string from some other source, can call parseUserCsv. Or if they want to handle errors in their own way, they can call readAndParseUserCsv. Or if they just want to read a file into a string for some other purpose, they can call readFile. They are not forced into an all-or-nothing interface like the original getUsers. In sum, the code has become more flexible, composable, and testable.

What is the essence of getUsers? I would say it doesn't have one in terms of the domain. Of course, we have not defined what the domain is for these examples. So let's say that the user needs these statistics to solve some business problem they have. They do not care where we get the user data from, how we store it, etc. The mechanics of how we do that isn't "essential" to the domain. However, we could consider that getUsers has its own mini-domain at its level of abstraction and think about what is essential/inessential from that perspective. For example, maybe we call it the domain of CSV parsing.

Avoid over-engineering

I'll end with a word of caution. There should be a good reason for separating and decoupling. Pursuing it as an end in itself leads to over-engineering. For example, in the cases above, if calculateUserStats was just a script I run once in a while, it's overkill to separate to this degree. Also, it's easy enough to just wait and separate this code if the need for it actually arises.

The vary the world heuristic is just a rule of thumb, a way of thinking, not an exact science. It's more about trying it, getting a feel for it, and being aware of the options for a procedure or function or module or system. I almost never explicitly think about it in low-level programming anymore, instead relying on intuition of what will be needed.

Have an opinion on separation vs over-engineering? Leave a comment or message me on Twitter.