DEV Community: Kirill Konshin

The ultimate Electron app with Next.js and React Server Components

Kirill Konshin — Mon, 21 Oct 2024 06:53:02 +0000

With the emergence of React Server Components and Server Actions writing Web apps became easier than ever. The simplicity when developer has all server APIs right inside the Web app, natively, with types and full support from Next.js framework for example (and other RSC frameworks too, of course) is astonishing.

At the same time, Electron is a de-facto standard for modern desktop apps written using web technologies, especially when application must have filesystem and other system API access, while being written in JS (Tauri receives an honorable mention here if you know Rust or if you only need a simple WebView2 shell).

I asked myself, why not to combine best of both worlds, and run usual Next.js application right inside the Electron and enjoy all benefits that comes with React Server Components?

Demo

I have explored all options available and haven’t found a suitable one, so I wrote a small lib next-electron-rsc that can bridge the gap between Next.js and Electron without running a server or opening any ports.

All you need to use the lib is to add following to your main.js in Electron:

import { app, protocol } from 'electron';
import { createHandler } from 'next-electron-rsc';

const appPath = app.getAppPath();
const isDev = process.env.NODE_ENV === 'development';

const { createInterceptor } = createHandler({
    standaloneDir: path.join(appPath, '.next', 'standalone'),
    localhostUrl: 'http://localhost:3000', // must match Next.js dev server
    protocol,
});

if (!isDev) createInterceptor();

And configure your Next.js build in next.config.js:

module.exports = {
  output: 'standalone',
  experimental: {
    outputFileTracingIncludes: {
      '*': [
          'public/**/*',
          '.next/static/**/*',
      ],
    },
  },
};

Here’s the repository: https://github.com/kirill-konshin/next-electron-rsc and the demo with all files.

And here’s my journey to create this lib.

Motivation to use React Server Components in Electron

Electron’s native way of providing access to system APIs is via IPC or, god forbid, Electron Remote (which was considered even harmful). Both were always a bit cumbersome. Don’t get me wrong, you can get the job done: this and this typed IPC interfaces were the best I found. But with IPC in large apps you’ll end up designing some handshake protocol for simple request-response interaction, handling of errors and loading states and so on, so in real enterprise grade applications it will quickly become too heavy. Not even close to elegance of RSC.

Important benefit of React Server Components in traditional client-server web development has same nature: absence of dedicated RESTful API or GraphQL API (if the only API consumer is the website itself). So developer does not need to design these APIs, maintain them, and app can just and just talk to backend as if it’s just another async function.

With RSC application all logic can be colocated in the Web app, so Electron itself becomes a very thin layer, that just opens a window.

Here’s an example, we use Electron’s safe storage and read from/write to file system right in the React Component:

import { safeStorage } from 'electron';
import Preview from './text';
import fs from 'fs/promises';

async function Page({page}) {
  const secretText = await safeStorage.decryptString(await fs.readFile('path-to-file'));

  async function save(newText) {
          fs.writeFile('path-to-file', await safeStorage.encryptString(newText));
  }

  return <Preview secretText={secretText} save={save} />;
}

Such colocation allows much more rapid development and much less maintenance of the protocol between Web and Electron apps. And of course you can use Electron APIs directly from server components, as it’s the same Node.js process, thus removing the necessity to use IPC or Remote, or any sort of client-server API protocol like REST or GQL.

Basically, this magically removes the boundary between Electron’s Renderer and Main processes, while still keeping everything secure. Besides you can shift execution of heavy tasks from browser to Node.js, which is more flexible how you distribute the load. The only problem is… you need to run an RSC server in Electron. Or do you?

Requirements

I had a few and very strict requirements that I wanted to achieve:

No open ports! Safety first.
Complete support Next.js: React Server Components, API Routes (App router) and Server Side Rendering, Static Site Rendering and Route Handlers (Pages router), you name it, with strict adherence established patterns
Minimal, easy to use, based on standards, basically an enterprise-grade, production ready stack for commercial use, mature and well known set of technologies
Performance

After some research I found an obvious choice called Nextron. Unfortunately, seems like it does not utilize the full power of Next.js, and does not support SSR (ticket remained open in Oct 2024). On the other hand there are articles like this or this, both very close, except for usage of server with an open port. Unfortunately I only found it after I came up with the approach I’m about to present, but the article validated it. Luckily I found it before writing this post, so I can give kudos to the author here.

So I started exploring on my own. Turned out, the approach is pretty simple. And all the tools are already available, I only needed to wire them together in some unorthodox way.

Next.js

First step would be to build Next.js app as a standalone. This will create an optimized build which contains all modules and files that can possibly be required in runtime, and removes everything that’s unnecessary.

module.exports = {
  output: 'standalone',
  experimental: {
    outputFileTracingIncludes: {
      '*': [
          'public/**/*',
          '.next/static/**/*',
      ],
    },
  },
};

Aaand, this is it for Next.js.

outputFileTracingIncludes is needed so that optional public and .next/static folders will be copied to standalone build. Next.js assumes you should publish this to CDN, but in this case everything is local.

Next step is a little trickier.

Electron

Now I need to let Electron know that I have Next.js.

One possible solution is Electron’s Custom Protocol or Schema. Or a Protocol Intercept. I chose the latter as I’m perfectly fine to pretend to load web from http://localhost (emphasis on pretend as there should be no real server with an open port).

Besides, this also ensures relaxed policy of one “popular video service”, that forbids embedding on pages served via custom protocols 😅.

Please note that I purposely excluded a lot of unnecessary code to focus on what matters to show the concept.

To implement the intercept I added following:

const localhostUrl = 'http://localhost:3000';

function createInterceptor() {
    protocol.interceptStreamProtocol('http', async (request, callback) => {
        if (!request.url.startsWith(localhostUrl)) return;
        try {
            const response = await handleRequest(request);
            callback(response);
        } catch (e) {
            callback(e);
        }
    });
}

This interceptor serves static files and forwards requests to Next.js.

Honorable mention here goes to awesome Electron Serve, which implements a custom schema for serving static files.

Bridging Electron and Next.js

Next step would be to create a file to provide some convenience to use the non-existing port-less “server”:

import type { ProtocolRequest, ProtocolResponse } from 'electron';

import { IncomingMessage } from 'node:http';
import { Socket } from 'node:net';

function createRequest({ socket, origReq }: { socket: Socket; origReq: ProtocolRequest }): IncomingMessage {
    const req = new IncomingMessage(socket);

    req.url = origReq.url;
    req.method = origReq.method;
    req.headers = origReq.headers;

    origReq.uploadData?.forEach((item) => {
        req.push(item.bytes);
    });

    req.push(null);

    return req;
}

createRequest uses a Socket to create an instance of Node.js IncomingMessage, then it transfers the information from Electron’s ProtocolRequest into the IncomingMessage, including the body of POST|PUT requests.

import { ServerResponse, IncomingMessage } from 'node:http';
import { PassThrough } from 'node:stream';
import type { Protocol, ProtocolRequest, ProtocolResponse } from 'electron';

class ReadableServerResponse extends ServerResponse {
    private passThrough = new PassThrough();
    private promiseResolvers = Promise.withResolvers<ProtocolResponse>();

    constructor(req: IncomingMessage) {
        super(req);
        this.write = this.passThrough.write.bind(this.passThrough);
        this.end = this.passThrough.end.bind(this.passThrough);
        this.passThrough.on('drain', () => this.emit('drain'));
    }

    writeHead(statusCode: number, ...args: any): this {
        super.writeHead(statusCode, ...args);

        this.promiseResolvers.resolve({
            statusCode: this.statusCode,
            mimeType: this.getHeader('Content-Type') as any,
            headers: this.getHeaders() as any,
            data: this.passThrough as any,
        });

        return this;
    }

    async createProtocolResponse() {
        return this.promiseResolvers.promise;
    }
}

ReadableServerResponse is basically just a regular Node.js ServerResponse from which I can read the body once Next.js finishes the processing. createProtocolResponse converts the ReadableServerResponse into Electron’s ProtocolResponse.

createProtocolResponse method returns a Promise which waits for the body and resolves into converted ReadableServerResponse as ProtocolResponse.

Next step is finally the “server” itself.

No server, no ports

import type { ProtocolRequest, ProtocolResponse } from 'electron';

export function createHandler({
    standaloneDir,
    localhostUrl = 'http://localhost:3000',
    protocol,
    debug = false,
}) {
    const next = require(resolve.sync('next', { basedir: standaloneDir }));

    const app = next({
        dev: false,
        dir: standaloneDir,
    }) as NextNodeServer;

    const handler = app.getRequestHandler();

    const socket = new Socket();

    async function handleRequest(origReq: ProtocolRequest): Promise<ProtocolResponse> {
        try {
            const req = createRequest({ socket, origReq });
            const res = new ReadableServerResponse(req);
            const url = parse(req.url, true);

            handler(req, res, url);

            return await res.createProtocolResponse();
        } catch (e) {
            return e;
        }
    }

    function createInterceptor() { /* ... */ }

    return { createInterceptor };
}

I use the NextServer from Next.js app’s standalone build to create a handler, a regular Express-like route handler which takes Request and Response as arguments.

Key function here is handleRequest.

It provides a dummy Socket to createRequest to create a dummy IncomingMessage, creates a dummy ReadableServerResponse. I feed both request and response to Next.js’s handler, so Next.js can work its magic, not knowing that there’s no actual server, just dummy mocks. Once handler finishes its job the ProtocolResponse is ready for Electron to send to browser. And this is it.

Note that I don’t actually start the Next.js or any other server anywhere, so Requirement #1 is achieved, no ports are open. You can take a look at Next.js documentation to learn more about regular way of setting up a handler with the server. And since I use regular Next.js way, Requirement #2 is achieved.

And since this whole approach works fine on highly loaded servers, and with Electron there’s just one user at any time, the performance Requirement #4 is achieved as well.

Bundling and publishing

I suggest to use Electron Builder to bundle the Electron app. Just add some configuration to electron-builder.yml:

includeSubNodeModules: true

files:
  - build
  - from: '.next/standalone/demo/'
    to: '.next/standalone/demo/'

For convenience, you can add following scripts to package.json:

{
  "scripts": {
    "build": "yarn build:next && yarn build:electron",
    "build:next": "next build",
    "build:electron": "electron-builder --config electron-builder.yml",
    "start:next": "next dev",
    "start:electron": "electron ."
  }
}

For separation of concerns I recommend to keep Next.js sources in src of and Electron soures in and src-electron, this ensures Next.js does not try to compile Electron.

Conclusion

Requirement #3 is achieved in full glory since it’s just one file, and it only uses standard APIs.

I was amazed when it actually worked… I was quite skeptical that it would be this simple and yet so elegant.

Now I can enjoy full access to file & operating systems directly from Next.js Server Components or Route Handlers, with all the benefits of Next.js ecosystem, established patterns, and while using Electron to deliver the complete app experience to users, since the app can be bundled and published.

P.S. I have done my due diligence and I have not found any articles that cover usage of Next.js with mocked requests and responses, especially in conjunction with Electron. Shame on me if otherwise, I must have forgotten how to Google 🤓… But even if I missed something, this article should help to explain why this approach is good.

P.P.S. MSW is a bit overkill and is used for different purposes, like other HTTP mocking libraries.

P.P.P.S. Few shady things in the code are using buffers to read response and synchronous reading static files, both can be improved with streaming, but for simplicity it’s good enough.

React, JSX, ES module imports (dynamic too) in browser without Webpack

Kirill Konshin — Mon, 04 Nov 2019 22:37:46 +0000

This article is an attempt to put together everything available at the moment and see if it is really possible to implement production ready React-based website without pre-compilation like Webpack/Parcel or at least to have as less pre-compilation as possible.

TL;DR

It is possible to serve React project with minimal compilation of JSX without using bundlers like Webpack or Parcel.

Bundlers pack code into chunks, which has drawbacks: chunks have to be small enough to deliver only what's needed and big enough to make better use of HTTP1.1 connections. Finding the right balance may be tricky and involves some automatic and manual tuning. In addition it slows down the build and dev mode.

We will use Service Workers to do all the work in runtime and some Babel for production build.

Everything described in the article is highly experimental and I am cutting corners everywhere. By no means I recommend to use any of that in production :)

Current state

Ability to use ECMAScript modules (<script type="module"/> with imports like import Foo from './foo'; and import('./Foo')) directly in browser is quite well known at the moment and has good browser support: https://caniuse.com/#feat=es6-module.

But in reality we don't just import our own modules, we import libraries. There is a great article on this topic: https://salomvary.com/es6-modules-in-browsers.html. Also there's another worth to mention project https://github.com/stken2050/esm-bundlerless.

Among other important things from the articles, these ones will be crucial in order to make React app work:

Package specifier imports support (or import maps): when we import react in reality we should import something like https://cdn.com/react/react.production.js
UMD support: React is still distributed as UMD and so far there's still an ongoing discussion how to publish it using ES modules
JSX
Import CSS

Let's solve these issues one by one.

Project structure

First things first, let's assume the project will have the following structure:

node_modules obviously a place where we will install all dependencies
src dir with index*.html and service scripts
- app app source code

Package specifier imports support

In order to use React like so import React from 'react'; we need to tell the browser where to find the actual source. This is quite simple, there's a shim for that: https://github.com/guybedford/es-module-shims.

Let's install the shim and React:

$ npm i es-module-shims react react-dom --save

In order to launch the app we can do something like this in public/index-dev.html:

<!DOCTYPE html>
<html>
<body>

  <div id="root"></div>

  <script defer src="../node_modules/es-module-shims/dist/es-module-shims.js"></script>

  <script type="importmap-shim">
    {
      "imports": {
        "react": "../node_modules/react/umd/react.development.js",
        "react-dom": "../node_modules/react-dom/umd/react-dom.development.js"
      }
    }
  </script>

  <script type="module-shim">
    import './app/index.jsx';
  </script>

</body>
</html>

Where in src/app/index.jsx we will have:

import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';

(async () => {
  const {Button} = await import('./Button.jsx');
  const root = document.getElementById('root');
  ReactDOM.render((
    <div>
      <Button>Direct</Button>
    </div>
  ), root);
})();

And the src/app/Button.jsx:

import React from 'react';
export const Button = ({children}) => <button>{children}</button>;

Does it work? Of course, no. Even though we've successfully imported everything.

Let's move on to the next challenge.

UMD support

Dynamic way

The issue now is that React is distributed as UMD, it cannot be consumed by imports, even by the shimmed ones (if the ticket is resolved, just skip this step). So we need to somehow patch the distributable to convince browser that it's a legit ES modules.

The above mentioned article led me to an idea that we can use Service Workers to intercept and pre-process network requests. Let's create the main endpoint src/index.js, which will bootstrap the SW and App and use it instead of the App directly (src/app/index.jsx):

(async () => {

  try {
    const registration = await navigator.serviceWorker.register('sw.js');
    await navigator.serviceWorker.ready;

    const launch = async () => import("./app/index.jsx");

    // this launches the React app if the SW has been installed before or immediately after registration
    // https://developers.google.com/web/fundamentals/primers/service-workers/lifecycle#clientsclaim
    if (navigator.serviceWorker.controller) {
      await launch();
    } else {

navigator.serviceWorker.addEventListener('controllerchange', launch);
    }

  } catch (error) {
    console.error('Service worker registration failed', error);
  }
})();

And then let's create the Service Worker (src/sw.js):

//this is needed to activate the worker immediately without reload
//@see https://developers.google.com/web/fundamentals/primers/service-workers/lifecycle#clientsclaim
self.addEventListener('activate', event => event.waitUntil(clients.claim()));

const globalMap = {
    'react': 'React',
    'react-dom': 'ReactDOM'
};

const getGlobalByUrl = (url) => Object.keys(globalMap).reduce((res, key) => {
    if (res) return res;
    if (matchUrl(url, key)) return globalMap[key];
    return res;
}, null);

const matchUrl = (url, key) => url.includes(`/${key}/`);

self.addEventListener('fetch', (event) => {

  const {request: {url}} = event;

  console.log('Req', url);

  const fileName = url.split('/').pop();
  const ext = fileName.includes('.') ? url.split('.').pop() : '';

  if (!ext && !url.endsWith('/')) {
    url = url + '.jsx';
  }

  if (globalMap && Object.keys(globalMap).some(key => matchUrl(url, key))) {
    event.respondWith(
      fetch(url)
        .then(response => response.text())
        .then(body => new Response(`
          const head = document.getElementsByTagName('head')[0];
          const script = document.createElement('script');
          script.setAttribute('type', 'text/javascript');
          script.appendChild(document.createTextNode(
            ${JSON.stringify(body)}
          ));
          head.appendChild(script);
          export default window.${getGlobalByUrl(url)};
        `, {
          headers: new Headers({
            'Content-Type': 'application/javascript'
          })
        })
      )
    )
  } else if (url.endsWith('.js')) { // rewrite for import('./Panel') with no extension
    event.respondWith(
      fetch(url)
        .then(response => response.text())
        .then(body => new Response(
          body,
          {
            headers: new Headers({
              'Content-Type': 'application/javascript'
            })
        })
      )
    )
  }

});

Here's what we've done here:

We have created the export map, which associates package id with global var name
We have created a script tag in head with contents of UMD-packaged script
We've exported the mapped global var as default export of module

For the sake of tech demo this method of patching should be enough, but it may break with other UMD declaration. Something more robust can be used to process sources.

Now let's adjust the src/index-dev.html to use the bootstrap entry point:

<!DOCTYPE html>
<html>
<body>

  <div id="root"></div>

  <script defer src="../node_modules/es-module-shims/dist/es-module-shims.js"></script>

  <script type="importmap-shim">... same as before</script>

  <!-- change the file from app/index.jsx to index.js -->
  <script type="module-shim" src="index.js"></script>

</body>
</html>

Now we're able to import React and React DOM.

Static way

It's worth to mention, that there's also another way. We can install ESM distributable:

npm install esm-react --save

And then use following map:

{
  "imports": {
    "react": "../node_modules/esm-react/src/react.js",
    "react-dom": "../node_modules/esm-react/src/react-dom.js"
  }
}

But unfortunately this project is quite stale, latest is 16.8.3 whereas React is 16.10.2.

JSX

There are two ways to do the JSX compilation. We can either go traditional way and use Babel to pre-compile or we can use it in runtime. Of course for production it would make much more sense to pre-compile, development mode can be more brutal. Since we already use Service Worker let's enhance it.

Let's install a special Babel package that can do it:

$ npm install @babel/standalone --save-dev

Now let's add following to the Service Worker (src/sw.js):

# src/sw.js
// at the very top of the file
importScripts('../node_modules/@babel/standalone/babel.js');

// activation stuff as before

self.addEventListener('fetch', (event) => {

  // whatever we had before

  } else if (url.endsWith('.jsx')) {
    event.respondWith(
      fetch(url)
        .then(response => response.text())
        .then(body => new Response(
          //TODO Cache
          Babel.transform(body, {
            presets: [
              'react',
            ],
            plugins: [
              'syntax-dynamic-import'
            ],
              sourceMaps: true
            }).code,
            { 
              headers: new Headers({
                'Content-Type': 'application/javascript'
              })
            })
        )
    )
  }

});

Here we've used same approach to intercept the network request and respond with slightly different content, in this case we use Babel to transform the original response. Please note that plugin for dynamic import has a different name syntax-dynamic-import, not a usual @babel/plugin-syntax-dynamic-import due to Standalone usage.

CSS

In the above mentioned article author used text transformation, here we will go a bit further and inject the CSS in the page. For that we again will use the Service Worker (src/sw.js):

// same as before

self.addEventListener('fetch', (event) => {

  // whatever we had before + Babel stuff

  } else if (url.endsWith('.css')) {
    event.respondWith(
      fetch(url)
        .then(response => response.text())
        .then(body => new Response(
          //TODO We don't track instances, so 2x import will result in 2x <style> tags
          `
            const head = document.getElementsByTagName('head')[0];
            const style = document.createElement('style');
            style.setAttribute('type', 'text/css');
            style.appendChild(document.createTextNode(
              ${JSON.stringify(body)}
            ));
            head.appendChild(style);
            export default null;
          `,
          {
            headers: new Headers({
              'Content-Type': 'application/javascript'
            })
          })
        )
    );
  }

});

Et voila! If you now open the src/index-dev.html in the browser you'll see the buttons. Make sure the proper Service Worker is being picked up, if you're not sure, open Dev Tools, go to Application tab and Service Workers section, Unregister everything and reload the page.

More production friendly case

The above mentioned code works fine for dev mode, but realistically we don't want all of the app users to compile the code in their browsers, it's impractical. So let's push things a bit further and make the minimalistic production mode.

In order to do that we will create one more index file src/index.html with following content:

<!DOCTYPE html>
<html>
<body>

<div id="root"></div>

<script type="module" src="index.js"></script>

</body>
</html>

As you see, no shims here, we will use different technique to do rewrites. Since we still have to use Babel to compile JSX we can also rewrite module paths there instead of using importMap.json for the shim. Let's install everything:

$ npm install @babel/cli @babel/core @babel/preset-react @babel/plugin-syntax-dynamic-import babel-plugin-module-resolver --save-dev

Let's add the scripts section to package.json:

{
  "scripts": {
    "start": "npm run build -- --watch",
    "build": "babel src/app --out-dir build/app --source-maps --copy-files"
  }
}

Next let's add .babelrc.js:

module.exports = {
  presets: [
    '@babel/preset-react'
  ],
  plugins: [
    '@babel/plugin-syntax-dynamic-import',
    [
      'babel-plugin-module-resolver',
      {
        alias: {
          'react': './node_modules/react/umd/react.development.js',
          'react-dom': './node_modules/react-dom/umd/react-dom.development.js'
        },
        // we replace as follows to make sure we stay in build dir
        resolvePath: (sourcePath, currentFile, opts) => resolvePath(sourcePath, currentFile, opts).replace('../../', '../')
      }
    ]
  ]
}

Keep in mind that this file will be used only for production, for dev we configure Babel in the Service Worker.

Also let's add production mode support to Service Worker:

// src/index.js
if ('serviceWorker' in navigator) {
    (async () => {

        try {

            // adding this
            const production = !window.location.toString().includes('index-dev.html');

            const config = {
                globalMap: {
                    'react': 'React',
                    'react-dom': 'ReactDOM'
                },
                production
            };

            const registration = await navigator.serviceWorker.register('sw.js?' + JSON.stringify(config));

            await navigator.serviceWorker.ready;

            const launch = async () => {
                if (production) {
                    await import("./app/index.js");
                } else {
                    await import("./app/index.jsx");
                }
            };

            // this launches the React app if the SW has been installed before or immediately after registration
            // https://developers.google.com/web/fundamentals/primers/service-workers/lifecycle#clientsclaim
            if (navigator.serviceWorker.controller) {
                await launch();
            } else {

navigator.serviceWorker.addEventListener('controllerchange', launch);
            }

        } catch (error) {
            console.error('Service worker registration failed', error);
        }

    })();
} else {
    alert('Service Worker is not supported');
}

And use the condition in src/sw.js:

// src/sw.js
const {globalMap, production} = JSON.parse((decodeURIComponent(self.location.search) || '?{}').substr(1));

if (!production) importScripts('../node_modules/@babel/standalone/babel.js');

Also replace

// src/sw.js
   if (!ext && !url.endsWith('/')) {
     url = url + '.jsx' with
   }

with

// src/sw.js
   if (!ext && !url.endsWith('/')) {
     url = url + '.' + (production ? 'js' : 'jsx');
   }

Now let's add a build script build.sh which will copy everything needed to build dir:

# cleanup
rm -rf build

# create directories
mkdir -p build/scripts
mkdir -p build/node_modules

# copy used node modules
cp -r ./node_modules/react       ./build/node_modules/react
cp -r ./node_modules/react-dom   ./build/node_modules/react-dom

# copy files that are not built
cp ./src/*.js        ./build
cp ./src/index.html  ./build/index.html

# build
npm run build

We do this to make node_modules leaner by skipping build dependencies.

Here's the final state:

kirill-konshin / pure-react-with-dynamic-imports

Pure react with dynamic imports in the browser

Now if you open build/index.html you will see the same output as for src/index-dev.html but this time browser won't build Babel, it will use pre-built files.

As you see, the solution now has duplicates, like importMap.json, the alias section in .babelrc.js and list of files to be copied in build.sh. For demo purposes it's fine but for real usage it would be better to automate this.

Here's the published build: https://kirill-konshin.github.io/pure-react-with-dynamic-imports/index.html

Conclusion

HTTP2 should take care of those small source files sent over the network, hopefully.

Here's the repo where I have put everything together: https://github.com/kirill-konshin/pure-react-with-dynamic-imports