MoonZoon Dev News (1): CLI, Build pipeline, Live-reload, HTTPS

It's alive! It runs!

It... doesn't do anything useful yet. Just like an average kitten. We have to wait until they grow up - but who doesn't like to watch progress?

---

Welcome to the MoonZoon Dev News!

MoonZoon is a Rust full-stack framework. If you want to read about new MZ features, architecture and interesting problems & solutions - Dev News is the right place.

---

There are two big news. I've written my first tweet ever! And also a couple MoonZoon lines of code - a build pipeline, live-reload, certificate generator and servers (GitHub PR).

Awesome Discord friends tested it on Fedora, Ubuntu, Windows and macOS with Chrome, Firefox and Safari. Live-reload works also on my older iPhone SE. Thanks @adsick, @UberIntuiter and @EvenWei!

Follow these steps to try it by yourself.

---

How the build process works?

When you run in examples/counter the command

cargo run --manifest-path "../../crates/mzoon/Cargo.toml" start
# or in the future:
mzoon start

then:

1. MZoon (aka MoonZoon CLI) loads the project's MoonZoon.toml. It contains only configuration for file watchers atm:
   

   [watch]
   frontend = [
           "frontend/Cargo.toml",
           "frontend/src",
   ]
   backend = [
           "backend/Cargo.toml",
           "backend/src",
   ]
   

2. MZoon checks if wasm-pack exists and panics if doesn't.

   • Note: MZoon will automatically install the required wasm-pack version defined in MoonZoon.toml and check the compatible Rust version in the future.

3. MZoon generates a certificate for the localhost domain using rcgen. The result is two files - private.pem and public.pem - saved in the backend/private directory.

   • Note: Git ignores the private directory content.
   • Warning: I recommend to set the certificate's serial number explicitly to a unique value (you can use the current unix time). Otherwise Firefox may fail to load your app with the error code SEC_ERROR_REUSED_ISSUER_AND_SERIAL.

4. wasm-pack builds the frontend part. If you used the parameter -r / --release together with the command start, then it builds in the release mode and also optimizes the output (.wasm file) for size.

5. A unique frontend_build_id is generated and saved to the examples/pkg/build_id. The build id is added as a name suffix to some files in the pkg directory. It's a cache busting mechanism because pkg files are served by the backend.

   • Note: pkg is generated by wasm-pack and its content is ignored by Git.

6. The frontend file watcher is set according to the paths in MoonZoon.toml. It sends an empty POST request to Moon (https://127.0.0.1:8443/api/reload) on a file change.

   • Warning: Browsers treat unregistered self-signed certificates as invalid so we must allow the acceptance of such certificates before we fire the request:

     reqwest::blocking::ClientBuilder::new()
         .danger_accept_invalid_certs(true)
   

7. cargo run builds and starts the backend. MZoons sets the backend file watcher and saves a generated backend_build_id to backend/private/build_id.

   • Note: If you like async spaghetti, you won't be disappointed by looking at the related code. Why?
     • We can't easily split cargo run to standalone "build" and "run" parts. We ideally need something like cargo run --no-build.
     • We need to handle "Ctrl+C signal".
     • We need to somehow find out when the backend has been started or turned off (from the MZoon's point of view).
     • (Don't worry, I'll refactor it later and probably rewrite with an async runtime.)

---

How the backend works?

The backend part consists two Warp servers.

• The HTTPS one runs on the port 8443. It uses generated certificates.
• The HTTP one runs on the port 8080 and redirects to the HTTPS one.
  • Question: What's the best way of HTTP -> HTTPS redirection? I don't like the current code.

We need HTTPS server because otherwise browsers can't use HTTP/2. And we need HTTP/2 to eliminate SSE limitations. Also it's better to use HTTPS on the local machine to make the dev environment similar to the production one.

Both servers binds to 0.0.0.0 (instead of 127.0.0.1) to make servers accessible outside of your development machine. It means you can connect to your dev servers with your phone on the address like https://192.168.0.1:8443.

I assume some people will need to use custom dev domains, sub-domains, ports, etc. Let me know when it happens.

Tip: I recommend to test server performance through an IP address and not a domain (localhost) because DNS resolving could be slow.

I've chosen Warp because I wanted a simpler server with HTTP/2 support. Also I have a relatively good experience with hyper (Warp's HTTP implementation) from writing a proxy server for my client.

---

How live-reload works?

When you go to https://127.0.0.1:8443, the frontend route is selected by Warp in the Moon. Moon responds with a generated HTML + Javascript code.

HTML and JS for app initialization aren't very interesting so let's focus on the live-reload code (Note: I know, the code needs refactor, but it should be good enough for explanation):

<script type="text/javascript">
    {reconnecting_event_source}
    var uri = location.protocol + '//' + location.host + '/sse';
    var sse = new ReconnectingEventSource(uri);
    ...
    sse.addEventListener("reload", function(msg) {
        sse.close();
        location.reload();
    });
</script>

What's {reconnecting_event_source}? And why I see ReconnectingEventSource instead of EventSource?

Well, welcome to the world of browsers where nothing works as expected, specs are unreadable and jQuery and polyfills are still needed.

{reconnecting_event_source} is a placeholder for the ReconnectingEventSource code. The library description:

This is a small wrapper library around the JavaScript EventSource API to ensure it maintains a connection to the server. Normally, EventSource will reconnect on its own, however there are some cases where it may not. This library ensures a reconnect always happens.

I've already found such "edge-case" - just run the app in Firefox and restart backend. Firefox permanently closes the connection. Chrome (and I hope other browsers) try to reconnect as expected. Question: Do you know a better solution?

Let's move forward and look at this snippet:

sse.addEventListener("reload", function(msg) {
    sse.close();
    location.reload();
});

It means we listen for messages with the event name reload. Moon creates them in the POST /api/reload endpoint this way:

Event::default().event("reload").data(""))

Warning: Notice the empty string in data(""). It doesn't work without it.

We should call sse.close() if we don't want to see an ugly error message in some console logs when the browser kills the connection on reload.

The last part, hidden under the ... mark in the first code snippet, is:

var backendBuildId = null;
sse.addEventListener("backend_build_id", function(msg) {
    var newBackendBuildId = msg.data;
    if(backendBuildId === null) {
        backendBuildId = newBackendBuildId;
    } else if(backendBuildId !== newBackendBuildId) {
        sse.close();
        location.reload();
    }
});

The only purpose of this code is to reload the frontend when the backend has been changed. The backend sends the message backend_build_id automatically when the client connects to /sse endpoint - i.e. when the SSE connection has been opened.

---

And that's all for today!
Thank YOU for reading and I hope you look forward to the next episode.

Martin

P.S.
We are waiting for you on Discord.

Comments

[Valeria]

Seems like a nice idea. I have just two questions.

• Did you run performance tests against nodejs? If so could you share the results?
• Is it possible to turn https off and use only http? Generally I ran deployments behind a load balancer that handles redirects and certificates updates

[Martin Kavík]

Hi Valeria and thanks for the questions!

• "Did you run performance tests against nodejs?"

The project doesn't use Node.js.

But I see you like Node.js so maybe it's just a "typo" - do you mean Moon or Warp?
Moon hasn't been finished yet. Warp should be fast enough and there is a chance I'll replace it with another library later. So performance tests don't make sense in this phase of development.

Or is the question related to the tip below?

Tip: I recommend to test server performance through an IP address.."

• "Is it possible to turn https off and use only http?"

I can imagine we can add to MoonZoon.toml something like:

[server]
https = false

What do you think?

[Valeria]

No I didn't imply that server should use nodejs , I was wondering if you ran performance tests and compared them with the performance test of a similar implementation in nodejs (or any other language) for that matter.

I think http should be the default option to avoid localhost certificate problem for instance, otherwise yeah something like that.

[Martin Kavík]

There is only a couple of stable frameworks based on virtual actors and they are written in C#, Java or Go. I can't find any usable Node.js virtual actor frameworks and there are only a few actor frameworks - probably because Node.js wasn't designed for multi-threading or actors (like Elixir) from start. However I've escaped from the Node.js world a couple of years ago so maybe I overlook something.

So I plan to somehow compare MoonZoon with other frameworks once it's mature enough but it will be hard to write some reasonable performance tests.

[Martin Kavík]

@valeriavg An update for you:

• It's now possible to switch between HTTPS and HTTP. HTTP is the default option as you suggested.
• I plan to optimize frontend and then publish benchmark results. I'll try to do the same for backend, once ready.
• You can find live demo and other news and links in the Dev News 2.