DEV Community: Ante Sepic

Can you please refresh (Or how we version our Single-Page Application)

Ante Sepic — Tue, 03 Dec 2019 20:32:01 +0000

In this article, I outline our approach to solving the problem of people not "getting" the latest version of our SPA.

A single-page application (SPA) is a web application or web site that interacts with the user by dynamically rewriting the current page rather than loading entire new pages from a server.

At work, I am responsible for the development of our client-facing SPA. We use Vue.js as our frontend framework of choice, but the problem I will describe in this article is framework agnostic.

Upon making changes and merging them to the master branch on GitHub, Travis (not a real person) runs our deployment process which includes building the app via Vue CLI and then uploading the new build to Amazon S3. So far so good, right?

The fundamental advantage of SPAs (people not having to load the whole HTML on route change) is also what was creating a problem. If people could technically never re-request the app fresh from your server, how do you deliver the latest version to them?

New version detection

The way we implemented a new version detection is pretty simple: We periodically do a fetch of the main index.html file and compare it to the old version. If there is a difference, it means that a new version got deployed. The good thing is that we don't have to do any versioning manually. index.html is guaranteed to be different on each build because Webpack generates a unique hash for each file during the build process, and hash is part of the file name embedded into the HTML. Since we only care whether the version is different or not (there is no concept of higher/lower version), this is enough.

Letting them know

We knew that we somehow want to let the people know that there was a newer version available. As for how we "deliver the message", there were three versions that came to our mind:

1.) Automatically refresh
This was discarded immediately because it could interrupt and/or confuse users. Imagine that you are filling out some form and a website refreshes for no reason, losing your progress.

2.) Intercept route change and reload from the server
Users would not be disturbed by this one since they are navigating to another page anyway. The only reason we did not go for this one is that it would break some flows where we rely on information being preserved in the Vuex store in between route navigations.

3.) Showing notification
In the end, we decided to go for showing an in-app notification, with a link that would trigger a refresh. That way our users can finish what they were doing and update the app when they are ready.

Implementation details

To periodically check for a new version, we decided to use AJAX polling since it will require no additional tooling, and other technologies like web sockets would be an overkill. The interval for the check is 10 minutes. A naive approach would be using a simple setInterval and firing a network request every 10 minutes. The drawback of that approach is that network requests are not free. Not in terms of bandwidth (HTML file is really small), but in terms of battery. You can read more about it here. The gist is: if a person is not using the network for some time on their device, the in-device modem will go into the low-power mode. Getting it back to the "normal" state takes some energy. If we just fire network requests every 10 minutes, we run the risk of draining our users' battery more than we need to.

Solution: Activity Based Timer

Here is the full code:

const ActivityBasedTimer = () => {
  let globalTimerId = 0;
  const timers = new Map();

  const maybeExecuteTimerCallback = ({ timerId, forced = false }) => {
    const timer = timers.get(timerId);

    if (timer === undefined) {
      return;
    }

    const {
      callback,
      interval,
      forcedInterval,
      forcedIntervalId,
      lastExecution,
    } = timer;
    const intervalToCheckFor = forced === true
      ? forcedInterval
      : interval;
    const now = Date.now();

    if (now - lastExecution < intervalToCheckFor) {
      return;
    }

    const newTimer = {
      ...timer,
      lastExecution: now,
    };

    if (forcedIntervalId !== undefined) {
      window.clearInterval(forcedIntervalId);
      newTimer.forcedIntervalId = window.setInterval(() => {
        maybeExecuteTimerCallback({ timerId, forced: true });
      }, forcedInterval);
    }

    timers.set(timerId, newTimer);
    callback({ forced, timerId });
  };

  const setInterval = ({ callback, interval, forcedInterval } = {}) => {
    const timerId = globalTimerId;

    if (typeof callback !== 'function' || typeof interval !== 'number') {
      return undefined;
    }

    const timer = {
      callback,
      interval,
      lastExecution: Date.now(),
    };

    if (forcedInterval !== undefined) {
      timer.forcedInterval = forcedInterval;
      timer.forcedIntervalId = window.setInterval(() => {
        maybeExecuteTimerCallback({ timerId, forced: true });
      }, forcedInterval);
    }

    timers.set(timerId, timer);
    globalTimerId += 1;
    return timerId;
  };

  const clearInterval = (timerId) => {
    const timer = timers.get(timerId);

    if (timer === undefined) {
      return;
    }

    const { forcedIntervalId } = timer;

    if (forcedIntervalId !== undefined) {
      window.clearInterval(forcedIntervalId);
    }

    timers.delete(timerId);
  };

  const runTimersCheck = () => {
    timers.forEach((_timer, timerId) => {
      maybeExecuteTimerCallback({ timerId });
    });
  };

  return {
    setInterval,
    clearInterval,
    runTimersCheck,
  };
};

export default ActivityBasedTimer;

The timer exposes an interface for running the code in an interval (just like setInterval does), but with no guarantee that the code will actually execute at that interval. It instead also exposes a function one should call to check for all timers and execute them as necessary. It basically loops through all the intervals, checks when they were last executed, and if more time than what is defined as an interval time has passed, it executes the callback. There is an additional, third parameter, in the setInterval implementation that takes a "forced" interval. This interval uses a native setInterval function so it more or less provides a guarantee of running every x milliseconds.

We then used this interval to check for updates periodically:

import ActivityBasedTimer from '@/services/activityBasedTimer';

const versioningTimer = new ActivityBasedTimer();

versioningTimer.setInterval({
  async callback() {
    const newVersionAvailable = await isNewerVersionAvailable();

    if (!newVersionAvailable) {
      return;
    }

    store.commit('setNewVersionAvailable', true);
  },
  // Normal interval is once every 10 minutes
  interval: 1000 * 60 * 10,
  // Forced interval is once per day
  forcedInterval: 1000 * 60 * 60 * 24,
});

Remember the function you need to call to check for the timers? We use that one in the router:

router.beforeEach((to, from, next) => {
  versioningTimer.runTimersCheck();
  next();
});

We bound it to the router route change because that's a sign of people actively using the app.

Conclusion

When we deploy a new version, after a few minutes, those people that did not close the tab in the meantime will get a notification telling them to click to update (which again is just a basic window.location.reload(true)).

How do you approach this problem in your SPAs? I would like to hear from you. Feedback on our solution is also more than welcome.

Finally, if you want a chance to solve cool challenges like these, Homeday is hiring. Come join us in Berlin!

Building my first desktop app: Electron initial impressions

Ante Sepic — Sat, 16 Nov 2019 08:20:55 +0000

TL;DR: I am building a desktop app with Electron. The app allows anyone to create widgets for their desktop with HTML, CSS and, optionally, JavaScript. Check it out here. In this article, I am sharing my first impressions, but also some struggles I had to overcome while working with Electron for the first time.

I have been using Ubuntu for most of my development career. What bothered me a lot was the lack of ways I could customize my desktop. On macOS, there exists Übersicht. On Windows, Rainmeter is very popular. Yet on Linux, I am stuck with pre-made widgets from various desktop managers that often look outdated. Even if you are lucky and use macOS/Windows, I found the experience with popular tools is not very user-friendly. I want to be able to write some HTML and CSS that I know how to use and create something beautiful on my desktop, without the steep learning curve.

Be the change you wish to see in the world

I decided to do something about this, and I think you'll like it. I am building Glitter, a tool for almost any OS that allows you to create (or use others') widgets with the technologies you already know - HTML, CSS, and JavaScript. Glitter widgets will be hosted and reviewed on the Glitter platform, making it easy and secure to distribute all kinds of widgets with the world.

Enter Electron

I decided to use Electron for building the app. It's popular to hate on Electron in some circles because of the size of the final application (it includes both Node.js and Chromium runtimes) and memory usage. I think it's an invaluable tool that even further expands the reach of web technologies. It also empowers people like me, who are not versed in "desktop" languages, to create something for the platform. Popular apps powered by Electron include Slack, Discord, VS Code, etc.

First Electron impressions

Getting started with Electron was fairly straightforward. I decided to make use of Electron Forge. They offer a CLI to kick-start the development and also provide handy tools for packaging your application. One other cool thing is the support for Webpack out of the box. If you are going to try it out, I suggest also joining the official Atom slack channel, which includes the #electron channel. The author of Electron Forge hangs out there (but also many other helpful people).

In Electron, the entry point for your application is the main process (basically a Node.js script). From there, you create as many renderer processes (browser window instances) as you need. This was immediately clear to me and it made sense. It also allows for a standard separation of concerns I am used to, where I have a backend (in this case the main process), and a frontend (renderer). I am using React for powering the UI, but Glitter widgets will come with a widget generator that supports Vue.js and React (more after the MVP).

Electron is being updated very often. The team is working hard and keeps introducing very handy features. For example, in version 7.x they introduced a much faster IPC method (basically a way for you to pass stuff between the main process and the browser window instances - it was already supported, but it had some issues).

It's not all roses.

Even though Electron is fairly well documented, I have run into a lot of things that simply don't work as expected.

For example, for my use case, I needed to spawn transparent, borderless windows for each widget. Doing so on Ubuntu by simply following the documentation and passing options to the BrowserWindow instance did not work at all. I instead had to set an arbitrary timeout of 500ms after the start of the application, and only then the option worked.
For some other options, setting them in the constructor never works, however calling the setter for that option after instantiating window, for some weird reason works

I am impressed with what's possible

I really enjoyed my time with Electron so far. I am super thankful for the team maintaining it and I wish only the best for the platform. It brought us many nice apps and will hopefully continue to do so in the future.

If you have any questions about development with Electron, feel free to drop a comment. I plan on writing about using TypeScript for Electron development, as well as how I decided to structure my code.

Also, don't forget to check out the app I am building. I welcome any feedback on that as well, of course.

Gaining insights into our GitHub repositories

Ante Sepic — Mon, 23 Sep 2019 15:15:11 +0000

On how we learned about ourselves and had fun in the process

Earlier this year, we have set an interesting key result (as part of our OKRs) - Average waiting time for two pull request reviews should be under 48 hours. The benefits of timely pull request reviews were clear to us:

not blocking your teammates (or yourself, if you depend on a feature inside the pull request)
checking the code while its context is still fresh

We felt that 48 hours was a good balance between taking time to properly review the code and moving swiftly.

What wasn't clear for us was how we will measure it. We also did not know our current "timing", but it felt like in some cases, it was taking longer than ideal.

Action plan

So we knew what we wanted, now was the time to figure out how to get it. After spending some time DuckDuckGo-ing, we were not able to find a tool that could help us. Luckily for us, we know how to code!

Thanks to the availability of a GraphQL GitHub API, fetching the necessary data was fairly straightforward. Afterward, it was just a matter of doing some calculations and we had our first stats. I built the tool in JavaScript (Node.js), and the first version had to be run via command line.

First version displayed the results directly in the terminal

This was very exciting, we could finally measure the times it takes us to review pull requests, among other things. But the output was not super friendly. I decided to give posting to Slack a chance. They make the process easy, you post a JSON to the webhook and it auto-posts to any channel for you. We now had the stats in Slack.

Displaying the results in a Slack channel

Slack message looked much better than the terminal output we had previously, but it was taking quite a bit of space. Once we ran the analysis for multiple repositories, it felt kinda spam-y.

We needed a better solution.

If only we knew some frontend people

Oh, wait.

Here at Homeday, every quarter we have something called Product Engineering Week (PEW for short). It's a time where we have no meetings and each member of the product team gets to work on projects that we would not tackle in normal sprints. I decided to work on a UI that would communicate with the backend script we already had and beautifully display the results.

I did decide on some rules (which later proved challenging):

For the backend, I wanted to use AWS Lambdas (I chose AWS since we already use AWS ecosystem for our whole infrastructure, it just made sense.)

Instead of this being an app only for Homeday (with perhaps a predefined list of repositories and a GitHub token stored somewhere on the backend,) I decided to integrate GitHub authentication so that anyone with an account could use the tool.

AWS Lambdas

I already had some previous adventures in the serverless world, but by no means would I call myself experienced with the technology. I chose Serverless because it makes it much easier to get started and develop your Lambdas (and serverless functions from many other providers), with a fairly quick feedback loop between making changes locally and them being reflected on Amazon servers.

It took some time to get it right, but the first version went quite smoothly. I migrated the previous Node.js code into a Lambda and through Serverless config file connected it to the API gateway so I could simply send some GET requests and get the results back.

Frontend

Now that the backend was ready, it was time to do what I wanted in the first place - provide a nice interface for interacting with the analysis tool. I had limited time so I chose Vuetify as a base for my interface. We have very recently open-sourced our own components library "Homeday Blocks", but it's still missing some layout components that I wanted in my interface. It was also a nice opportunity to get to know the highly praised Vuetify and learn from it for our library.

There are three parts to the flow:

Authenticating with a GitHub account (for this I used Auth0 which made it fairly painless)
Providing the list of repositories that can be analyzed and also getting the date range
Displaying the results

Here are some images showing what it looks like:

Welcome screen

Selecting repositories

Selecting the date range

Displaying the results

I added some sugar on top like adding buttons for a quick selection of most often used date ranges and Vue Kawaii to make the app a bit more friendly. Everything was fine and dandy in my universe, but then everything changed...

The Murphy awakens

Luckily, no fire nation attacked me. It was AWS! The issue happened when I tried to run the analysis tool on one of our more active repositories. It kept timing out!

Uh oh...

"I'll simply increase the Lambda timeout" - an innocent child

It made no difference, my Lambda was still timing out after exactly 29 seconds. Three and a half DuckDuckGo-s later, and I found out that it's a hard limit imposed by AWS for any Lambda triggered by the API gateway. Sh*t.

Return of the Jedi

It did not help that it was already Thursday and our PEW finishes on Friday. I had about 6 more work hours left before our "demo ceremony" and had to come up with a solution. Of course, I could have just picked some simpler repositories for the demo, and that was indeed my plan B. Luckily, I never had to use it.

I decided to go with what I dubbed "The two Lambdas approach (Temporary Name)" ®.

I implemented one Lambda that took a request, generated a unique identifier and replied with it. It also kicked off the second Lambda (and thus avoiding the hard limit) which did the calculation and stored it to an S3 instance as a JSON file (using the unique identifier we sent to the frontend as its name).

On the frontend, after receiving the response, the code polls every 5 seconds for the expected file on S3, and once it finds it, displays the results.

It's a hacky/dirty/whatever approach but it works, for the scale we expect, it should not crash AWS.

Summary

The app is live and kicking. You can try the hosted version here: Homeday GitHub Stats

You can also find the code (and contribute to it) at the following links:

Backend: https://github.com/homeday-de/homeday-github-stats-backend

Frontend: https://github.com/homeday-de/homeday-github-stats-frontend

All feedback is welcome (especially Lambda shaming).

Finally, if you want a chance to work on cool projects like these, Homeday is hiring. Come to Berlin and take part in our next PEW!