DEV Community: Modeinspect

From a Day to 17 Minutes: How We’ve Dealt with Slow Build Times

Peter Bokor — Fri, 08 Mar 2024 15:26:30 +0000

The problem of a long deployment process isn't just the waiting; it's how this waiting seeps into every part of the development process, fundamentally changing how teams work. Test builds are less frequent and bigger in scope. This leads to slower turnaround times on feedback and testing. Release cycles are long and there is less space for experimentation and prototyping.

We experienced this in full fashion with 8 hour deployment times with a decent amount of manual work. Here's how we finally took action to significantly reduce the time it took.

Common Pains of Shipping an Electron App

acreom first started as a web app and deploying it was simple. Only after we have decided to support other platforms by creating an Electron version, we saw that things started to move much slower.

Distributing desktop versions of our app became a tough and tedious task. We would spend an entire day just getting the versions ready for release. It was not only time-consuming, but also highlighted several issues we faced, such as managing different platform requirements, and ensuring consistent performance across all of them.

Building the Frontend for Different Operating Systems

For the frontend part of our app, we use Vue+Nuxt. However, when we work with Electron, we don't use Nuxt's feature for server-side rendering. Instead, we build the frontend as a single page app, which means it's made up of static files like HTML, CSS, and JavaScript. It's important to note that we create a separate build for each operating system because there are some differences in how the app works on each one, such as the way users log in, deep links, and file management.

We are also building a tiny executable called acreom-assistant that is running a local language model for parsing dates. This executable is packaged with electron. You can learn more about how we ship it.

App Signing and Distribution Across Different Platforms

To make sure our app runs on Windows without triggering the SmartScreen warning, it has to be digitally signed. This process involves getting a certificate, which comes on a physical USB token (a USB drive with a private key on it). These tokens are bought from certificate authorities and cost around $300 a year. Because of this, the app's build process needs to happen on a computer that has this USB token plugged in, which we ended up doing manually on one of our developer's Windows computers.

For macOS, to avoid users having to manually allow the app in the Privacy & Security settings, the app must also be signed and then notarized by Apple. Notarization requires an Apple Developer account, which costs $100 annually. Fortunately, we had an old MacBook Pro that we dedicated to the tasks of building, signing, and notarizing the app.

When Apple released computers with the new M1 chips, we had to adapt again. We added another MacBook to our setup because we use ZeroMQ for communication between the Electron app and the acreom-assistant, and ZeroMQ has some native bindings that required adjustments for the M1 architecture.

Linux was a bit different; it was the only platform for which we set up the build and distribution process to run automatically through GitHub actions from the beginning. This approach for Linux was much simpler compared to the manual processes required for Windows and macOS, making it our "golden" platform in terms of ease of deployment.

Sharing Builds: Adapting to M1 Challenges on GitHub

The last step for each platform's build process is to upload the app to our GitHub releases repository. We automated this step from the beginning, but when we started creating builds for the new M1 chip Macs, we had to add a manual step. This involved merging files needed for the auto-updater to work with the M1 builds.

Making the Dev Team Happy with Faster Deployments

Originally, it took us a whole day to get the builds ready for users. We had a dedicated machine to run the non-automated build, then we had to upload the builds to a shared folder, and send each other messages that the build was done. Repeating this many steps made the whole process painfully slow. That’s when we started with automation and moving the pipeline to the cloud, which didn't work out that well at first.

Automated Internal Builds for Testing

We ship our products across multiple platforms. This includes web, desktop (like Windows, MacOS, Linux), and mobile devices (iOS and Android). We use the same steps for each platform, using continuous deployment to share our work with our team and the users.

Every time we add a new branch, GitHub Actions automatically makes an inside version of our app and puts it in a Google Cloud Platform bucket, which the team has access to and can try them out. Each feature has a specific Discord Forum, where we discuss and explore ideas.

Quick Beta Builds on Web

Our approach for web apps mirrors our desktop apps. We deploy each branch to a unique URL (e.g., [branchname].preview.acreom.com) using Netlify. We share some early versions of our projects hosted on Netlify with our Discord community. To protect user data, these test builds connect to a staging backend and database, instead of the production environment. While they aren't fully ready for release, they are prepared for users to explore, envision how they could use them, and share their feedback on Discord.

Impact on the Way We’re Building

Having this in place helped us tremendously iterating on our product. To show a real example after all the improvements: We knew there was something wrong with our sidebar so we've made an effort to improve doing a series of explorations with clear goals. We did 4-6 builds within 2 weeks, iterating on various parts. We explored and discussed what worked well and what didn’t. During that time, we have learnt everything - tradeoffs and consequences of decisions, so that we could confidently ship a good thing. If deployment time would take us day a week, our team couldn’t be focusing on solving user problems. We’ve talked about some of our learnings in common mistakes of building a productivity tool. But it also resulted in some unexpected benefits.

More Demos Over Discussions

Faster deploy times helped us rediscover an old mantra - show, don’t tell. Rapid prototyping and iterating on the end result is the best way to explore ideas. We don't have to spend hours talking about ideas in meetings without anything concrete to look at. The prototypes we do are often very basic and just good enough to communicate an idea, even if some parts are broken. They also encourage us to take action. It's hard not to try something when it doesn't cost much, while learning a lot from it. If we can build something in a day, we don't spend too much time talking about it. Combined with our team local-first approach, it results in much faster shipping times.

More Options to Choose From

Having multiple options and ideas makes it easier to pick the best one. Rapid prototypes leave you with a catalog of possible solutions clearly demonstrating its pros and cons. After that you have a rough understanding of costs and tiniest details of the implementation.

Targeted User Feedback

Shipping things quickly means you can share more with your users. Keeping in touch with your users is key to showing how you are solving their problems. Not everyone will test your prototype, but somebody will be interested to try out and help you to shape it. Talk to people, show them how you're solving their problems, and you might end up with a group of fans willing to help you.

by Adam Pavlisin & Slavo Glinsky
➤➤➤ https://acreom.com

The Offline-First Sync

Peter Bokor — Fri, 05 Jan 2024 14:53:50 +0000

Transferring data between devices is a common feature these days. Most apps you use everyday implement some kind of mechanism for syncing information between the app and the server it's connected to, enabling instant access, no matter where you connect from.

Syncing gets difficult the moment you make your app offline first. Your data, including all changes made on a different device, must be accessible offline.

One of our core values is to make your knowledge base available no matter the system you use. We've specifically tailored acreom to work primarily offline, with sync being just an extension for our offline-first system. No matter where you access from, whether using a web browser, mobile, or desktop apps, your data is primarily stored on your device and only then synced to your other devices. This ensures that you always own your data.

Offline-first sync comes with its own specific problems that pile above all the issues of other sync solutions. Engineers at Linear had an insightful talk about the problems and workings of a similar sync solution and so did Figma. We have decided to base our sync engine on a similar concept, with some interesting modifications.

Offline first

Before we dive into how the sync extension works, let’s start with covering the offline workings of acreom.

In most cases, offline-first apps store data either in a local database or directly on a disk. Internally, acreom uses indexedDB. Each entity modification creates a change object that is saved in the Changes Queue. A change describes what exactly happened to an entity. It contains the ID of an entity, the table it belongs to, the type of the change, and all the modifications. To make the sync work, three types of changes are used - create, update and delete.

{    
    key: <uuid>, 
    type: 2, // 1 - create, 2 - update, 3 - delete
    table: “documents”,
    modifications: {
         …
    }
}

Change objects in the queue are thrown away the moment they are processed. The change queue allows multiple processors to consume changes. In our case, there are two: a Cloud processor that sends the change to the server, and a local file system processor that writes the data to the filesystem. Additionally, search indices are updated and any 3rd party apps are notified of a change if necessary. All of these are running as workers on separate threads.

Note that this process is invisible to the user. The frontend layer will update the data optimistically and instantly for a seamless experience.

Sync

The sync protocol is initialized when you log in. It establishes connection to our sync server, fetches all already synced data, and registers websocket listeners.

Syncing between devices

Websocket connection enables real time sync of data between different devices. acreom protocol communicates with the sync server in two ways:

The protocol takes changes on entities marked as syncable and sends them to the sync server using REST. It also fetches incremental changes on each app startup, ensuring the data present in the app is always up to date.
The second part of the communication is based on websockets. In case you have multiple devices online at the same time, changes are sent to devices using websockets, resulting in the devices getting the updates in real time.

All changes that are synced are grouped by the entity. If you edit a single entity multiple times in a row, these changes are grouped into a single change object.

Coming online

The same approach is used when you do some work offline, and then (when online), try to sync. Each change gets saved separately and is resolved into a single change object right before being synced. Data is therefore synced almost instantly, no matter the changes you've made while being offline.

The backend

The server side of the sync is quite simple: the sync server receives requests from acreom and compares the entity received to the one stored in the database. If the entity received is a newer version of the entity stored, it is saved and propagated to all online devices using websockets. If it’s not, the change is discarded. All devices that are not online will fetch the latest state from DB when they come online.

Conflict resolution

The current conflict resolution of the sync is "latest version wins". We plan on bringing CRDTs and transactions to replace the existing implementation alongside the collaboration later this year.

Handling network states

The biggest challenge we've faced was not the synchronization itself, but the management of the synchronization state. Disconnecting the protocol (e.g. when acreom goes offline, when there are connection issues on our backend, or when you lock your screen), turned out to be a bigger challenge than we thought it would be.

acreom is an electron app, so we rely on solutions available for the web. Electron window has a listener for online and offline events, but they are not reliable all the time. When you go offline but stay connected to the wifi, the listeners won’t fire and the protocol continues sending requests. This results in requests not going through and protocol getting stuck. Same goes for putting your pc to sleep. Proper handling of online/offline state became a priority.

While working on the mobile apps, we've noticed that iOS had a great offline detection. We've decided to look into it and use the same solution. The solution boils down to pinging different DNS servers in a few second intervals. The pinging is done after a request fails because of network error, or an offline event is received. After the first successful ping, the protocol tries to reconnect, but if it fails, the process is repeated from the first step Upon success, acreom protocol fetches all data that changed since last sync from the sync server, applies local changes on top of the newly fetched data, and syncs local changes to the sync server.

Takeaways

While building the sync protocol, we've obtained a few key insights, which helped us iterate to the right solution.

Managing sync state is key. Knowing when to sync and when not to sync will make your life much easier. Being able to rely on knowing when you are online and offline takes a big chunk out of the sync protocol complexity.
When syncing data in a "last change wins" kind of fashion, some scenarios are not worth solving. You won’t solve the problems that come with it completely and the minor gains are worth it only on paper, users won’t notice much of a difference.
Merging changes in text without CRDTs or transactions and without knowing the chronological order is not worth it. If you were to keep all the changes, just go with CRDT like Y.js and avoid implementing solutions from scratch.
It’s better to not sync one update than not sync anything at all. If you have issues on the backend with saving a certain entity, it's better to skip a single update and let the user know about it, than to stop the whole sync. Yes, you can fix the sync server, but the user can lose their unsynced data in the meantime, for example from changes that happened on other devices.

➤➤➤ https://acreom.com

Lessons From Implementing Themes: Design & Engineering Perspective

Peter Bokor — Thu, 04 Jan 2024 08:34:55 +0000

Introduction

One of the most extensive and challenging projects we've tackled involved a significant overhaul of our CSS. We're talking about refactoring and deduplicating the CSS of over 400 components.The end goal was to introduce a light theme into an existing codebase built with Vue.js and Nuxt.js. This journey was full of learnings, and we are excited to share them with you from the perspectives of both a designer and a developer.

Starting Small - Proof of Concept

Designer: Unlike most of the apps, acreom started dark. We knew we would love to introduce a light theme one day, but for a long time, our priorities were elsewhere.

Before we’ve actually started editing the codebase, we’ve built a light theme skeleton of one the screens. It served both as a proof of concept, and also as a way to explore the possibilities of our current color palette. It was our main aid when defining the right color hierarchy.

From the single proof of concept screen, we transitioned to higher fidelity details, while constantly tweaking the colors. After having a base structure nailed down, we’ve jumped into the first component: a dropdown. What led us to more detail. What makes dropdown work in light and what in the light theme?

Dropdown is often the highest level of hierarchy, so it needs to stand out on various backgrounds. To make that

Dropdown on background without appropriate shadow.
on dark background you need to add an make component stand out from the background, inner shadow with lighter color will do that.

It took multiple iterations to match the visual / color hierarchy, define a visual language we want to use, and explore what works and what does not.

Developer: Once we had our first prototype in design, we began by implementing a single component - dropdown. Some questions we wanted to answer with this experiment were:

Do we use SCSS variables or CSS variables?
Do we define variables in component files or in one global config file?
How do we “toggle” the actual theme?

Our codebase is written in Vue and 99% of our components are written as single file components - html template, typescript logic, and styling in single file.

First prototypes defined the variables as SCSS variables saved in the component file.

Component:

App root:

While this approach works it has several issues. First, saving variables for each component is not scalable in the long term. Second, there are patterns that repeat across the app so we don’t need to define the same colors each time. Third, the logic for applying theme color based on class in each component adds a lot of unnecessary code, plus adding more than one theme requires you to rewrite this everywhere.

After several iterations and experiments we have ended up with the following implementation:

Variables:

Theme files:

Component:

We ended up with a mix of SCSS and CSS variables. We could have gone with CSS variables only, but parts of our codebase, where themes are not relevant, still use SCSS vars. We have created a theme file for each theme. The theme file has all the variables from the app, similar to VS code or Sublime text theming. Adding a new theme is just a matter of duplicating existing theme file and updating color values. These variables are inserted to the :root css scope by changing the class (theme-dark / theme-light) on the root component.

Once we did this with a few more components, we were comfortable to go ahead with this approach.

Establishing Naming Conventions for CSS Variables

Developer: As rumour has it, the world of computer science has two hardest problems: cache invalidation and naming variables.

For our theming task, we have established a systematic approach for naming CSS variables, which proved to be crucial.

Our formula was straightforward, yet effective:

--{component}-{type}__{state}

For instance:

This naming evolved organically as we refactored the application, ensuring consistency and ease of understanding across our CSS. It made our theming process a whole lot smoother.

Designer: Continuously, we’ve updated each of the core components listed below separately:

Colors, Buttons, Dropdowns, Select Menus, Dialog Windows, Tooltip, In app Notification, Checkbox, Switch, Segmented Picker, Editor (Bubble menu, inline components, etc...)

We’ve used various colors while tweaking the components. The current color scheme stopped working, when the underlying color of the app changed from a dark one to a light one, so we had to introduce new accent colors. Great resource to inspect is apple mac os figma template by Joey Banks, where you can inspect the difference, so you get an idea how to start moving. Video of acreola was also a good resource to grasp that challenge.

We’ve also added opacity to colors. It is a nice trick to provide a hierarchy later on. Instead of using #fff or #000, we’ve used color from our color palette to respect a hue. Blue Grey 500, has a variation with alpha channel 16% and 32%. Bringing an opacity color to our theme allows our developers to move faster constantly without giving it an extra thought. Plus it looks cool.

Prioritizing Function Over Form in Early Stages

Developer: We learned that it's more efficient to start with what looks "okay" - so we’ve kept our designer away. Then refined the semi-broken, developer-made design iteratively with our designer's input focusing on important details. This approach helped us focus on the right parts at the right time and speed up the whole process, allowing for rapid development and continuous improvement.

For those looking to deepen their understanding of theming, especially dark mode, I highly recommend the article “Illuminating Dark Mode” from the Figma blog. It’s a treasure trove of insights.

Designer: After the first iteration of the transition was done and we had a rough app in the light theme, we’ve gone deeper into specific details. We’ve used a lot of sticky notes in Figma. Some screens needed to be re-drawn completely. We did visual QA iterations in multiple passes, tweaking colors and seeing what works. Some specific components were useless to match to our defined patterns, so they needed to be customized.

Learnings

Developer

Start with a single component. Experiment and try to find a system that is scalable and maintainable.
Keep all the CSS variables in a single theme file. This way you can easily add new themes in the future.
Naming convention matters. Find name convention for CSS variables. This will make life easy for your teammates.

Designer

Rapid fixes in passes help a lot. Depends on the structure of your team, but overall I think the fluid process worked best. It provided a vital pacing of the project, and didn’t overwhelm and trap us in overthinking. It’s overwhelming, it’s hard, but it works.
Use Color overlays. Before the redesign we used certain shades of color for hover states (bluegrey900 on bluegrey950 background). Bringing an opacity color to our theme allowed us to move faster constantly and look good.

Let us know if you have any questions or would love to know more.

➤➤➤ https://acreom.com

Common Mistakes of Building a Productivity Tool

Peter Bokor — Thu, 14 Dec 2023 16:53:52 +0000

Great apps are opinionated, and laser focused on solving a single problem in the best way possible. For a long time, we were trying to combine 3 functional components together: markdown pages, tasks, and calendar. We have been inadvertently trying to do it all. This caused confusion to our users, led to an unintuitive hierarchy of entities, and made development cycles longer for us.

This blog is about the mistakes, decisions, and the eventual course correction we have made, while building a slightly different kind of devtool - a knowledge base for developers.

Trying to Do It All

Our mission is to help developers organize their work - from the moment they are assigned a new issue, all the way to the point it's merged into the main branch. Our goal is to guide them throughout this process and make it easy when it comes to organizing their work.

It was clear to us from the beginning that we had to establish focus. We knew who we were building for - developers, but we haven’t defined which problems we are solving, and which ones we are not. We have found ourselves on the verge of becoming one of the tools that try to do it all, but fail to solve one problem properly.

Figuring Out What to Do
What helped us was being in touch with our users, talking to them on a daily basis, and relentlessly chasing critical feedback from them.

The primary issues were the lack of a main entity in the app and the presence of too many options without a clear preferred choice. We have also learned that most of the users were using acreom for a frequent job - breaking down large features, or issues they work on, into actionable steps, using pages. While our focus was scattered by solving every other job, we forgot to support the main job of our users.

We have decided to embrace this user job, and cut back everything else to place a clear opinion and focus in the app.

Making Hard Decisions

In order to shape the current product version into what we decided to solve, we had to cut features that users frequently use which is a painful process. We wanted to be fast and do what's necessary in the long-run, even if it meant sacrificing the happiness of some users in the process.

Laser Focus

Chances are, there already exists a solution for whatever you are building, with a single focus, that deals with a single job, and does it great. If you try to solve the problem, plus something else, you just can’t compete - your resources and focus are split into solving 2 problems.

In our case, the problem we were solving was the calendar, which was only a small part of our app. We have decided to cut it, because we could never meet the user's expectations when it comes to being a full-fledged calendar - we just didn’t have the right platform to do it.

Single Top-level Entity

Laser focus means having a clear hierarchy of entities and explicit opinions on how to use the product from day 1. One of the major mistakes we have made early was designing around same-level entities. In our case these were tasks and pages, representing action items.

Having to do so often slowed down our development cycles, because we had to support edge cases and confused ourselves about what is truly important. One had to go, and, based on our learnings from the feedback mentioned before, it was clear what we had to double down on pages.

This change made everything clear for us and the user. Suddenly things clicked and there was a clear opinion on how to get things done in the app.

Validate

Getting feedback on early concepts from users throughout this process is critical for a successful product pivot. What we did right here was to iterate fast on our explorations and share these iterations with our community. We were waking up to long threads of feedback which was essential for us to course correct and navigate these changes.

An important takeaway here is that a major change in the product UX is usually going to cause a pushback. Take this pushback as an opportunity to explore the job that profoundly matters to users, by diving deep in the followup questions, and compensate for it in a new design if it belongs to your newly defined direction. Also, keep in mind that the pushback is always part of changing the status quo.

Learnings and Closing Thoughts

Making everyone happy is hard, especially if you are making a simple app. There will always be a pushback from the users that are used to a certain way of how the app works. It is your responsibility as the maker to incorporate the feedback in a way that will make the tool you are building better.

Cutting features users are used to is a pain, but is sometimes necessary for further growth. The most important part of this process is to listen to the feedback, and decide what really is important. In our case it has led us to a better app, and some great solutions we wouldn’t have made otherwise - such as the Timeline of your day.

Above all, having a concise goal, purpose, and a main use case should be the main priority. From the moment they are assigned a new issue, all the way to the point it's merged into the main branch.Diverging from those leaves your efforts scattered, and the tool you are making, full of friction.

Now that we have regained our course, we are not diverging from it. We are building acreom around the single entity - pages - locally stored in your markdown files. And our every decision will be made in a way to keep the app laser-focused and concise.

TL;DR

Learnings on building a great product:

Define a clear and narrow focus where you can deliver a 10x product.
Great products are simple. Focus on multiple same level entities distract you from simple hierarchy and sidetrack you from your mission.
Talk to users, a lot. By speaking to people we’ve found some things were not as important as we thought, and helped us focus on the core value.
Don’t be afraid to make difficult decisions.

Capturing at Speed of Thought

Peter Bokor — Thu, 07 Sep 2023 14:24:39 +0000

Most of the workflow management interfaces we use today prioritize organizing before capturing. Working with them often is a burden, and we lose focus simply by having to make sure we put the right information in the right place.

We felt this friction ourselves, hence one of the core philosophies for building acreom is capture-first, organize-later.

Capturing for developers

Oftentimes you come across a bug or an optimisation you need to do later, or simply want to note something down quickly, but as soon as you leave your IDE, your focus breaks and you lose your flow state, which takes time to regain.

Tools that help us capture issues with context do not provide an easy way to keep us in the flow state. Here’s a brief background on how we thought about designing such a feature from scratch.

Welcome Quick Capture

Quick capture is our way to capture your thoughts, not your attention. We address the problems with capturing in the following ways:

Switching context => floating window on top of your current app
Losing focus => everything captured has a clear destination
Losing flow => after writing down your thoughts, you perform a single action - capture
Other distractions => no unnecessary UI. You get an editor and a capture button

The Problem of Version 1.0

The main flaws of the first version was the lack of features. It could capture text to My Day, but that was it. The iteration only included a simple markdown editor. We did have tools and shortcuts to “postprocess” the captured text later in the main app - like convert to task / convert to page, but that required more user interaction than we would have wanted.

The Quick Capture V1 turned out to be too limited in functionality and having to almost always perform a follow-up action with your captured content was annoying. Our own experience and the community feedback made it clear that the functionality needed to be extended in a way that kept Quick Capture simple to use.

The Road to 2.0

We knew from the feedback that the editor in Quick Capture resembled a Page, which suggested that you could: 1. capture a Page, and 2. use plugins - such as Tasks, when capturing. That was exactly what we set out to add in V2.

Early Redesign

The image shows the drafts of an early V2. The capture would be split into 3 “screens”:

Entity picker with a markdown editor - capturing to My Day
Task capture with a simple input and a switch to enable capturing multiple tasks (the Quick Capture would automatically re-open after capturing)
Page capture with a title input + markdown editor

The decision of what you wanted to capture would be made before capturing, but you could always switch between different capture types.

The main problem with this version was the changing content of the whole window. Every capture entity had a completely different screen, which was confusing and overwhelming, especially when switching the different capture types. Furthermore, it put the burden on you to decide before capturing. So we decided to explore a different direction.

Unifying the Content

We have decided to unify the different capture types in order to mitigate the confusion. We have made the header - now with entity type selector (shown above), and the footer persistent across all entities, with only the “body” changing (shown below).

The feeling started to be better, but what surfaced was a problem with Tasks - they just weren’t fitting in with the Page and My Day.

The Problem with Tasks

Our next question was: Why does the Task Capture feel weird?

We came up with a few answers:

The Task capture does not feel and look the same as Task modal in-app, creating confusion
It was missing the functionality to add date / labels
There was no way to include Tasks in text, removing one of the essentials of acreom - capturing tasks with context

The solutions were not great:

We could add page, date, and labels pickers to Task capture, basically making it Not-very-quick Capture and overwhelming to use
There were 3 solutions: add full-fledged suggestions similar to task modal - bad feeling, couldn’t fit the window; make custom suggestions - feels weird and disconnected from app; don’t show suggestion, but show “pills” with parsed data - no user control, could lead to frustration (shown in picture below)
There would need to be a page selector, again making it slow and overwhelming.

One more solution to the tasks is shown below - the picture shows an exploration combining 2 actions - capturing a task, or capturing as the selected entity, covered under a tooltip. We found it only usable for power users, and that it may result in different data input than expected. For tasks you would also be missing feedback - what date and what label has been parsed. It could be done, but we wanted to go with a unified logic, where you capture only plain text.

In the end, the solution addressing all the problems mentioned above was simple - remove the Task capture altogether, and move Tasks inline. The Tasks would then act exactly as the Tasks in the app - you add a Task, get suggestions, accept whichever one you want. You could, naturally, capture tasks with their context, and, the cherry on top - we have removed one decision you have to make before capturing, making it more accessible and flowy. It all just made sense.

Quick Capture 2.0

With the Tasks out of the way, the functionality was mostly finished. What remained was making Quick Capture as non-distracting and easy to use, as possible.

We unified the capture to My Day and Page capture completely - moving the entity picker to the footer and making the action you perform as clear as possible - “Capture”. The secondary action - cancel is a necessity, as ‘escape’ shortcut to close the window is not known to everyone. We have also added a “add inline task button” to make the tasks more accessible.

We ended up with the bare minimum - a single editor with inline tasks, exactly what you are used to and would expect from using acreom.

In the final version we have included:

Capturing to My day
Creating a new Page
A markdown editor with tasks
A single action to capture

We have not included:

A separate task capturing
Complicated controls
Multiple screens and decision points increasing friction and reducing accessibility

Quick Capture Under the Hood

acreom is a single window electron app. In order to implement Quick Capture, we needed to make it a separate electron window. The window should be displayed above everything, in a place where you would expect it. It should also have a lightweight feel and take as little screen space as possible. That has brought a couple of technical challenges.

Display

Displaying an electron window above all other apps and hiding OS controls is straightforward. Simple config does the trick:

new BrowserWindow({
   title: 'Quick Capture',
   show: false,
   resizable: false,
   movable: true,
   minimizable: false,
   maximizable: false,
   closable: false,
   fullscreenable: false,
   autoHideMenuBar: true,
   frame: false,
   transparent: true,
   opacity: 1,
   alwaysOnTop: true,
   titleBarStyle: 'hiddenInset',
});

Displaying Quick Capture Where You Expect It

One thing to make the Quick Capture experience seamless is displaying it in the most convenient place: the screen you are looking at, and the position where you would expect it - exactly where you left it the last time.

In the early versions, quick capture was always shown in the center of your main screen. This turned out to be inconvenient - you had to switch context, and was confused when not seeing it on your secondary display. Lucky for us, electron has a way to check which screen is focused - more specifically, which screen the cursor is on, by calling the getDisplayNearestPoint method with the current pointer position (obtained by getCursorScreenPoint).

Not having to switch screens when capturing was a great improvement. A problem has arisen though - not all screens are made equal (in their resolution), and if you dragged the QC on a larger screen, dropped it out of bounds of a smaller screen, and tried to display it, it would not show - the image illustrates this perhaps better than words.

To solve the problem, we needed to recalculate the quick capture coordinates relative to the new screen before showing it. We used the following approach:

Get the screen with pointer - as described above
Get the original screen the window was shown on - using getDisplayNearestPoint to the window
Normalize the window position according to the old screen space - calculating for the x coords we get normX = (qc.x - window.bounds.x) / window.width;, where bounds is a rectangle containing the bounds of the display.
Multiply the normalized position with the new window dimensions to get the final adjusted position - again calculating for the x coords - x = newWindow.x + normX * newWindow.width;
Display quick capture

Now the Quick Capture is showing exactly where you need it and also expect it.

Sizing / Resizing

When the content in editor overflows the window, an event with the new content height is emitted to electron. Electron then resizes the Quick Capture window accordingly.

// on Frontend
resize() {
   window.electron.resize(
      editor.clientHeight ?? 235,
   );
},

// in Electron
ipcMain.handle('resize', (_event, height) => {
   if (quickCapture.isDestroyed()) return;
   quickCapture.setMinimumSize(
      quickCapture.getSize()[0],
      height,
  );
  quickCapture.setSize(quickCapture.getSize()[0],height);
});

Platform Specific Challenges

Different platforms have technical limitations with regards to windows and app focus management. We’ve had to make some workarounds for specific platforms to deal with them.

Windows Tray

We do not expect users to have acreom always open. However there are some features that users expect to work even when acreom is closed, like notifications, or invoking the Quick Capture window.

We have enabled tray on Windows to address this issue. Having the app present in the tray will keep the global shortcut bound and the user can invoke Quick Capture. Adding tray functionality to electron apps is easy. Read more.

Note: some users do not appreciate apps inserting themselves into the tray, so we have added an option to disable having acreom in the tray.

macOS Focus Handling and Fullscreen Apps

While working on Quick Capture - a floating window, which could show above the fullscreened app, we have encountered an interesting issue. Upon closing the Quick Capture, the acreom main app was always being focused, instead of the correct app - the one underneath the Quick Capture window.

Turns out, there is an issue with the electron window not returning focus correctly on mac - https://github.com/electron/electron/issues/5495. The trick to solving is to treat quick capture as a screensaver. When closing, you hide it by setting the opacity to 0 and sending hide: command to the first responder.

Another problem that has come up was the Quick Capture window couldn’t render over fullscreened apps. The result was, your desktop was switched whenever you wanted to capture. To deal with that, we set the window to screen saver. Combined with removing the app from the tray before rendering the window for the first time enables it to render over fullscreen apps.

After you render the window in screen saver mode you can once again show your app on the tray. You will of course always have another window on the background but since the opacity is 0 and mouse interactions are also disabled, you won’t be able to interact with it other than through the keyboard shortcut.

Final Words

We hope you find this blog useful. This is how we approach product building at acreom. If you like this style of blogs, have a look at The Quest for a Great Search.

We’ve introduced Quick Capture in v1.8.0, you can already try it out after downloading acreom - acreom.com/downloads. Tweet us what you think @acreom!

Shipping large ML models with electron

knarik — Tue, 11 Apr 2023 13:53:34 +0000

by @matoantos

How do I ship a large machine learning model with an electron app? Not so long ago, I couldn't find a resource to solving this problem, so I decided to do a write up of my experience, which could be useful for others. Let's dive right in.

Since the beginning of our work on acreom, we wanted it to have an IDE-like experience with real-time autocomplete suggestions in the context of knowledge base and tasks.

The first problem we decided to experiment with was to classify free text as a task or event. It turns out that building a binary classifier for such a use case is relatively easy; however, shipping it with Electron is the tricky part.

But why ship it with electron in the first place? It mainly boils down to speed of inference and user privacy. Hence, the problem presents the following specs:

fast inference (required for real-time suggestions)
minimum memory footprint
a good user experience means high accuracy with low false positive rate
user privacy - no API calls, fully offline, shipped on the client.

The ML part: exploration & fine-tuning

In spite of having no prior datasets available, this was a relatively fast and easy task. I have manually created a small dataset of roughly 1200 samples, where a task / event class looks like this: code tomorrow morning and negative class like code is simple, with a roughly 50/50 class balance.

An interesting side observation was to learn the semantics of such examples where 2 opposite samples share the same words but not the meaning. Later on, this allowed me to take a few actions to increase the overall performance of the model. I left out the lemmatization in the preprocessing pipeline and created feature engineering that applies additional weights to queries which start with verbs or include time for example. I have used quickadd, an open-source library for parsing time & date I have forked from ctparse (and upgraded with a lots of modifications).

After many experiments with different techniques and models, I settled with a bi-directional LSTM written in Pytorch. This worked surprisingly well considering the tiny dataset it was trained on. After some additional fine-tuning, I was happy to end up with F1 scores around 0.95, which is in the production territory for this use case. Great, the model works. Now all I need is to figure out the electron stuff.

Figuring out the electron stuff

This is where things get hairy. Firstly, The trained LSTM model with its custom word embeddings was not small in size by any means. Its dependencies, with custom word embedding and ~70k parameter model, had over 4GB in size all together!

Secondly, I wanted to keep our ML development process lean and fast when it comes to shipping in production. A few fundamental building blocks were necessary, so I could build future models systematically.

Okay, so maybe I can have some sort of an API interface written in python that would somehow communicate with the electron and it would all be freezed as a separate executable with the model, shipped alongside electron? Maybe this could work.

Inspired by the IDE language server protocol, I created an API interface between the electron and the Python ML interface. ZeroMQ turned out be an invaluable resource as a fast and lightweight messaging queue between the two.

Now all I needed was to freeze the Python interface into an executable that would accept requests from the electron, infer from model, and send response back.

PyInstaller seemed like the most maintained and developed tool to freeze python script into an executable, so I went with it. As expected, the freezed interface with the model was gigabytes large, so I had to figure out how to squeeze this. Fortunately, Onnx worked wonders and packaged the model into an inference only state, so I could throw away the Pytorch and Torchtext dependencies when freezing with Pyinstaller.Now the size of the executable with the model was 43MB instead of 4GB.

Pyinstaller throws a curveball every now and then with missing .dylib files in the process, but nothing that can't be figured out with symbolic links to the local dependencies. What did the trick was to offload heavy Pytorch and Torchtext libraries with their dependencies to the bare minimum so the script could work.

Here's brief rundown of how all of this works:

When the electron is opened for the first time, the main process retrieves available port and runs the ML executable, listening to the port. I have built in a retry logic for error handling and for disconnecting handlers on close.
The electron then sends an initialize message through the ZeroMQ to initialize the ML model, so it listens for requests. This came as an additional logic to prevent sending requests to the executable which was not yet initialized in the step #1. After the initialization, it listens for queries as JSON objects. Here's a sample query:

'{"requestId":{ID}, "action":"infer","service":"classifier","data":{"data":"code tmrw 7-9pm"}}'

When initialized, the executable listens for messages with the service type, reads its request, and runs it through the appropriate model for the inference.
Since this model is a binary classifier, the response propagated back through the messaging queue to the fronted of the application, like this:

'{"data": "1", "requestId":{ID}, "service": "classifier"}'

The Frontend takes care of triggering the visuals and converting the text into a task component upon the confirmation from the user within a timeout of the listener. The end result looks pretty solid!

This experiment went to production soon after, and while it's not the best and most desired UX implementation, it served with good learnings for future work.

If you have any questions or feedback, feel free to reach me martin@acreom.com

Building startup local-first

knarik — Tue, 28 Mar 2023 09:33:25 +0000

by @matoantos

For the majority of the past 2 years, our small team of 5 (1 designer, 4 engineers) have been working remote-first. While this lately popular trend worked for us in the beginning, soon after we felt something was missing. It was when we set up our first office, the real change came.

Looking back, it was one of the best decisions we’ve made. Here‘s what we have learned.

Communication is instant

When working remotely, we relied on Discord to handle all of our communication. This often meant that a meaningful chunk of our communication was async. We would occasionally miss notifications or had to coordinate our debugging and pair programming sessions. This often resulted in communication delays , and made it more difficult for us to convey information real-time.

Working in-person, most of these issues have disappeared. We are able to have more productive and efficient discussions, and we can clear up any misunderstandings or ambiguities more quickly.

Being in an office setting can lead to distractions from others' conversations when you are trying to stay focused on your work. ANC headphones turned out to be an invaluable tool for us, serving as a 'do not disturb' indicator and enabling us to focus on deep work.

Shipping and learning faster

Our productivity and learning curve velocity have risen significantly once we started working together in a physical space. We are shipping higher quality software, faster and more frequently. This is partly due to just being together and not having too many distractions in our own familiar space around us.

Within the first weeks of working together we have found our rhythm and routine which further helps us being more focused on the work by minimizing the need to think about what to do during the day. Our day begins at around 9am by a focus period until 11:45am. After lunch we have a cooldown period until around 1:30pm. In the afternoon, we do another stint of focus until around 5pm to 6pm, interrupted only by the bi-weekly sync at 3pm.

Another factor contributing to our faster delivery is the accountability we have for each other. If we do not deliver or do a poor job, we receive almost immediate feedback to do better. On the other hand, if we deliver good results, we receive positive feedback to stay on the right path.

Spending time together outside of work

The serendipity of talking about work as well as life in general, during lunchtime or breaks, often led us to new perspectives and advancements in the development process. It also does wonders for alignment on different issues and the overall direction.

Similarly, having random discussions about the task you are currently working on can help problem solving both by talking to another person about your problem, and perhaps letting things settle in your head, as well as hearing another opinion and point of view.

Pair programming

Pair programming plays a big role in the development process. Whether we are stuck on an issue, or want to speed up making of a new feature, we use pair programming. It helps us both to code faster, as well as be more error proof.

We also do pair VQA with our designer to do final adjustments on features. The effect of this is we always tweak the design just right and also learn to do UI better and speed up future development by understanding the concepts our app is based on.

Pair programming sessions quite often (intentionally or otherwise) turn into open forums where everyone chips in with their opinion and perspective. Having such discussions serves to align ourselves on the issue, serves as an early feedback session, which in turn speeds up the development process.

Culture

Building culture is easier in-person. As a small team building a startup from the ground, we work, eat and have fun together. Firstly, there’s so much serendipity that happens just by being side by side in this process. From exploring new ideas randomly to laughing together about how miserably we have failed at something - real-time. We of course could have done all of that remotely, but being present just adds a little bit of something magical to the equation.

Secondly, we believe culture is a sum of everyone’s decisions. It’s how we approach things, handle situations and work together as a team. More importantly, we not only get to know each other by having a first row seat to this experience, we influence each other. Hiding behind a screen can water down this experience.

Startup is an intense rollercoaster of highs and lows we get to experience - together. It would indeed suck if we sat in our own isolated cabins throughout the ride.

Takeaways

The switch to a physical office has had numerous benefits for our team, and we have gathered some important lessons from the experience.

Communication is more effective making it easier to collaborate and coordinate.
ANC headphones are an awesome 'do not disturb' indicator.
Productivity and learning curve velocity increases with ideas and resources being shared more easily.
Interacting throughout the day leads to improved development and improves alignment on different issues.
Pair programming is more effective when done in-person.
Working in a physical office helps us build a culture and makes the whole experience more enjoyable.

➤➤➤ https://acreom.com