DEV Community: Yipei Wei

AFFiNE Local User Guide: Keep Your Data In Your Hands

Yipei Wei — Mon, 29 Aug 2022 02:16:00 +0000

AFFiNE is a local-first personal knowledge base, which means you have full control over your data. In this article we'll help guide you how to use AFFiNE and keep all your data locally stored, and then we'll talk more about the technicals and how we do it.

0x01 Sync data to local hard drive

AFFiNE provides an out-of-the-box live demo that allows you to use AFFiNE without any downloads, installing or compiling. And you'll still be able to keep your data local.

You can head over to our official website, and click Try it Online. You'll be sent to our live demo where you can try out the latest developments of the product.

Whenever you open the demo - that includes closing the tab or refreshing the page - you will have to repeat the following process.

First, let's SYNC TO DISK - you'll find that button in the upper right corner.

Then you'll be asked to create (or select) a directory as your working folder, this is where all your data will be stored.

You can create a new folder if it's your first time - or open an existing folder to load your data.

please select "view files"

please "save changes"

To ensure your security, these permissions will be requested every time you load the app. Both permissions are required for the app to function correctly.

After the authorization is completed, the SYNC TO DISK button in the upper right corner will be hidden, this also indicates now because you have completed the process successfully. All your modifications will be automatically saved to the folder you chose.

You can transfer/sync this data to another PC if you wish, allowing you to freely access your data. Just follow these steps again, this time choosing the existing folder where your data is stored.

0x02 How we make AFFiNE local first
1. Why other products need to be used online
A lot of personal knowledge base software claims that online services can provide a better experience than local services. Sometimes that comes down to technical limitations. But we at AFFiNE want to do things differently - without compromise.

When you wish to allow remote access to a document, one of the easiest ways to do this is by uploading the file. Then, the server marks a version for the file, and when you update the file, the version updates too when the new file is uploaded.

However, what if we need to make changes to only a part of the file? Do we need to upload the whole file again and again? There are alternatives and the way we approach collaboration is with a series of algorithms collectively called Operational Transformation. This process calls each of your edits an operation, collects them, and executes them one by one on a remote server, and returns the result after execution.

But no matter how it is optimized, the core logic of OT is to establish causality for operations, so that they can operate on a piece of data in a fixed order. This means it is often necessary for them to have a central server, and all clients participating in the collaboration record their operations, submit them to a central server to perform the operation, and then wait for the results to be feedback. So, how can we do this without a central server?

2. How do we solve this problem
We want to make a local-first application, which means we want users to be able to edit offline, on multiple clients and be able to seamlessly submit changes when going back online. We consulted many papers and tested many libraries, and finally chose Yjs, which is based on YATA CRDT[1].

CRDT stands for Conflict-free replicated data type. It aims to build an algorithm that allows two sets of data to be merged together without conflict. YATA is one type of CRDT, and Yjs is the implementation of that YATA algorithm.

Why does CRDT allow two pieces of data to be merged without conflict? Maybe the simplest way we can explain this is as follows. We can design two copies of data to be commutative, eg: op1 + op2 = op2 + op1, which means if we make 2 edits (or operations) to the same document on 2 different clients, as long as both parties can hand over their operations to the other, they can get the same final result regardless of who generated the operation first.

exchange operation without server

We applied Yjs as the data storage layer at the bottom, and built a new library based on it, we call it JWT[2], which will provide Block abstraction to the upper layer, and provide post-processing capabilities such as full-text search and data analysis. This means that the upper layer (in AFFiNE, the upper layers are our block editor Virgo and our vector editor Phasor) only needs to understand that what it is operating on is a block.

3. Your data is always yours
Based on the powerful capabilities of the CRDT algorithm, the excellent optimization of Yjs, and the abstraction of JWT[2]'s ability to block, we are able to provide a set of local-first data layer infrastructure, which enables AFFiNE to provide powerful offline collaborative editing capabilities and local editing experience.

Currently, all editing of AFFiNE is based on JWT[2], and data processing does not require a server. Therefore, whether you use AFFiNE in our Live Demo, a locally built Docker container or a self-hosted server, all the data you generate is saved to your local hard disk through the SYNC TO DISK feature, and your data is always yours.

[1] Near Real-Time Peer-to-Peer Shared Editing on Extensible Data Types

[2] Our current version of the data backend is written in TypeScript and called JWT, and the next generation data backend is written in Rust and called JWST, both codenames inspired by James Webb Space Telescope

We welcome every user to offer feedback and suggestions. Please feel free to contact us on:

Reddit: https://www.reddit.com/r/Affine/
Discord: https://discord.gg/Arn7TqJBvG
Telegram: https://t.me/affineworkos
Twitter: https://twitter.com/AffineOfficial
Medium：https://medium.com/@affineworkos

How we build AFFiNE with Typescript/React/Rust

Yipei Wei — Thu, 04 Aug 2022 09:01:43 +0000

While Miro(visual collaboration platform) and Notion(note-taking tool) are silky smooth and powerful enough to use, they are not open source solutions.

So, what about the next-gen knowledge base that is more private, transparent, customizable, and available for Windows/OS/Linux？

That's where AFFiNE sparkles!

Unlike Miro and Notion, which focus on whiteboard and page, AFFiNE defines its positioning as an all-in-one KnowledgeOS. It supports kanban, table, and rich-text paragraphs as building blocks to form pages or whiteboards. It carries out integrated processing here in document editing, data processing, or brainstorming.

Build with Rust and Typescript, and run the entire project with one command line, AFFiNE gives all developers maximum imagination in the simplest way.

AFFiNE is a Perfect Combination of Privacy and Experience of Developer and User

AFFiNE is fairly new.

It is an open-source project that aims to overcome some limitations of Notion and Miro in terms of security and privacy. It helps you carry the to-do list recorded in the meeting, the goal written in the document, the brainstorming on the whiteboard of the video conference, and the feedback obtained in the customer interview in a unified place. AFFiNE can become a place to store, integrate and manage all workflows.

On the basis of safeguarding data security and user privacy, the experience of users and developers also matters. And, AFFiNE does a decent job at it, if not better than Miro and Notion.

Note that AFFiNE is still in its alpha phase.

Currently, the aim of this project is not only for functionality, but also for a customized experience, community-driven opportunities, and data privacy.

Notion vs. AFFiNE: What Are Your Priorities?
While it is meant to replace Notion as an open-source solution, it may not be for everyone. So, if you are going to choose AFFiNE over Notion, you will get the following benefits:

Privacy
Notion can be used as closed source software to directly access your private data in the cloud. Compared to this, you can host AFFiNE as per your preference.

With AFFiNE, you own your data, no matter what. With no sacrifice to collaboration and share, all your data, along with a slice of AFFiNE itself, is in your hand.

Transparency
AFFiNE is an open-source project, so you are always welcome to modify and view the code.

Performance and Native Experience
AFFiNE is built using Rust and typescript, which provides a modern and stable user experience while maintaining performance.

AFFiNE is fully built with web technologies so that consistency and accessibility are always guaranteed on Mac, Windows, and Linux. The local file system support will be available when version 0.0.1beta is released.

Features of AFFiNE
Shape your page
An always good-to-read, structured docs-form page is the best for your notes, but a boundless doodle surface is better for collaboration and creativity.

We make the shift and reshape easy, so you can shape the docs your way.

With AFFiNE, every page comes with 2 views, and you can access and edit fully functioned blocks anywhere, in any form.

Plan your task
We really appreciate the idea of Monday, Airtable and Notion database. They inspired what we think is right for task management. But we don't like the repeated works -- we don't want to set a todo easily with markdown but end up re-write it again in Kanban or other databases.

With AFFiNE, every block group has infinite views, for you to keep your single source of truth.

Sort your knowledge
A good, semantic knowledge base is made of reusable, linked elements. AFFiNE is completely made on top of the idea of blocks. Blocks can be 2-way linked, duplicated, reused, and synced between docs.

As semantic web lovers, we don't like the idea of folders that much, and we make everything easy to find.

The biggest function of AFFiNE is that any block can be converted with one click: the page editing mode + Kanban view mode + whiteboard mode can be seamlessly linked - making it easy, convenient, and burden free for all users to use.

In addition, AFFiNE has more page views and whiteboard views, which can help different users brainstorm and align progress with other partners at any time while precipitating and processing data sets, and write data-related reports and complete meetings without changing the working tools flow.

You can expect more feature additions as the development continues. Some existing highlights include:

Native cross-platform support.
Add to-do in text mode, and transform to board mode.
Ability to self-host it or install it on your computer.
Customizability.
Data privacy (top priority).
A single code base for better maintenance.
Community-driven extensibility.
Multi-column view and advanced formatting support.
Click anywhere and type with MarkDown support.
Keyboard shortcuts for editing cell/grid.

Installing AFFiNE on Linux/Windows/MacOS

In either case, you can check AFFiNE’s official documentation to build it from the source. Explore more about it on its official website

Wrapping Up
If you need a simple Notion-like or Miro-like application with a native experience, AFFiNE is an interesting choice.

AFFiNE has a variety of views such as whiteboard, and document editing, and different views can realize data linkage and one-click switching. Any block element can be easily reorganized and reused to help users reduce invalid working hours, improve work efficiency and release time on work with more creative value.

Therefore, in 2022, if you want to have a tool that can store, integrate, manage and create all workflows, you might as well try AFFiNE!

If you are interested in AFFiNE after reading this article, please click here to know more.

If you have any questions about productivity tools or AFFiNE products, please feel free to ask questions on Reddit: https://www.reddit.com/r/AFFiNE/

This is another way to contact AFFiNE:

Telegram: https://t.me/AFFiNEworkos
Twitter: https://twitter.com/AFFiNEOfficial
Medium：https://medium.com/@AFFiNEworkos

Hope you have a nice time with AFFiNE!