DEV Community: Abdullah Atta

Notesnook is now open source!

Abdullah Atta — Sat, 03 Sep 2022 17:10:40 +0000

Hi! I am Abdullah Atta, the co-founder of Notesnook. I am really, really excited to share this news with the dev.to community. We have officially open sourced all of our client apps (sync server to be open sourced soon) on GitHub. We'd love if the community chimes in & support us on this new journey!

streetwriters / notesnook

A fully open source & end-to-end encrypted note taking alternative to Evernote.

Notesnook

An end-to-end encrypted note taking alternative to Evernote.

Overview

Notesnook is a free (as in speech) & open-source note-taking app focused on user privacy & ease of use. To ensure zero knowledge principles, Notesnook encrypts everything on your device using XChaCha20-Poly1305 & Argon2.

Notesnook is our proof that privacy does not (always) have to come at the cost of convenience. We aim to provide users peace of mind & 100% confidence that their notes are safe and secure. The decision to go fully open source is one of the most crucial steps towards that.

This repository contains all the code required to build & use the Notesnook web, desktop & mobile clients. If you are looking for a full feature list or screenshots, please check the website.

Developer guide

Technologies & languages

Notesnook is built…

View on GitHub

What is Notesnook?

Notesnook started out as a passion project back in 2019 (3 years after Standard Notes) after I got really frustrated by the lack of good, usable & privacy note taking apps in the market. It might not sound like a pain point to you but privacy is a critical issue in today's digital world & my notes contain my most private thoughts. If you don't share this opinion, I don't blame you.

Google has popularized the "100% free" concept in almost all markets. Look at Google Keep, Gmail, Calendar, Maps & the whole GSuite. Most of it is offered free of charge. Why? Are they crazy? It makes no sense until you realize that Google is primarily an Ads company. They thrive on user data.

If it's free, you are paying with your data.

Notesnook is our take on how note taking should be done. It features 100% end-to-end encrypted sync (free & unlimited for all), encrypted attachments (unlimited storage for as low as $0.99/mo) & encrypted publishing of notes (again free for all). It is a fully cross-platform app built using React & React Native with a primary focus on feature parity. In fact, all our clients have 99% feature parity (something no other note taking app can claim).

Why open source? Why now?

The truth is that we are late to the party. Notesnook was our first rodeo & we didn't realize how important open sourcing is for our users to inspire trust & confidence until much, much later. Better late than never though, right?

Our mission is to make Notesnook a quality product in the open source space like Docker, Nginx, PostHog etc. When we started out Notesnook, we asked ourselves 1 question:

Is it possible to create a privacy respecting note taking service without compromising on convenience & usability?

And here is our proof.

What kind of open source?

We are open sourcing Notesnook under (A)GPL-3.0-or-later. We will not tolerate any compromise on our users' freedom, privacy & peace of mind.

We open sourced Notesnook on September 1 and the response has been huge so far. I summarized our overall experienced in this tweet (would love if you give a follow!)

// Detect dark theme var iframe = document.getElementById('tweet-1565826085043986433-931'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1565826085043986433&theme=dark" }

How can you help?

We are a small company based in Pakistan entirely self-funded without any VC backing us. We don't have thousands of dollars for ad revenue or influential patrons who'll push us to the edge. All we have is platforms like these.

We'd love if you can star our GitHub repo, become a contributor, spread the word of privacy & tell others about your experience with Notesnook. If you have a platform (podcast, blog, etc.), we'd love to get featured. If you'd like to collaborate with us, we are listening.

That's all. I hope Notesnook will make a difference.

— May privacy reign

Constants In JavaScript (and all that bulls**t)

Abdullah Atta — Wed, 18 Mar 2020 17:10:00 +0000

If you have been programming in JavaScript for a little while you'd have noticed that const in JavaScript does not mean you can't change it. There's also no memory, performance or any other benefit. WTF!

Okay, okay. Look at this piece of code:

const toastStatus = "burnt"; // why though?
// let's change this
toastStatus = "not ok";

You will be expecting that it will give an error, right? Yup. You are right. It does give this error:

Uncaught TypeError: Assignment to constant variable.

So far so good. Let's say we want to give an id to the toast status:

const toast = {
    id: "da_best_toast",
    status: "burnt"
};

And something happens and this toast suddenly becomes unburnt (accidentally of course):

toast.status = "all ok"

You'd be an expecting the same old error, right? But guess what? No error. Not even a hint. It's all okay (as the toast said). WTF! Same is the case with any Object (i.e. Arrays, Objects, Classes, Functions etc. etc.)

WTF!!

Okay, okay. I guess that's a good enough rant for now. SO what's the problem?

The Problem

Nothing is immutable in JavaScript. Nothing is unchangable in JavaScript. Anything can become anything in JavaScript. That's why you have TypeScript but even then there are some quirks in JavaScript that will make you pull your hair out.

So what do we do if we want perfect immutability? Such that no one can change the object. At all?

You could use a third-party library like: Immutable-Js or immer. But why? Why, why, why? When JavaScript can do it but for some reason doesn't do it by default (via the const keyword)?

If you haven't noticed by now: Objects are extensible in JavaScript. Meaning you can add, delete or edit without any restriction. Any Object. Even the standard ones like global or window. Now, I understand that that is a cool thing and all but it has many, many disadvantages. So let's try to make our objects...constant.

The Solution..s

`Object.freeze`:

From MDN:

The Object.freeze() method freezes an object. A frozen object can no longer be changed; freezing an object prevents new properties from being added to it, existing properties from being removed, prevents changing the enumerability, configurability, or writability of existing properties, and prevents the values of existing properties from being changed. In addition, freezing an object also prevents its prototype from being changed. freeze() returns the same object that was passed in.

I think you got the point. It makes your object unchangable. Forever.

Here's the same example as above:

const toast = Object.freeze({
    id: "da_best_toast",
    status: "burnt"
});

Now if you wanted to change the status like this:

toast.status = "all ok"

It will give this error (be a good kid and turn on strict mode):

TypeError: Cannot assign to read only property 'done' of object '#<Object>'

And later if you accidentally add another property:

toast.bread = "baba bread";

It will give another error:

TypeError: Cannot add property bread, object is not extensible

And then if you try and delete the status (accidentally, of course):

delete toast.status

You'd get:

TypeError: Cannot delete property 'done' of #<Object>

In this way you can make any Object immutable. You could also write a utility function like this, to make it a bit flashier:

function im(obj){
    return Object.freeze(obj)
}

// and then use it like so:

const immutable = im({ data: "some_data" });

And if you wanted to check whether an object is frozen, simply do:

Object.isFrozen(toast); // === true

Now what if you wanted to only disallow new properties from being added/deleted but allow existing properties from being changed. In other words, what if you wanted "partial" immutability?

Well, for that you have:

`Object.seal`:

From MDN:

The Object.seal() method seals an object, preventing new properties from being added to it and marking all existing properties as non-configurable. Values of present properties can still be changed as long as they are writable.

So basically, it will only allow existing properties from being edited (not deleted). Let's see an example.

const network = Object.seal({
    stable: true,
    speed: "50mbps",
    status: "connected"
});

// let's suppose, the network is disconnected suddenly, you'd simply do:
network.status = "disconnected";

// however if you wanted to add upstream speed, it will throw:
network.upstream = "25mbps";

// or if you try and delete, it will also throw.
delete network.stable

You can easily check if an object is sealed or not using:

Object.isSealed(network); // === true

So far so good. But what if you wanted to disallow only additions but allow edits and deletions? For that you have Object.preventExtensions.

`Object.preventExtensions`

From MDN:

The Object.preventExtensions() method prevents new properties from ever being added to an object (i.e. prevents future extensions to the object).

Let's see an example:

const person = Object.preventExtensions({
    name: "Baker Yota",
    age: 35,
});

// now let's say you wanted to add a new property, it will throw.
person.father = "Samhil Yota"

// But you can edit the age:
person.age = 36;

// And delete it too:
delete person.age;

And if you wanted to check whether an object is extensible or not, simply do:

Object.isExtensible(person); // === false

So there you have it: Immutability in JavaScript.

Notes:

Objects that are manipulated in this way do not have their references altered. So frozen_object === real_object gives true.
You can put in any kind of Object. Be it an Array, Function, Class etc.

Conclusion:

Javascript is a nice language but it does have some wacky areas a beginner would not be expecting. Hopefully, this little lesson in immutability will have made you understood that area of "science". Now don't be annoying and give me a reaction. :D Just joking. But seriously, do leave a comment with your thoughts.

Thanks for reading,
thecodrr

P.S. Here is my latest project in case you are interested:

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s

The Fastest Directory Crawler & Globber for NodeJS

⚡ The Fastest: Nothing similar (in the NodeJS world) beats fdir in speed. It can easily crawl a directory containing 1 million files in < 1 second.

💡 Stupidly Easy: fdir uses expressive Builder pattern to build the crawler increasing code readability.

🤖 Zero Dependencies*: fdir only uses NodeJS fs & path modules.

🕺 Astonishingly Small: < 2KB in size gzipped & minified.

🔥 All Node Versions Supported: Unlike other similar libraries that have dropped support for Node versions < 10, fdir supports all versions >= 6.

🖮 Hackable: Extending fdir is extremely simple now that the new Builder API is here. Feel free to experiment around.

* picomatch must be installed manually by the user to support globbing.

Support

Do you like this project? Support me by donating, creating an issue, becoming a stargazer, or opening a pull request. Thanks.

…

View on GitHub

How I Wrote the Fastest Directory Crawler Ever

Abdullah Atta — Mon, 16 Mar 2020 10:36:45 +0000

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s

The Fastest Directory Crawler & Globber for NodeJS

⚡ The Fastest: Nothing similar (in the NodeJS world) beats fdir in speed. It can easily crawl a directory containing 1 million files in < 1 second.

💡 Stupidly Easy: fdir uses expressive Builder pattern to build the crawler increasing code readability.

🤖 Zero Dependencies*: fdir only uses NodeJS fs & path modules.

🕺 Astonishingly Small: < 2KB in size gzipped & minified.

🔥 All Node Versions Supported: Unlike other similar libraries that have dropped support for Node versions < 10, fdir supports all versions >= 6.

🖮 Hackable: Extending fdir is extremely simple now that the new Builder API is here. Feel free to experiment around.

* picomatch must be installed manually by the user to support globbing.

Support

Do you like this project? Support me by donating, creating an issue, becoming a stargazer, or opening a pull request. Thanks.

…

View on GitHub

If it's not fast, the work is not yet complete.

Said no one ever.

Writing code fast and writing fast code are two very different things. One could even say they are opposites. If you are in the habit of writing code very fast there's a great chance that it will be slow. Writing fast code is not just about choosing the fastest language, the fastest platform, the fastest libraries etc. Anyone could do that. What makes code truly fast is the little things; the loops, the conditionals, the assignments, function calls etc.

Introduction

I woke up Thursday morning, groggy, upset and very, very sleepy. My head was hurting. I had been coding all night and I had finally finished the first version of fdir; the fastest directory crawler for NodeJS. I opened my laptop after a good breakfast, splendid tea and a nice walk; ran the benchmark again: fdir was up against 13 other contenders. Always fdir would come out on top in both synchronous and asynchronous crawling. But it was not ready yet...

The First Try

Writing fast code starts from the very first line.

The purpose of fdir is simple; crawl as many directories as possible in as short a time as possible. The first version of fdir used recursion; it went something like this:

Note: This is an overly simplified version.

function sync(dir) {
  const dirents = fs.readdirSync(dir, { withFileTypes: true });
  const paths = [];

  dirents.forEach(dirent => {
    const res = `${dir}${path.sep}${dirent.name}`;
    if (dirent.isDirectory()) {
     sync(res).forEach(push.bind(paths));
    } else {
      paths.push(res);
    }
  });
  return paths;
}

This could already beat almost everything out there. There's nothing special in it. Just some loops, recursion etc. etc. So what made it faster than everything?

The first line.

withFileTypes: true to be specific. This allowed me to skip the fs.lstatSync syscall for each item in a directory. Yup. You can imagine the speed boost.

This line must be making you jump out of your underwear. Why didn't you use path.join?!!

....
const res = `${dir}${path.sep}${dirent.name}`;
....

Because it's slow. It's much slower than just using path.sep. I benchmarked it. It's about 50% slower.

v1 Benchmark:

As you can see, only rrdir.sync comes even close to fdir and that's because it uses a similar approach.

The Defenders Arrive

Saturday evening, I was posting about fdir on Reddit. 2 hours later, the author of rrdir opened a PR to update his library to improve async performance. I was heavily refactoring fdir and adding support for Node version < 10 so his PR couldn't be merged. After an hour, though, I manually updated his library and ran the benchmarks again.

fix benchmark, bump rrdir, add rrdir.stream #2

silverwind posted on Mar 14, 2020

async rrdir shouldn't look so bad anymore with this

View on GitHub

Results:

Async:

Sync:

Two hours after that, the author of fs-recursive opened a PR to include his library in the benchmark. The title of the PR was: "I am fastest now". And it was. By quite the margin (50%). Of course I merged it.

Now I'm the fastest #3

simov posted on Mar 14, 2020

Thanks for the benchmark 👍

View on GitHub

The Rewrite

And of course, I couldn't let fs-recursive take the first place. I had spent "one whole night" writing the fastest crawler. I couldn't back down now. So I rewrote the whole algorithm. From top to bottom. It removed recursion (from fdir.sync), stopped array recreation, only used a single Promise per fdir.async call etc. etc. The code now looked like this:

function sync(dir, options) {
    const paths = [];
    const dirs = [dir];
    var i = 0;
    while (i < dirs.length) {
        const dir = dirs[i];
        const dirents = fs.readdirSync(dir, readdirOpts);
        dirents.forEach(function(dirent) {
            let fullPath = `${dir}${path.sep}${dirent.name}`;
            if (dirent.isDirectory()) {
                dirs.push(fullPath);
            } else {
                paths.push(fullPath);
            }
        });
        ++i;
    }
    return paths;
}

The code is pretty self-explanatory; we keep adding directories to the dirs array so the loop never ends until there are no more directories. But fdir.sync was already the first so I didn't really need to optimize it further but I couldn't resist. Removing the multiple array initialisation, recursion gave a good speed boost and overall made the code quite clean (imo).

The real challenge was optimizing the async version. As you all know, looping with asynchronous/callback functions is quite a PITA. So after everything this came into being:

function async(dir, options) {
  return new Promise(function(resolve) {
    const paths = [];
    const dirs = [dir];
    let cursor = 0;
    let readCount = 0;
    let currentDepth = options.maxDepth;
    function walk() {
      // cache the total directories before starting the walk
      let total = dirs.length;
      for (; cursor < total; ++cursor) {
        const dir = dirs[cursor];
        fs.readdir(dir, readdirOpts, function(_, dirents) {
          dirents.forEach(function(dirent) {
            let fullPath = `${dir}${path.sep}${dirent.name}`;
            if (dirent.isDirectory()) {
                dirs.push(fullPath);
            } else {
                paths.push(fullPath);
            }
          });
          // check if we have walked all the directories we had
          if (++readCount === total) {
            // check if we got any new ones
            if (dirs.length === cursor) {
              resolve(paths);
            } else {
            // walk again if we have new directories.
              walk();
            }
          }
        });
      }
    }
    walk();
  });
}

The concept is quite similar to fdir.sync but we retained recursion (although a new version of it). I couldn't find a way to reliably remove recursion.

The Results

And, fdir was back on top.

Async:

Sync:

The Takeaway

The moment you have all been waiting for. The takeaways. What I learned. What I didn't. Etc. etc. However, I don't have "don't use X but Y" kind of lessons for you. I am sorry. The reason is, performance is dependent upon use-case.

Don't write code fast. You will have to rewrite it again and again. And if it's a large codebase, it will very soon become a PITA. So write it carefully, take all the precautions, do all the optimizations.
A single millisecond matters. Oftentimes, we do not add an optimization just because it adds only a millisecond. But "drop by drop a river is born" right?
NodeJS is very fast, you just have to write honest code. Don't make it complex just for the hell of it. Keep it simple, keep it fast.
Benchmark. Benchmark. Benchmark. JavaScript has a lot of ways of doing one thing, multiple loops, iterators etc. You won't know what's fastest until you benchmark. I ran benchmarks for each line of my code that could have an alternative. Remember, every millisecond matters.

But I am gonna give a few "use X not Y" lessons anyway.

Use as few conditionals as possible. Each branch adds an overhead and although the engine optimizes it, you have to be careful.
Prepare for errors beforehand. Try-catch is expensive. Be careful.
for, forEach and array.reduce are all very fast. Use whichever works for you. Actually, use them all and see which one makes your code faster.
Research the API before using it. More often than note, there's something in the API that will reduce unnecessary calls, bootstrapping, error-checks etc. Like withFileTypes: true.
Use string methods as less as possible. Actually, use strings as less as possible. Pushing a string into an array is much slower than pushing an int. (I didn't get to apply this).

So What Happens Now?

Well, I will keep benchmarking and finding ways to make it faster. I will try using WebAssembly, Workers etc etc. Innovation, my friend, innovation. Currently, fdir can crawl about 1 million files in ~900ms but I want to reduce it down to 500ms. The current code is as optimized as it can get. So let's see what I try.

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s

The Fastest Directory Crawler & Globber for NodeJS

⚡ The Fastest: Nothing similar (in the NodeJS world) beats fdir in speed. It can easily crawl a directory containing 1 million files in < 1 second.

💡 Stupidly Easy: fdir uses expressive Builder pattern to build the crawler increasing code readability.

🤖 Zero Dependencies*: fdir only uses NodeJS fs & path modules.

🕺 Astonishingly Small: < 2KB in size gzipped & minified.

🔥 All Node Versions Supported: Unlike other similar libraries that have dropped support for Node versions < 10, fdir supports all versions >= 6.

🖮 Hackable: Extending fdir is extremely simple now that the new Builder API is here. Feel free to experiment around.

* picomatch must be installed manually by the user to support globbing.

Support

Do you like this project? Support me by donating, creating an issue, becoming a stargazer, or opening a pull request. Thanks.

…

View on GitHub

Support fdir on ProductHunt

Thanks for reading,
thecodrr

The (so) Many Ways of Concatenating an Array

Abdullah Atta — Fri, 13 Mar 2020 07:53:48 +0000

The truth behind performance is benchmarks. JavaScript is the type of language in which there are many ways of doing just one simple thing. When you have many ways of doing just 1 thing, it gets complicated; you have to make a choice. And how do you know which way is the fastest? That is what makes JavaScript confusing and complicated.

Have a look at this code:

let array = [1,2,3,4,5,6,7,8];
let second_array = [9,10,11];

How would you concatenate these 2 arrays? In JavaScript there are about 10 ways of concatenating (joining) 2 arrays. Which one would you choose?

Array.prototype.concat:

This is the simplest possible way.

array = array.concat(second_array);

Array.prototype.push:

Again relatively simple, although, a little verbose.

array.push(...second_array);

Array.prototype.reduce:

Just a teeny-tiny bit more complex.

array = second_array.reduce((all, current) => {
   all.push(current);
   return all;
}, array);

For-each:

second_array.forEach(item => array.push(item));

For-of:

for (var item of second_array) {
    array.push(item);
}

For-in:

for (var i in second_array) {
    array.push(second_array[i]);
}

For:

for (var i = 0; i < second_array.length; ++i) {
    array.push(second_array[i]);
}

The Spread-Operator:

A little bit more verbose but still understandable.

array = [...array, ...second_array];

Which way is your go-to when it comes to concatenation? Leave a comment below.

You'd think that JavaScript engines and compilers would be intelligent enough for you to choose any of these ways and be equally fast. You'd be wrong. 100% of time.

Let's Benchmark:

After benchmarking all these methods of array concatenation, here are the results for Google Chrome:

Array.prototype.concat x 6,006,271 ops/sec ±1.51% (61 runs sampled)

Array.prototype.push x 15,125,549 ops/sec ±0.59% (62 runs sampled)

Array.prototype.reduce x 26,952,859 ops/sec ±0.59% (64 runs sampled)

For-each x 26,682,412 ops/sec ±0.46% (64 runs sampled)

For-of x 23,628,346 ops/sec ±0.83% (64 runs sampled)

For-in x 2,951,756 ops/sec ±0.52% (64 runs sampled)

For x 25,458,988 ops/sec ±0.56% (64 runs sampled)

The Spread-Operator x 15,266,695 ops/sec ±0.57% (65 runs sampled)

You can go run and see the results yourself here.

Which should I choose?

You should of course always choose the fastest. But oftentimes, the fastest on one browser is actually slower on the other. In my opinion, your go to method should be the for-loop one. It is the best optimised on almost all the browsers. But if you are targetting NodeJS's V8, go with Array.prototype.reduce or the ForEach method. If great readability is your concern, I'd go with Array.prototype.push or the spread operator.

Why is X method faster than Y?

I don't know very deeply about what happens under-the-hood in each method. Maybe someone with more knowledge can specify? : )

Thanks for reading, hope you learned something.
Have a great day!

DEV Community: Abdullah Atta

Notesnook is now open source!

streetwriters / notesnook

A fully open source & end-to-end encrypted note taking alternative to Evernote.

Notesnook

An end-to-end encrypted note taking alternative to Evernote.

Overview

Developer guide

Technologies & languages

What is Notesnook?

Why open source? Why now?

What kind of open source?

How can you help?

Constants In JavaScript (and all that bulls**t)

The Problem

The Solution..s

Object.freeze:

Object.seal:

Object.preventExtensions

Notes:

Conclusion:

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s

The Fastest Directory Crawler & Globber for NodeJS

Support

How I Wrote the Fastest Directory Crawler Ever

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s

The Fastest Directory Crawler & Globber for NodeJS

Support

Introduction

The First Try

v1 Benchmark:

The Defenders Arrive

fix benchmark, bump rrdir, add rrdir.stream #2

Results:

Now I'm the fastest #3

The Rewrite

The Results

The Takeaway

So What Happens Now?

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s

The Fastest Directory Crawler & Globber for NodeJS

Support

The (so) Many Ways of Concatenating an Array

Array.prototype.concat:

Array.prototype.push:

Array.prototype.reduce:

For-each:

For-of:

For-in:

For:

The Spread-Operator:

Let's Benchmark:

Which should I choose?

Why is X method faster than Y?

`Object.freeze`:

`Object.seal`:

`Object.preventExtensions`