DEV Community: sasa cocic-banjanac

A Practical Guide To Browser Extensions - From Zero To Publishing

sasa cocic-banjanac — Tue, 03 Nov 2020 15:48:18 +0000

A Practical Guide To Browser Extensions

Recently I've had a serious problem with wasting my time on watching Youtube, Netflix, HBOMax, sports, and other brainless entertainment. I love watching the stuff otherwise I wouldn't be doing it. After spending way too much time on it I decided that I need to do something about it. But before I do something about it I want to shoutout Lovecraft Country on HBO, because the show was great. If you like horror/spooky/mystery stuff check it out.

If you've been following Learning Computations you'll know that I installed Arch Linux recently, and talked about all the stuff I learned in the process. While configuring Arch it really inspired me to make my own stuff after seeing just how many solutions there where for the same problem. It made me think why don't I create a tailor made solution for my own problem. So I did. I made a browser extension to fix my problem of not being able to stop myself from watching brainless entertainment.

Here's what we'll do:

Define the web extension that will stop me from being a lazy piece of trash
Look through extension docs and figure out what an extension is and what it's made of
Build an extension
Finish off with publishing an extension to Firefox and Chrome add on stores

Chapter 1: What's being built? What is an extension?

Alright let's start by defining what the web extension should do. The extension I want should let me

Create categories and add time limits to those categories
Add websites to categories and track my time on those websites
Block me from all the websites in that category once I hit the limit
Set a bed time. Once it's bed time all websites I visit are blocked

To keep this article focused I'm only going to implement the bed time feature. I want to focus on web extensions, and not logic specific to my application.

The first place to look was the docs. The your first extension tutorial in the Mozilla extensions docs seemed like a logical place to start. In this tutorial I built an extension that changed the border of pages belonging to the mozilla.org domain. Lets briefly cover this tutorial.

In following this tutorial I created a directory with some files that looks like this:

borderify
- manifest.json
- borderify.js
- icons/...
The first thing it asked me to do is create a manifest.json file and fill it out with the contents they provide. What is manifest.json? They don't say, but we'll answer this question in a bit.
One key in the manifest.json is content_scripts I'll let the tutorial explain this

The most interesting key here is content_scripts, which tells Firefox to load a script into Web pages whose URL matches a specific pattern.
In this case, we're asking Firefox to load a script called "borderify.js" into all HTTP or HTTPS pages served from "mozilla.org" or any of its subdomains.

Once you link borderify.js by adding it to content_scripts in manifest.json you add some JS to borderify.js to make the border of mozilla.org domains red.

If you have some time I'd recommend doing the tutorial as it isn't too time consuming, and it'll make things more concrete. If you don't then don't worry we'll cover everything it does. The tutorial doesn't go into much detail, but it offers a starting point.

Great. I've done this tutorial, created these files, but I'm not really sure how all the pieces fit together, what is an extension made of exactly, and what else can extensions do? Let's try to figure these out so we have a better picture of what's going on.

Alright so what is an extension? The next place in the docs I checked out was What Are Extensions, and it was a bit more helpful.

An extension adds features and functions to a browser. It’s created using familiar web-based technologies—HTML, CSS, and JavaScript.
It can take advantage of the same web APIs as JavaScript on a web page, but an extension also has access to its own set of JavaScript APIs.
This means that you can do a lot more in an extension than you can with code in a web page.....
Extensions for Firefox are built using the WebExtensions APIs, a cross-browser system for developing extensions. To a large extent, the API is
compatible with the extension API supported by Google Chrome and Opera. Extensions written for these browsers will in most cases run in
Firefox or Microsoft Edge with just a few changes. The API is also fully compatible with multiprocess Firefox. - What Are Extensions

Ok now I'm getting somewhere. Web extensions aren't that different from normal JS, CSS, and HTML apps, but they have access to a special API. The Web Extensions API. The nice sounding part about this is it seems like the code I write will be compatible with other browsers! Which is great to hear I don't want to write different code for basically the same thing. There's some gotchas here, but we'll cover them later. I'm focused on building my extension for Firefox right now, but once I get to Chrome you'll see the mistakes I made.

Ok I have an idea about what a Web Extension is and the technology it uses, but still don't know how the tutorial app fully ties into this. Lets figure that out.

Chapter 2: What is an extension made of?

The your first extension tutorial mentions the Anatomy of an Extension article. Here we'll figure out what an extension is actually made of.

An extension consists of a collection of files, packaged for distribution and installation - Anatomy of an Extension

Alright then. An extension is just some files. Very cool I guess.

Here's the answer to "what is manifest.json?":

This is the only file that must be present in every extension. It contains basic metadata
such as its name, version, and the permissions it requires. It also provides pointers to other files in the extension.

In other words manifest.json is the glue that hold together my extension. It's the file that tells the browser "hey I'm an extension, and here's my name, version, permissions, and all the files I use to do what I need to do Mr. browser".

So all an extension is a manifest.json + other files (like the content_scripts key) which manifest.json points to. This is exactly what the tutorial app is. Things are starting to make more sense.

Chapter 3: Lets build this shit

manifest.json

Now i've got an idea what an extension is, and what its made up of. Next on the agenda is figuring out what my extension needs. Based on Anatomy of an Extension this is what I'll add:

Icons - For the extension and any buttons it might define.

Obviously my extension has to look ver very cool so I'll need some icons

Sidebars, popups, and options pages - HTML documents that provide content for various user interface components.

I'll need a way to set a bed time so I'll use one of these to create an HTML form.

Content scripts - JavaScript included with your extension, that you will inject into web pages.

I'll need to block websites after the bed time I set, and changing the HTML of existing sites seems like an easy way to do this. The only question here is how will I get my bed time into the content script?

All of these things will be part of my manifest.json, which will get setup as we go along. Remember manifest.json is our glue. manifest.json has a lot of keys we won't get to, but it's worth checking out the reference to see all the details: manifest.json reference

Oh also while digging around in the docs I found this about manifest.json

It is a JSON-formatted file, with one exception: it is allowed to contain "//"-style comments.

This is fucking cool. If you've worked with JSON you'll know it doesn't let you have comments. This seems like a massive technological advancement so I'll be using it, but this might be the time to ask yourself has technology gone too far? Anyways, this is very exciting.

Bad news is when I published to the Chrome web store I ran into issues with the comments I added into my manifest.json. I didn't have these issues when I published to Firefox. If you want to comment your manifest.json you'll need to remove them when you publish to Chrome.

Icons

First up is figuring out a way to add icons. To start I'm going to create my initial manifest.json. Here's what I used to start out:

manifest.json

{
    "author": "you already know it's ya boi",
    "manifest_version": 2,
    "name": "sleepy-time",
    "version": "1.0",
    "description": "get that good sleepy-time you need",
}

If your wondering about any of the keys then the manifest.json reference above can give you more information.

To add Icons we simply need some images of the appropriate size, and to link to them in our manifest.json. Here's what that looks like:

"icons": {
    "48": "icons/trust-nobody-v2-48.jpg"
},

The 48 here is the size of the icon (48px X 48px) and icons/trust-nobody-v2-48.jpg is the location of the icon relative to manifest.json

Sidebars, popups, and options pages

Next up is figuring out a way to set the bed time. A UI seems like a natural place to put this so lets see how I can add one. The docs say there are 3 options

Sidebar - A pane that is displayed at the left-hand side of the browser window, next to the web page
Popup - A dialog that you can display when the user clicks on a toolbar button or address bar button
Option - A page that's shown when the user accesses your add-on's preferences in the browser's native add-ons manager

I'm going to go with a popup as I'm not too picky about how I set my bed time. Here's what the docs say about creating a popup:

The popup is specified as an HTML file, which can include CSS and JavaScript files, as a normal web page does. ....
The HTML file is included in the extension and specified as part of the browser_action or page_action key by "default_popup" in the manifest.json: - Popups

Looks like to get a popup I only need to add an HTML file, update manifest.json with a browser_action property, and then specify the HTML file in the default_popup key under it. Here's what that looks like:

"browser_action": {
    "default_popup": "popup.html"
}

Here's what my HTML looks like:

popup.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <script src="mypop.js"></script>
    <link rel="stylesheet" href="styles.css" />
  </head>
  <body>
    <div>Hello popup</div>
    <button id="my-button" onclick="logSome()">Click this for something</button>
  </body>
</html>

I've also added a JS file that looks like this:

popup.js

function logSome() {
  console.log('clicked a button. Nice!');
}

So I click my extension and the popup, well pops up. I click my log button and it doesn't log... I look in the console and I see

Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”).

Fuck. CSP. If your not familiar with CSP I'd recommend looking at this and this. Basically CSP stops you from doing things that you might normally e.g. onclick="logSome()" in the good name of security. In this case the default CSP policy is blocking me from executing inline Javascript. In order to satisfy CSP I need to remove my inline Javascript and do everything in popup.js and it'll work. That code looks like:

popup.js

function logSome() {
  console.log('clicked a button. Nice!');
}

document.addEventListener('DOMContentLoaded', function () {
  var clickyButton = document.querySelector('#my-button');
  clickyButton.addEventListener('click', logSomething);
});

After these changes my log button works!

Storing Data

I've got my UI up, but I don't have any way to store the bed time value or get it so I can use it in my extension. To fix this we'll take our first look at using the Web Extensions API.

The Web Extensions API gives extensions superpowers. Basically it allows extensions to do things that normal web applications can't. In some cases it's necessary to ask for permission in order to use specific APIs. How do you ask for permissions you might ask? If you guessed manifest.json you is right. We'll see how that works in a bit. Finally, all APIs are accessed through the browser namespace and we'll see an example of this as well.

There's a lot of ways to store data, but I'm going to use the storage API, which will let me store and retrieve data in my extension. So I go to the docs as one does. I find and look through the storage docs to understand how this API works, and there's a couple things that jump out at me.

There's three types of storage, but I'm interested in one called sync. sync will let me store and retrieve data across all the browsers that I'm logged into. I want this so I can set my bed time across different computers for example. The storage docs have more info on storage types if you'd like to check it out.
sync provides me with two methods to get and retrieve data: storage.sync.get and storage.sync.set
> To use this API you need to include the "storage" permission in your manifest.json file. - storage docs
> Note that the implementation of storage.sync in Firefox relies on the Add-on ID. If you use storage.sync, you must set an ID for your extension using the browser_specific_settings manifest.json key. - storage docs

Let's put all this together now. I'll start by requesting the storage permission, and setting an Add-on ID. Here's what that looks like:

manifest.json

"permissions":[
    "storage"
],
"browser_specific_settings": {
    "gecko": {
        "id": "myID@ID.id"
    }
},

browser specific settings docs - I didn't really touch on this, but here's more info if your interested.

permissions info - more info on permissions

Now I have the correct permissions and I've set an Add-on ID. Now I'm free to use the storage API. I'm going to replace the code I used for logging with the new storage code. Here's what that looks like:

mypop.js

function setBlockTime(blockTime) {
  var blockTimeEle = document.querySelector('#block-time');
  if (blockTime.blockTime) {
    blockTimeEle.value = blockTime.blockTime;
  }
}

document.addEventListener('DOMContentLoaded', function () {
  // populate the form if a value exists in the store
  browser.storage.sync.get('blockTime').then(setBlockTime);

  var form = document.querySelector('#settings-form');
  form.addEventListener('submit', (event) => {
    event.preventDefault();
    let timeToBlock = document.getElementById('block-time').value;
    browser.storage.sync.set({
      "blockTime": timeToBlock,
    });
  });
});

Here's what the update HTML looks like:

popup.html

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <script src="popup.js"></script>
        <link rel="stylesheet" href="styles.css" />
    </head>
    <body>
        <div>Blacklist settings</div>
        <form id="settings-form">
            <label for="">Sleep Time</label>
            <input id="block-time" name="" type="text" value=""/>
            <button type="submit">set sleep time</button>
        </form>
    </body>
</html>

storage is just one of many APIs available in the Web Extensions API. To see everything it offers you can look at the Javascript API listings in the Javascript APIs page. There's ways to get at tabs, windows, HTTP requests, and much more.

Alright I've got a way to store and retrieve data. To put the finishing touches on this now I just need to block pages I visit past my bed time.

Content Scripts

To finish off let's see how to add content scripts. Again I go to the one thing I consider holy the docs. In particular I go to the content scripts docs

Here's what they tell me about content scripts

A content script is a part of your extension that runs in the context of a particular web page...

Just like the scripts loaded by normal web pages, content scripts can read and modify the content of their pages using the standard DOM APIs...

Content scripts can only access a small subset of the WebExtension APIs, but they can communicate with background scripts using a messaging system, and thereby indirectly access the WebExtension APIs.

We aren't going to talk about background scripts here, but they're very useful for certain applications, and I suggest looking into them if your building an application of your own. Sadly content scrips aren't allowed full access to the Web Extensions API, but they are allowed to use storage.

There are 3 ways that content scripts can be loaded.

At install time, into pages that match URL patterns - Using the content_scripts key in your manifest.json, you can ask the browser to load a content script whenever the browser loads a page whose URL matches a given pattern.
At runtime, into pages that match URL patterns - Using the contentScripts API...
At runtime, into specific tabs - Using the tabs.executeScript() API...

I don't have a need for the second or third ways here so I'm going to focus on the first way. In this scheme I just need to update manifest.json with a content script and a URL pattern. Here's what that looks like:

manifest.json

"content_scripts": [
    {
        "matches": ["*://*/*"],
        "js": ["block-website.js"]
    }
]

manifest.json - content scripts

The matches key is what specifics the URL pattern. In my case I have a catchall. Here's more info on match patterns.

All that's left to do is read the bed time value, check it against the current time and then block the page if it's past bed time. Simple enough. Here's the code:

block-website.js

function getCurrentHours() {
    let date = new Date();
    return date.getHours();
}

function blockPage(blockTime){
    if(blockTime && blockTime.blockTime && getCurrentHours() >= blockTime.blockTime){
        document.body.innerHTML = "<div> Sorry you can't look at this website it's past bed time! </div>";
    }
}
browser.storage.sync.get("blockTime").then(blockPage);

Chapter 4: Compatibility with Chrome

Everything done so far has been for Firefox. I knew at the start I'd have to do some work to port it over to Chrome, but it's something I should have looked more into before writing code. Lets look at the trouble this got me into.

Obviously if I want to publish this on the Chrome store I have to get it working on Chrome. So I loaded the extension into Chrome and got errors as expected. Lucky for me Mozilla wrote a great article explaining the incompatibilities between FireFox and Chrome: Firefox and Chrome incompatibilities. This was one of the first places I looked to when trying to get things running in Chrome. Here are the changes I had to make:

The browser namespace doesn't exist in Chrome. All the code I wrote using that namespace needed to be changed to chrome. E.g. browser.storage.sync.get... would become chrome.storage.sync.get...
The Web Extensions API is async. Firefox handles this with promises, but Chrome does so with callbacks. All the code that looked like:

// promise based
browser.storage.sync.get('blockTime').then(setBlockTime);

needed to become

// callback based
chrome.storage.sync.get('blockTime', setBlockTime);

I didn't run into this, but it's worth mentioning. There are other small inconsistencies between the APIs. In general they're mostly the same, but it's might be worth developing extensions in tandem to help avoid headaches later down the road. One example of these inconstancies can be seen in the tabs.create method. It takes an object called createProperites, but what properties that object can have differs on the browser.

It would have been better to develop the extension on Chrome and port it to Firefox and heres why:

As a porting aid, the Firefox implementation of WebExtensions supports chrome, using callbacks, as well as browser, using promises.
This means that many Chrome extensions will just work in Firefox without any changes.

This isn't true for all browsers, but it is for Chrome and Firefox. I think Chrome will eventually use browser since that's what the standard being developed specifies, but for now this is what we got. Here's more info on the spec/standard

Once I made these changes to the extension it worked in Chrome. For more information on the differences checkout the Firefox and Chrome incompatibilities article linked above.

Chapter 5: Packaging & Publishing

Alright I've got a web extension that I'll actually use, and will help me get my sleep schedule back in order. So now what? How do I publish it so other people can use it? Lets take a look at how we can publish an extension on Firefox and Chrome.

in a nutshell all publishing requires is packaging your extension and then submitting it to the store.

Packaging your application

I've got my code in a place that I like so the next step is to package the extension. All that's needed is to create a ZIP archive of all the files that make up the extension. I create a ZIP of the following files:

manifest.json
icons/trust-nobody-v2-48.png
popup.html
popup.js
bock-sites.js

Mozilla also has a tool called web-ext-build that can be use for this. I didn't bother looking into it, because creating a ZIP was so easy. Thought it was worth mentioning though. More info on packaging your app and specific directions on how to do it can be found here.

Publishing to Firefox web store (AMO)

Once the extension is packaged it's almost time to submit it. Mozilla has a step by step guide on submitting here. I'll summarize the points in it, because it really just came down to these things for me:

Look over the Add-on Policies and Developer Agreement. If you violate these your extension could be rejected or taken down.
If you don't have an AMO account you'll need to create one.
If you have an account head over to the "add-ons-developer-hub". This is where you can submit the extension.
Follow the flow that AMO has setup for you to submit. From here it's just about filling out a few forms.

Once you submit you'll get an email notifying you of your submission and that it's being reviewed. If your exentsion is accepted then it'll be on the store for other people to download it! I submitted my application on Wednesday and it was accepted Thursday. Less than a day to approve my application. Overall the process was pretty easy. Package your app, create an add-ons account, fill out some forms, submit and wait for approval.

Publish to the Chrome web store

Chromes process is very similar to Mozillas. Just like Mozilla they have a step by step guide on submitting you can follow here. Again, the process isn't to hard so I'll summarize what it came down to for me:

Again the first thing you'll need is a packaged version of your extension
If you don't have a developer account create one.
If you have a developer account then register as a developer with the Chrome web store. It'll cost you \$5 to do so 😭.
Use the Chrome developer dashboard to upload your package.
Finally fill out the necessary information and forms. Chrome requires you have an Icon and screenshot of your extension.

I submitted on Oct 29th, but still haven't heard back. My status says pending review so it might take a while to get done cause of Covid n'all. We'll see how long it takes for them to accept my extension.

Chapter 6: The End Dawg

There it is. An extension from start to finish, and enough information to give you a solid foundation to build your own extensions. I didn't create my whole extension in this article, but I'm working on it! Using what I've built so far has actually helped me avoid staying on the internet past my bed time. Obviously there's more things I want to add, but one thing at a time. If you think having something block your browser after a certain time might be beneficial to you then you can checkout these links for the extension:

Here for Firefox
Here for Chrome - As I mentioned they haven't accepted my submission, but I'll update this page when it's approved.

I'm currently working on adding the other features I described at the beginning of the article, and I'll update the extension as I get to them.

To stay up to date with writings like these go checkout Learning Compuations

I already said there it is, but there it is. A practical guide to web extensions. All you need to do from here is expand on the foundation that you've built in web extension land. Now go build an extension and publish it! Get building and see you next time!

I Learned A Bunch of Shit Installing Arch Linux and You Can To! Pt.1

sasa cocic-banjanac — Wed, 21 Oct 2020 22:41:51 +0000

This article comes to you from my weekly news letter Learning Computations! If you enjoy it consider subscribing.

This week we'll be looking at the process of installing Arch Linux and learning a bunch of stuff along the way. If you don't know what Arch is here's how their about page describes it

Arch Linux is an independently developed, x86-64 general purpose GNU/Linux distribution versatile enough to
suit any role. Development focuses on simplicity, minimalism, and code elegance. Arch is installed as a
minimal base system, configured by the user upon which their own ideal environment is assembled by installing
only what is required or desired for their unique purposes. GUI configuration utilities are not officially
provided, and most system configuration is performed from the shell by editing simple text files.
Arch strives to stay bleeding edge, and typically offers the latest stable versions of most software...

Arch Linux about page

In Arch your responsible for setting up the system to your needs. Anything you want you've got to figure out how to add. This gives you a lot of freedom, but at the same time you've got a lot to figure out (if you don't already know how it works).

I installed Arch and it left me wondering about a lot of things. So I started digging into everything I felt I didn't have a clear picture of. That inspired this weeks edition of Learning Computations where we'll cover some cryptography and partitioning. There's way more I wanted to write about, but to keep this edition somewhat concise we'll be focusing on just these two things. In the future we'll be covering topics like the boot process, file systems, networking and more.

Alright 5 things I want to cover before getting started.

I'll loosely talk about the Arch installation process, but this won't be a step by step guide. We'll be focused on the ideas and learning about them. If your looking for how to install Arch go check out their installation guide it's really good.
I want this edition to be more engaging than in the past. All too often with learning if you don't have something to apply the learning to it's forgotten. Like I just said this isn't a tutorial, but I recommend installing Arch. By doing so it'll give you a way to make practical use of your knowledge from the things you learn in this edition and the ones to come. So if you want to get the most from your learning install Arch, read this edition, read the arch installation guide and figure things out.
You don't have to install Arch linux to learn something from this edition. You can still read it and get the knowledge, but to reiterate putting that knowledge to use might be harder.
Unix is used by 71.2% of all the websites whose operating system we know. - w3techs.com. If your an engineer and you work with/on the web it might not be a bad idea to know a bit about Unix/Linux considering more than 2/3s of websites are running on Unix/Linux.
We won't make it through the full install process in this edition, but I encourage you to try and install the whole thing on your own. We'll probably completely finish it in the next edition or two.

Let's get started

Alright let's take a look at the Arch Linux installation guide. We'll use this to guide our learning as we go along. If you're looking to follow along and install Arch then checkout the installation guide above. Also, Arch has guides per computer e.g. there's a specific installation page for Macs so check those out as well.

Cryptography land

The first thing we're asked to do is acquire an installation image. Cool. Download the image. Next they ask us to verify the image signature if you downloaded it from an HTTP mirror, which I did and for the sake of learning I'll assume you did as well 😁. I downloaded Arch from this mirror. If you look at the files in there you'll notice two. One called md5sums.txt and another called sha1sums.txt. So what is verifying the image signature and what are md5sums.txt and sha1susms.txt for? To understand we need to take a trip to cryptography land. First let's go over what verifying the image signature is. The signature they're talking about here comes from public key cryptography (PKC). Nerds have many names for the same thing sometimes this is also called asymmetric cryptography.

In PKC you have 2 keys. One private and one public. The private one you keep hidden and don't tell anyone. Not even your mom. The public one is public so you can tell anyone on the street your public key. These two keys together are called a key pair.

There are two things you can do with a key pair:
    - You can encrypt some data with the public key. The only way to decrypt that data is with the corresponding private key
    - You can sign some data with the private key. Anyone who knows the corresponding public key can verify the signature, proving which private key produced it.

taken from Everything you should know about certificates and PKI but are too afraid to ask

We've found the answer to "what verifying the image signature is": It's using the public key to verify the signature of our file.

So what about md5sums.txt & sha1sums.txt?

These two files are referencing the SHA1 & MD5 cryptographic hash functions. Let's cover cryptographic hash functions so we're on the same page. The main thing we're worried about here is that the file's (the installation image) integrity hasn't been compromised. This just means no one has maliciously changed the file or it hasn't been messed up on accident somehow. Crypto hash functions let us be almost certain that the file's integrity is intact. How can they almost certainly do that? Good'ol math and know how. I'm not going to explain the math or the know how, but let's go over what an ideal hash function has and how it helps us here.

The ideal cryptographic hash function has the following main properties:

    - it is deterministic, meaning that the same message always results in the same hash
    - it is quick to compute the hash value for any given message
    - it is infeasible to generate a message that yields a given hash value (i.e. to reverse the process that generated the given hash value)
    - it is infeasible to find two different messages with the same hash value
    - a small change to a message should change the hash value so extensively that the new hash value appears uncorrelated with the old hash value

Wikipedia page on cryptographic hash functions

So lets assume we have a hash function with these properties. If we have some file and we put it in our ideal hash function we get out a hash. If for any reason that file changes, maliciously or by accident, and we put it in our hash function again we would get a different hash value.

Alright now we've got a grasp of PKC & crypto hash functions. So how do verifying signatures and crypto hash functions work in practice? SHA1 and MD5 are pretty easy to use. There's already cli tools for them. All you have to do is use those cli tools, look at their outputs and compare them to the values in the files.

There's another cli program called GnuPG that allows you to do all the wonderful things PKC let's you do. You can use it to verify the signature of the installation image. GnuPG does a lot more than just verify signatures, but this isn't an article about GnuPG! If you're interested in learning more about it I'd recommend the user guide on GnuPG to start!

Once you do these checks you can be pretty sure the file you have isn't messed up.

Flashing, Partitions and Stuff

So we've made sure the installation image we downloaded was good. Yay! 😇 the next step is to install the image on a USB. All you need to know here is how to find the USB drive, and to put the image onto it. There's not anything cool I'd like to talk about here and it's pretty simple. If you just follow the wiki it's a pretty straight forward process.

Partitioning Part 1

If you decide you want to dual boot you'll need to partition your disk here. If your not "multi-booting" then you'll do all your partitioning in Arch later. I decided I wanted to dual boot so I partition my hard drive 50/50. I did this by using an app on OS X called disk utility. Don't worry too much about the partition as we'll be deleting it later when we boot into Arch.

Also it's a good idea to backup your data before you partition. There's a small chance for data loss while partitioning. I didn't do this, because I'm a maniac with nothing to lose (and I did this on a laptop I didn't care about), but if your not one of those consider backing up your data.

So what is Partitioning? Partitioning is actually really simple. It's just taking some storage (like a hard disk or SSD), and splitting it into parts. That's it.

How partitions are stored is more complex. Partitions are stored in partition tables. MBR(Master Boot Record) & GPT(Guid Partition Table) are types of partition tables. I'm exclusively going to talk about GPT going forward. Why? GPT is more modern and designed to succeed MBR. In GPT each partition is assigned a unique ID and type (the type is also represented by a different unique id). The ID identifies the partition amongst others and the type labels what kind of data the partition will hold. For example if you're creating a linux root partition you'd want that partition to be of type Root partition (x86-64) - 4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709 (the second part here is the unique id type) if you system is x86-64. You can check out all the types here

To summarize: partitioning is splitting up your disk. Partitions are stored in partition tables. Every partition on your disk has a unique id that identifies it, and a type which is also represented by a different unique id that explains what kind of data the partition stores.

Boot the live environment

I'm deliberately going to leave some things unexplained here, because I'll be talking about the boot process in the next article. Again there isn't anything I'd like to talk about in this section. Your computer should have some way to boot from a USB. Figure it out ya know. Checkout the Arch guide for your computer. I was on a Mac all I had to do was hold down the option key on startup and it gave me different options to boot from.

Once you figure that out Arch should now start up on your Computer. There's more work to be done before you can boot without the USB, but we're on our way! Pat your self on the back, eat a cookie or something you deserve it. Or don't you don't have to. I just patted my self on the back and I can tell you my Ego immediately 10xed. Alright let's keep it movin.

Partitioning Part 2

At this point you should be booted into Arch Linux. Now it's time to properly partition your disk. Your going to need 2 things

A program that'll allow you to partition disks
Which hard drive to partition

When I did this I deleted the partition I made in OS X and created two new partitions. A root partition and a home partition. I already had an existing EFI partition. We'll talk more about this partition next time since it's related to booting (The EFI System Partition only applies to GPT. We aren't talking about BIOS at all). You'll most likely want something similar.

The root and home partitions are straight forward. The root represents the root of your filesystem. This is where all your files will reside. The home partition represents your home directory i.e. /home.

Alright. At this point you should have the Arch installation image written to a USB, and partitioned your disk. That's all I got for this edition. Next time we'll go over the boot process and some networking. See you next week!

What You Should Know About DNS Before Deploying Your Site

sasa cocic-banjanac — Sat, 26 Sep 2020 13:55:36 +0000

If you don't know what DNS is your going to have a hard time getting traffic to your website. If you know what it is, but don't understand how it works your walking in the dark. At a minimum you should know how DNS works at a high level. That being said this article gives you resources to understand DNS in breadth and depth.

DNS

DNS stands for domain name system. It's the thing that turns www.google.com into something that a computer can actually understand. An IP address.

Free

DNS made easy - This is a great intro to DNS. It gives the big picture about what DNS does and how it works. The amount of information in here will probably leave you asking questions about certain things, but that being said I haven't found a more entertaining video about DNS than this one.

How DNS works - Computerphile - This is quite similar to DNS made easy, but it provides a bit more detail on how things work. Hearing the same information in a different way often helps solidify your understanding.

DNS the good parts - This puts the ideas in the previous two resources into practice. This article talks about DNS, but more practically gives you a way to poke at DNS with thedig (domain information grouper) utility.

Paid

DNS & BIND - This is by far the most in depth resource here. This book covers just about everything you could want to know about DNS. If your looking to understand DNS on a deeper level this is where you'll want to go.

Final Thoughts

How much you'll need to know about DNS will vary. For most It'll probably be good enough just to know how it functions at a high level. Others might need to know the nitty gritty of how DNS operates.

If your looking to understand how DNS works at high level check out

DNS made easy
How DNS works - computerphile
DNS the good parts

If your looking for a deep dive on DNS then go read

DNS & BIND

That's all I got for ya. If you enjoyed this article there's more of em at the Learning Computations Archive.

The DS & Algorithm Resources I Used To Land Top Jobs

sasa cocic-banjanac — Fri, 25 Sep 2020 00:34:03 +0000

I thought that my education in college would prepare me for interviewing as a software engineer. I was wrong. It helped don't get me wrong, but it wasn't what companies tested me on in interviews. I got tested on my understanding of data structures and algorithms and showcasing the extent to which I could use them. Below are the resources that helped me truly understand the subject and, helped me land my jobs.

Data Structures + Algorithms

A big part of programming is organizing your data so that you can store and retrieve it effectively. This is what data structures allow us to do. They're the lifeblood of programming. Almost every program written has given thought to how do I store this? How do I retrieve this? This is why at most universities there are whole classes dedicated to data structures, this is why when programmers go to job interviews they ask them about data structures. So much of what a programmer does depends on how they organize their data.

It's hard to talk about data structures without mentioning algorithms, because they're so intertwined. Algorithms In general are instructions on how to do something. Algorithms say things like "this is how you sort data" and "this is how you find the shortest path". Interestingly enough there are many ways to do something, and the way you do it can effect your program dramatically.

Note: going forward I'll refer to Data Structures as DS, and I'll refer to data structures and algorithms as DS+A

Free Resources

Free Code Camp - This video (course? Its 8 hours) is great introductory material if you've not been exposed much to analysis and DS+A. It provides some motivating thoughts to get you thinking, and then goes to the heart of the topic. No non-sense and straight to the point.

MIT - Intro To Algorithms - This is MIT's course for an introduction to algorithms, and it's high quality as you would expect. Unsurprisingly like many university courses online it focuses a lot of time on mathematical aspects that aren't often used in real life. There is still a lot of value in these videos, but if you don't understand the math it'll be hard to grasp all the concepts.

Paid Resources

The Algorithm Design Manual - Steven Skiena is the author of this book and a CS professor known for his algorithmic work. This book is all about DS+A and their analysis, and it's one many students use to study algorithms. Again like most university resources there isn't a shortage of math, but that being said it still has great insights, descriptions, and implementations of DS+A. Oh also Skiena has a free course on YouTube

Leetcode - Leetcode is a website that is filled with DS+A questions (programming challenges). Most of them are for interviews, but the most effective way to understand both DS+A is to use them in the context of problems. They provide solutions and sometimes articles on how to solve the problems. After a while they just feel like puzzles with a dependency on DS+A knowledge and can be quite enjoyable.

Interview Cake - Interview Cake is similar to Leetcode, except that it is more structured. It provides guidance through readings, and then problems to test your knowledge of concepts.

Cracking the coding interview - This book may come across as a book for interviewing (and it is), but it's also a great resource on practical DS+A. Programming interviews are notoriously based in DS+A knowledge and analysis. This book makes approaching big O and DS+A simple. It's on its 6th edition, and they've been able to clearly and succinctly communicated these ideas into digestible chapters. Also, It's filled with exercises and answers to the exercises so you can test yourself. This book helped me get a job, and I'm sure many other developers.

Final Thought

If you want to get a good understanding of DS+A in a way that abstracts most of the rigorous mathematical techniques then stick to these resources

Leetcode
Cracking the coding interview
Interview Cake
Free Code Camp

If you want a more traditional and mathematical understanding of DS+A then check these out

Skiena book
MIT - intro to algorithms

That's it. If you enjoyed this writing there's more coming in my newsletter! To stay updated subscribe here!

OAuth 2.0 Resources You Can Actually Understand

sasa cocic-banjanac — Tue, 15 Sep 2020 18:13:02 +0000

OAuth 2.0

Have you ever tried to use an API that belongs to a large corporation like Spotify, Twitter, or Facebook? If you have you've most certainly run into OAuth 2.0. The first time I saw it like most I was pretty confused, and I didn't really understand it. Eventually I came to grips with it a bit, but never really intuitively understood it. I would just google "why isn't x working" and "this thing failed what does that mean?" until I got things working. It wasn't until I found the resources below that I really understood how things worked.

a quick note: I'll use OAuth and OAuth 2.0 interchangeably. They are different, but here I'm always referring to OAuth 2.0

Free

OAuth 2.0 and OpenID Connect in plain english - This is a video from Okta (an identity and access management company), and they do the best job I've ever seen of explaining OAuth 2.0. Personally this video took me from kinda understanding OAuth to intimately understanding it. Nate (the guy in the video) explains what things where like before OAuth 2.0, what problems it solves, how it works at a high level and detailed level, and finally does some demos to really solidify everything. One thing I really enjoyed about this video is that it talks about OAuths historical context, which isn't usually mentioned. That context provides the information for what OAuth is trying to do and why. I can't recommend this video enough. If you've struggled with OAuth in any way I think this video will help tremendously. It's one of the few resources I've found that actually makes OAuth simple to understand (finally!).
OAuth 2.0 Simplified - Describes its self as "... a guide to building an OAuth 2.0 server. Through high-level overviews ..." I think most people won't need to know how to build an OAuth 2.0 server. That being said, it's still a great resource filled with valuable knowledge. Additionally, sometimes in order to learn things we need to look at them from different angles, and implementation is an angle worthwhile viewing. This resource is separated into 25ish articles. It covers everything from client registration to authorization grant types. If you're not understanding how a particular part of OAuth works then this is a great place to look.
OAuth 2.0 RFC - The RFC (Request for comments) is where OAuth 2.0 is specified. Every detail you could possibly want to know about OAuth 2.0 is in here somewhere. It's a lengthy document so prepare a cup of coffee before you sit down and read the thing.

Final Thoughts

If you want to understand OAuth 2.0 at an intuitive level then I'd recommend these resources.

OAuth 2.0 and OpenID Connect in plain english
Okta oauth website

On top of these resources I'd also suggest getting your hands dirty. Spotify is a great example of an OAuth API you can play with. Additionally, Spotify has published an authorization guide that shows you how OAuth functions with their API.

Finally if you truly want to be an OAuth master then use the above resources and read the RFC in it's entirety

OAuth 2.0 RFC

That's that! If you enjoyed this article then you can stay up to date with my writings at Learning Computations. Every week you'll get an email covering a CS/programming topic that includes resources, info, and a path for learning that particular topic!

The Resources I Wish I Had When I Learned About Databases

sasa cocic-banjanac — Thu, 10 Sep 2020 02:39:44 +0000

When I learned about databases it was all about ORMs and SQL. When I got to the real world those still mattered, but knowing databases intimately became more important. I wish these resources had been brought to my attention earlier in my career to help me build the foundation I needed instead of hopeless and constant stackoverflow googling.

Databases

Databases have been and will be an important topic in computer science and more importantly the real world. They aren't going anywhere, they're fundamental to the survival of most businesses, and they're crucial to understanding how most system works.

Free Resources

History & Content - Like almost everything in engineering databases where not built in a vacuum, and usually built to solve or improve upon a specific set of issues. Knowing the history and context around anything your learning really helps frame your understanding of it. This wiki link is supplementary.

Stanford database course - This is what I believe to be the quintessential undergrad database course. This course is all about how to use databases. There are details about how things work, but it is more breadth than depth. One thing that kinda sucks is that the resources for the course don't exist anymore. When I took the course it was available on Stanford lagunita, but it doesn't seem to be there any longer. The videos are still there which are the main source of information.

CMU database course - In this course you actually build a database. There is no better way to understand a system than to build it. The professor is great and cares deeply about databases. A quote from his Fall 2018 class - "I really only care about 2 things in my life. The first one is my wife. The second one is databases"

How does a relational database work - To quote the article "the aim of this article is NOT to understand how to use a database". As the name suggests it's about the internals and structure of how a database works.

Use the index luke - to quote the site "A site explaining SQL indexing to developers—no crap about administration." This obviously doesn't cover databases as a whole, but provides a good look at the power of indexes.

Paid Resources

Designing Data Intensive applications - This book looks at a lot more than just databases, but it does spend a fair amount of time explaining and helping you understand them. Also, it does a great job for framing why databases are integral to data intensive applications. You’ll see this book mentioned around hacker news, programming subreddits, and everywhere programmers hangout. One negative is that it’s not a great beginner resource, but it’s great if you already have a foothold in databases.

Final thoughts

There is a lot (and I mean a lot) of technical content in the resources I just listed. So I’ll provide some guidance on what you should do based on your goals.

If you’re serious about getting an overview, and understanding how to use a database then checkout the Stanford course.

If you already know how to use a database, and are trying to get to the next level of understanding then checkout one of the below resources.

The CMU database course (again it’s a course so it’s a large resource)
Designing Data Intensive applications
How does a relational database work
Use the index Luke

An aside for self learners

One thing I often hear self learners say is "I don't know this I didn't get a cs degree", or "It's harder for me to understand this because I didn't get a cs degree". I'd like to let you know I got a CS degree and I'm still confused as hell! This stuff isn't easy! How do you build the right foundation of knowledge? On any topic there's an infinity of resources, and it can be hard to find the right direction to go.

Every week I write about CS and programming concepts and give readers top resources to learn the things that were covered. I try and highlight a few things.

Compare what college students learn, and what a self learner might have to do to achieve parity.
Expose readers to new concepts and ideas that they might not otherwise stumble upon. The world of programming and CS is vast, and there's actually quite a bit of cool stuff in there.

if your interested in staying up to date with the writings feel free to subscribe: https://buttondown.email/LearningComputations