DEV Community: Benjamin

Monorepos and Workspaces - an introduction

Benjamin — Tue, 04 Jan 2022 01:56:27 +0000

tldr;
When making a monorepo, it generally consist of workspaces (can use npm, yarnor pnpm), but additional tooling exists like lerna and turborepo (and lots of others too) to help with easing the problems of managing lots of interlinked packages

see some real world examples

What is a monorepo?

There are lots of other posts that dive into monorepos and their benefits and recommend reading them before diving into the tooling of one. Here is my quick tldr;

A monorepo is a single repository, that contains multiple loosely coupled packages within it. It differs from a monolith in that it usually has a focus on distributing individual packages, and saves time by allowing multiple changes that might usually need to be made across multiple repository's in a single repository and PR.

Challenges

It's important to understand the challenges that monorepos present before diving into tooling and weighing up the best suite of tools for your use case.

1 - Versioning

The largest challenge people generally face with monorepos managing versions and deployments of packages. One scenario might be that you make a change in one component, bump the version and then want that version to propagate to all the other packages that might depend on it. This is of particular concern for breaking changes as you want to make sure you're not going to break anything for the end users.

2 - Deployment and CI

Another challenge can be how to manage deployments, CI. Though a lot of CI tools have functionality to assist with this (e.g. Github Actions, CircleCI, you may need to think about whether you want to run your CI suite against a single package, or the whole repository, and if any automation might differ depending on the change.

3 - PR's

As monorepos can get quite large, figuring out who is best suited for reviews and/or approvals can get a bit tricky. If you use github, adding a CODEOWNERS can help with automating this

Tooling

The below tooling is a list of the most popular tools used for monorepos and managing workspaces. There is not necessarily a magic combination, but rather what works for you and your existing tooling. Not all of these tools are exclusive either, for example, yarn workspaces and lerna can be used together to create a more rich experience.

Workspaces

Most monorepos in the javascript ecosystem are built upon workspaces, a feature that is now readily available in most of the popular package managers. There isn't necessarily one that is better than the rest and generally follow a somewhat similar syntax pattern, but there may be one that your more comfortable with.

Additional tooling

These are optional dependencies that are used to help with deployments, caching and linking packages. There is not necessarily a magic combination, but rather what works for you and your existing tooling.

Real world use cases

1 - Component library

One common use of monorepos is a design library. It allows for multiple publishable components and allows for users to import only what they need if desired, and encourages best practices for designing and creating your components by forcing them to be independent entities.

Example 1 - Chakra UI: https://github.com/chakra-ui/chakra-ui
Example 2 - Material UI: https://github.com/mui-org/material-ui

2 - Optional configurations and extras

Another common use case is that a product might have different available options that are made available to the end user, but don't necessarily fit within the core product. This could be using different build tools / versions, different variations of linting or packages that might only cater for specific use cases.

Example 1 - Storybook: https://github.com/storybookjs/storybook
Example 2 - OpenWC: https://github.com/open-wc/open-wc
Example 3 - Lit: https://github.com/lit/lit.dev

More resources

[list] Awesome monorepo

Curated list of monorepo resources

[blog] 11 Great Tools for a Monorepo in 2021

Probably the best tools to develop and build your monorepo.

[blog] What is a monorepo

Monorepos can foster rapid development workflows. In this post, we’ll examine if they are the right fit for you and your company.

[blog] **Trying out npm and yarn workspaces

A practical walkthrough of npm vs yarn workspaces

Integrating Prism into a create-react-app project

Benjamin — Sun, 19 Sep 2021 17:19:33 +0000

Prism + create-react-app

Prism doesn't have a setup akin to much of the modern web, which can create some friction when integrating with modern tooling for ever changing frameworks. There are a few methods of installing prismjs, but this guide follows their recommended approach. Assuming a project already exists that was created with create-react-app, these steps should get you up and running with Prism quickly.

Download and Installation

Navigate to https://prismjs.com/
Click Download and select the theme you want, languages and plugins you would like to support
Download JS and CSS
Copy the JS and CSS to the public folder
In your public/index.html, add this tag:

<!-- index.html -->
<script async src="%PUBLIC_URL%/prism.js" />
<link rel="stylesheet" href="/prism.css" />

Usage with dyanmic content

You will need to do one of the below if you are changing content that needs syntax highlighting. For example, wisiwyg editor. Add one of the below code blocks where you're update your data.

Option 1 - Reload all highlight on value change.

const [value, setValue] = useState("" /* your dynamic code block */);

useEffect(() => {
  window.Prism?.highlightAll();
}, [value]);

return (
  <pre>
    <code className={`language-json`}>{value}</code>
  </pre>
);

Option 2 - Reload single code block (recommended)

This approach is recommended if you have more than 1 code block, as it will save you unnessesary reloads.

const [value, setValue] = useState("" /* your dynamic code block */);
const ref = useRef(null);

useEffect(() => {
  if (ref.current) {
    window.Prism?.highlightElement(ref.current);
  }
}, [value, ref.current]);

return (
  <pre>
    <code ref={codeEl} className={`language-json`}>
      {value}
    </code>
  </pre>
);

The Aternative

If the above approach doesn't suit your application or team, there is an npm package you can use, however you will need to also use the supplied babel plugin. When using a no config solution like create-react-app, this becomes difficult and you may will may need to use craco or eject your configuration.

// SomeComponent.jsx
import Prism from "prismjs";

// code here

Prism.highlightAll();

// code here

For the babel plugin see: https://github.com/mAAdhaTTah/babel-plugin-prismjs

Full Example

To see a full working example, PrismJS has been implemented in the following web app:
https://github.com/bmpickford/dynamoconverter

ShouldIKeepWatching - Digital Ocean Hackathon

Benjamin — Fri, 08 Jan 2021 23:38:58 +0000

What I built

A web app to help you decide whether or not to invest any more time in a TV show you're watching. No one wants to waste their time on bad TV!

Category Submission: Random Roulette

App Link

https://shouldikeepwatching-kpcfk.ondigitalocean.app/

Screenshots

Link to Source Code

https://github.com/bmpickford/shouldikeepwatching

Permissive License

MIT

Background

With great new TV shows coming out every other week, it's hard to know what's actually worth watching, especially if you're going to be investing your time in lots of seasons. I'm always hesitant to start a TV show that I know is going to go downhill (e.g. Heroes, Dexter) so I want to know upfront how bad it gets. This heatmap I created for the TV shows, gives a quick visual representation of a TV show so I know what I'm in for.

How I built it

I used NextJS (for the first time) to try and explore the realm of SSR using React. It was a suprisingly nice developer experience, and a much smaller learning curve than expected. Will definitely look to NextJS for future projects that could benefit from SSR.

Firebase Auth with React Hooks in Typescript

Benjamin — Thu, 08 Oct 2020 05:05:50 +0000

Overview

Firebase is a great provider for hooking up easy and simple authentication to a new or existing project, and can easily be integrated with other features of Google Cloud Platform. Adding it to your application can be simple, but a lot of tutorials don't cater for larger scale applications, and how it could be implemented cleanly and with quality assurance in mind.

Assumed Knowledge

Firebase
Typescript
React
React Hooks (specifically useContext)

Setting Up

GCP and Firebase

To start, you'll need to setup a GCP account and login to the Firebase console at https://console.firebase.google.com/. You'll then need to setup a project for us to use in this run through. For this walkthrough, I've named my project "HooksAuth"

I'm then going to enable email/password and Google Authentication by navigating to the "Authentication" page from the left menu and enabling them there

React

To setup your react project from scratch, you can use create-react-app by running the following command in your cli: npx create-react-app hooksAuth --template typescript

The next (optional) step is to remove all the boilerpate content from your React App. e.g. react images, manifest, App.tsx content etc. This helps me to start from a cleaner slate and not have create-react-app content in my site

Setting up the hooks

We are going to abstract our firebase app away through useContext so it's state can be shared through the app, and any abstractions and access can easily be updated and refactored.

Create FirebaseContext.ts
Add your config to this file (this can be retrieved by registering a web app in the firebase console), and will look something like:

import firebase from 'firebase';

const firebaseConfig = {
    apiKey: "KEY",
    authDomain: "somedomain.firebaseapp.com",
    databaseURL: "https://somedomain.firebaseio.com",
    projectId: "hooksauth-ID_HERE",
    storageBucket: "hooksauth-ID_HERE.appspot.com",
    messagingSenderId: "SENDER_ID",
    appId: "APP_ID"
  };
  // Initialize Firebase
  firebase.initializeApp(firebaseConfig);

Now we want to secure these values using .env (before we accidentally commit anything). To do this, create a file called .env.local in the root of the project, and add this file to .gitignore. In that file, you'll want to define your ID's in something similar to this format:

REACT_APP_API_KEY=KEY
REACT_APP_AUTH_DOMAIN=somedomain.firebaseapp.com
REACT_APP_DB_URL=https://somedomain.firebaseio.com
REACT_APP_PROJECT_ID=hooksauth-ID_HERE
REACT_APP_STORAGE_BUCKET=hooksauth-ID_HERE.appspot.com
REACT_APP_MESSAGING_SENDER_ID=SENDER_ID
REACT_APP_APP_ID=APP_ID

and update your config to be:

const firebaseConfig = {
    apiKey: process.env.REACT_APP_API_KEY,
    authDomain: process.env.REACT_APP_AUTH_DOMAIN,
    databaseURL: process.env.REACT_APP_DB_URL,
    projectId: process.env.REACT_APP_PROJECT_ID,
    storageBucket: process.env.REACT_APP_STORAGE_BUCKET,
    messagingSenderId: process.env.REACT_APP_MESSAGING_SENDER_ID,
    appId: process.env.REACT_APP_APP_ID,
};

Move the firebase initialisation. To stop the firebase initialisation happening multiple times, we want to wrap it in an if statement to make it look like:

// Initialize Firebase
if (!firebase.apps.length) {
    firebase.initializeApp(firebaseConfig);
    firebase.analytics();
}

Now we want to actually do some React stuff. We want to create a context for us to use in the rest of our app. So using react hooks, we will create a context that contains the auth providers we will use, and a way to access our firebase object:

export interface IFirebaseContext {
    firebase: firebase.app.App,
    authProviders: string[]
}

export const FirebaseContext = React.createContext({} as IFirebaseContext)

And then our provider to wrap the app:

export const FirebaseProvider = ({ children }: any) => {
    return (
        <div>
            <FirebaseContext.Provider value={{ 'firebase': firebase.app(), authProviders } as IFirebaseContext}>
                {children}
            </FirebaseContext.Provider>
        </div>
    );
}

Now to wrap the rest of our app. So in App.tsx, we want to wrap the root div in <FirebaseProvider>. And now in the rest of our App, if we want to access our auth object, we would simply call const firebaseContext = useContext(FirebaseContext); and access it there. This can be combined with the firebase-ui package to get your authentication up and running in no time

For an example of usage with the react-firebaseui package and saving user data see my github repo: https://github.com/bmpickford/firebase-auth-example

Open to any improvements so feel free to submit a PR or an issue

Practical Beginners Guide to Problems with JS async/await

Benjamin — Mon, 30 Mar 2020 22:44:15 +0000

What is it

I don't want to go too in depth into what async/await is, and how it came about as there are many other good blog posts about it, but it is basically a syntax sugar around Promises, and gives your code the ability to be asynchronous

How to use it

The async/await syntax has given us the ability to have Promises in a more readable and cleaner way then the native Promise syntax. The basic usage of it, looks something like:

const GetSomeResource = async () => {
    return fetch('https://someurl.com');
}

const DoStuff = async () => {
    const result = await GetSomeResource();
    ...
}

This has the obvious problem of not handling errors, but there are cases where maybe we don't want to.

Problems

There are a few gotchas with async/await, especially if your application lacks the linting around it. I have seen both Senior and Junior devs fall into the simple trap of forgetting an await somewhere, which can make debugging and testing quite unreliable and confusing. So what happens when someone forgets the simple await syntax? e.g.

const AddSomeResource = async () => {
    await fetch('https://someurl.com/user', { Method: 'POST', body: ... });
}

const GetSomeResource = async () => {
    return fetch('https://someurl.com');
}

const DoStuff = async () => {
    AddSomeResource();
    const result = await GetSomeResource();
    ...
}

What would happen here is that the AddSomeResource function would run in the background while other tasks happened. So if in this example, anything relied on that function posting data, there's a chance that it wouldn't have been finished executing, and could cause intermittent errors in your application.
There is another case however where you don't rely on that resource elsewhere in the same scope, and everything seems to be working fine when it gets tested.

However once the external service is down and starts timing out, because the error hasn't been caught or passed up the chain to be dealt with elsewhere it starts logging errors while your other code is running. But the chain of functions associated with that call still seem to be working fine, until something gets called that requires that resource somewhere else in the application. This can send you down a bit of a rabbi hole and waste possibly several hours trying to debug what's actually going on

Forgetting await can be easier than it seems, especially when not using linting and some kind of typing framework such as Typescript. e.g.

const DoSomeStuffAndReturnBool = async () => {
    const getNum = await GetSomeNumAsync();
    if (getNum === 0)
        return true;
    return false;
}

const DoStuff = async () => {
    const isTrue = DoSomeStuffAndReturnBool();
    ...
}

As you can see, especially to someone starting out in JS, it could be easy to not think about the async keyword and just see it returns a bool. But in the case of async, it always returns the result as a Promise

Thinking an await, awaits all the way down

I recently had a discussion with another developer who thought that the await keyword, would await all the child functions too, even the ones without await keywords. e.g.

const AddSomeResource = async () => {
    await fetch('https://someurl.com/user', { Method: 'POST', body: ... });
}

const DoStuff = async () => {
    AddSomeResource();
    ...
}

const DoParentStuff = async () => {
    // This does not automatically await the AddSomeResource call in DoStuff()
    await DoStuff();
}

Conclusions

Add some linting around async and Promises so you don't forget to await, otherwise you could spend too much time debugging your application or getting inconsistent results

Basic Go HTTP server with Postgres

Benjamin — Fri, 16 Aug 2019 11:44:23 +0000

I've found myself creating simple Golang microservices using the same basic design to start out. In this post I'll go through the steps from scratch to help create a simple starting point for a Go microservice or API with the ability to create a customer

Throughout this brief tutorial, I have purposely tried to keep it as minimal as possible to get to know the native packages (net/http, encoding/json)

Basic HTTP server

So presuming you have to go environment all setup, let's start by creating a main.go file in our new project with a basic http server

package main

import (
    "log"
    "net/http"
)

const port = "8080"

func main() {
    log.Fatal(http.ListenAndServe(":8080", nil))
}

From here, if you run go run . you shouldn't get any errors and should be a good indicator your go setup is correct

So now we have a basic server, it's time to setup a handler to check our configurations are all good. Here we will create a simple response for GET /. To do this, add this handler function to main.go:

func handleHomePage(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, the site is running :)")
}

And then this to your main() function:

http.HandleFunc("/", homePage)

Your main.go file should now look like:

package main

import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", handleHomePage)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

func handleHomePage(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, the site is running :)")
}

Now to test this works, hit the server with curl using curl -X GET 0.0.0.0:8080/ and your response should be Hello, the site is running :)

Postgres setup

So now we have a basic HTTP server, it's time to add our postgres DB. Assuming you have postgres already installed, lets add a Database and a table. To do this, open postgres in the command line and execute:

CREATE DATABASE go_project;
CREATE TABLE customer (id SERIAL, username VARCHAR NOT NULL);

Go/Postgres connection

To setup our connection, we will be using the pq package for golang (use go get github.com/lib/pq to download the package). We will not create a new file called postgres.go, that will contain a singleton that will be used in our subsequent files to access the database.

package main

import (
    "database/sql"
    "log"

    _ "github.com/lib/pq"
)

var db *sql.DB

func GetDB() *sql.DB {
    var err error

    if db == nil {
        connStr := "user=postgres dbname=go_project"
        db, err = sql.Open("postgres", connStr)
        if err != nil {
            panic(err)
        }
    }

    return db
}

What we are doing here, is creating a connection using the name of our database and a user for our postgres instance. This might change depending on your setup, and may need some authentication

Finishing our handlers

Now it's time to expand our application to include some real logic. To start, we have to create our Customer struct. Firstly though, we have to create a new file for our handlers. I have created one called handler.go but you can name it anything you like. In this file, add the package name and our new struct as follows:

package main

type Customer struct {
    ID       int
    Username string
}

Now it's time to add our handler. For this, we need to create a decoder that will decode the body of our request (which will be JSON format) into our struct. This can be done using the following snippet:

func handleCustomerPost(w http.ResponseWriter, r *http.Request) {
    var customer Customer

    decoder := json.NewDecoder(r.Body)
    err := decoder.Decode(&customer)
    if err != nil {
        panic(err)
    }
    w.WriteHeader(201)
}

Now we have the ability to decode a request body into a Customer model, it's time to add our handler to main.go as we did with our root response. This time though, we have to filter the POST request because the native http package doesn't have the functionality for that. If you add gorilla mux, this can be handled for you, but for this case I want to keep it as minimal as possible. So we need to add another handler in main, as follows:

func handleCustomerRequest(w http.ResponseWriter, r *http.Request) {
    if r.Method == http.MethodPost {
        handleCustomerRequest(w, r)
    }
    w.WriteHeader(201)
}

and a call to it in the main() function as such:

http.HandleFunc("/customer", handleCustomerRequest)

Linking it all together

Now we need to create our query for saving a customer. To do this, we need to create a new file (i called mine database.go), and add the following function that takes in a username

package main

func SaveCustomer(username string) error {
    db := GetDB()
    _, err := db.Exec("INSERT INTO customer (username) VALUES($1)", username)
    return err
}

And now in handler.go, call this function using the customers username as follows:

err = SaveCustomer(customer.Username)
if err != nil {
    panic(err)
}

Now it's time to test our functionality so far. So start up your go service using go run . again. Now if you hit your endpoint, you should get a 201, and a customer with an ID and username in your database

Conclusion

I hope you have learnt something about creating a basic http server in golang. If you want to see the full codebase, see https://github.com/bmpickford/goHttpServer

Rewriting your (git) history

Benjamin — Fri, 02 Aug 2019 04:01:25 +0000

So you probably know a lot of basics of git, but if you have been using it for a while you have probably encountered a situation where you want to change a commit that's already been pushed. Some argue this is bad practice, and others say it's bad to have an unclean commit history. Although I probably lean more towards the latter, this post is simply about the tools you can use if you want to change some of your already pushed commits

1. Change most recent commit message

git commit --amend

If you have been using git for a while you probably already know about this one. It's generally used to change your most recent commits message if you found a type or wanted to change your commit message layout. It can also be used to changed the author by appending --reset-author to the amend

The interactive rebase

For these examples, I am going to be using 3 commits I have created (see below) and all of these examples first involve the interactive rebase command:

git rebase -i HEAD~3

This will lead you to a terminal output that looks something like this:

pick 74c349f [1] Dummy commit 1
pick f490852 [2] Dummy commit 2
pick 717eaa6 [3] Dummy commit 3

# Rebase b8d17a8..717eaa6 onto b8d17a8 (3 commands)
#
# Commands:
# p, pick <commit> = use commit
# r, reword <commit> = use commit, but edit the commit message
# e, edit <commit> = use commit, but stop for amending
# s, squash <commit> = use commit, but meld into previous commit
# f, fixup <commit> = like "squash", but discard this commit's log message
# x, exec <command> = run command (the rest of the line) using shell
# b, break = stop here (continue rebase later with 'git rebase --continue')
# d, drop <commit> = remove commit
# l, label <label> = label current HEAD with a name
# t, reset <label> = reset HEAD to a label
# m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
# .       create a merge commit using the original merge commit's
# .       message (or the oneline, if no original merge commit was
# .       specified). Use -c <commit> to reword the commit message.

I will use this as the basis for the rest of the examples

2. Squashing commits

Squashing commits is useful if you have some related commits that could be merged into one larger commit, or in some cases, used to make code reviews easier

To squash the commits simply replace the word pick with s for commits 2 and 3 to merge them both into commit 1.
e.g.:

pick 74c349f [1] Dummy commit 1
s f490852 [2] Dummy commit 2
s 717eaa6 [3] Dummy commit 3

This will lead you to a new commit where you can edit/merge the commit message after you save and exit the commit message. After saving your commit message you should be able to git log and see your changes all in one commit

3. Changing an old commit message (reword)

If I wanted to change the message of commit 3, i would do:

pick 74c349f [1] Dummy commit 1
r f490852 [2] Dummy commit 2
pick 717eaa6 [3] Dummy commit 3

Then continue on like a normal git commit --amend

4. Dropping a commit

If you are dropping a commit, be extra careful about the contents and consider squashing instead. However if you really want to delete a commit, just use the drop command. e.g.

pick 74c349f [1] Dummy commit 1
d f490852 [2] Dummy commit 2
pick 717eaa6 [3] Dummy commit 3

This will delete [2] Dummy commit 2 from your commit history and the changes along with it

5. Editing a commit

If you want to add or remove a file/change from a previous commit, you can use the edit functionality. For example, if you added a file but it should actually belong in the previous commit, you would do:

pick 74c349f [1] Dummy commit 1
e f490852 [2] Dummy commit 2
pick 717eaa6 [3] Dummy commit 3

Then you can add/remove files as you wish. Once you are done, then use the git commit --amend command, followed by a git rebase --continue

This can also be used to split a commit in two. As an example, if you follow the above edit step, and want to add a new commit before it you would do the above and then:

git reset HEAD^

From here to you add the files you would like, and create new commits. Then when you are done, execute the git rebase --continue command

Thank you for reading, and I hope this clears up some of the interactive rebase functionality