DEV Community: Ahmad

Running both nodejs and bun apps in turborepo

Ahmad — Sun, 25 Jun 2023 23:31:53 +0000

So finally today, after a little bit of tinkering about, I figured out how to use a bun-powered app, simultaneously with my other nodejs apps in a turborepo.

And so here I am, writing this article so that someone else who is seeking a solution on the web for this exact kind of problem can benefit from this article. Or at least learn something from it.

So without falling into anymore tangents, let's get down to how I put it together.

Quick overview of the tech involved:

Turborepo
Pnpm as the package manager
Bun
- HonoJS
NodeJS
- Fastify
- NextJS
other...

I'll setup a fresh turborepo with the following command

npx create-turbo@latest
# where : dual-runtimes-turborepo
# package manager : pnpm

At the root, we get the current setup

ROOT_DIR:

apps
 - apps/docs
 - apps/web
packages
 - packages/eslint-config-custom
 - packages/tsconfig
 - packages/ui
package.json
pnpm-lock.json
pnpm-workspaces.yaml
turbo.json
... other files

Run it with pnpm run dev to ensure it works properly.

Some cleanup

I prefixed all the local package names with "@local101/", to make sure my package names never conflict with any external npm packages
I removed the eslint-config-custom, to simplify the example, and because I don't use it anyways.

Core config required to make it all work

Open up the root package.json and add the following to it

{
  "name": "dual-runtimes-monorepo",
  "private": true,
  "workspaces": [
    // example app/package, for the example setup,
    // And don't copy/paste comments into JSON
    "apps/aserver",
    "packages/logic"
  ],
  "scripts": {
    "build": "turbo run build",
    "dev": "turbo run dev",
    "lint": "turbo run lint",
    "format": "prettier --write \"**/*.{ts,tsx,md}\""
  },
  "devDependencies": {
    "@turbo/gen": "^1.9.7",
    "eslint": "^7.32.0",
    "prettier": "^2.5.1",
    "turbo": "latest"
  },
  "packageManager": "pnpm@7.15.0",
  "volta": {
    "node": "20.3.1"
  }
}

Here I added a couple of entries. Ignoring the volta config(which is a Toolchain manager), the workspaces part is the key component required for bun to function. Here you specify the path to the packages and apps which you require for bun to pickup.

This is a little bit of downside for yarm and pnpm users like me, as bun follows the npm spec, and I'm using pnpm workspaces to manage for example. I just had to do their respective config, which is just a handful of more lines of code.

Important Note: This may not be required BUT if you run into a bun linking error, I have a helper script (link-packages-to-bun.sh) in the root of the monorepo to make it easier to register your packages to bun's repository.

#!/bin/bash

# iterate over "packages" directory, cd into each package and run "bun link" for each package and then cd out of the path
for package in packages/*; do
 echo "[...] Linking $package to bun"
 # $package is "packages/pkgdir"
 cd $package
 # get the "pkgdir" part from the string
 package_name=$(echo $package | cut -d'/' -f2)
 # ensure that this matches the "name" in each package.json in your local packages.
 # this only works if your package name matches the dir name ex. "ui" for dir "ui" etc.
 bun link "@local101/$package_name"
 cd ../..
 echo "[+] Done linking $package to bun"
done

So before doing pnpm i(don't run until the next section is done), run ./link-packages-to-bun.sh to link packages

For now, let's move onto setting up our aserver app

Bun app setup - Hono

Lately, I've been checking out different bun frameworks, the latest one being hono. So I just chose it for this example.

Now to setup the app, I did the following:

rm -rf apps/docs packages/eslint-config-custom
# ^ optional command, I did it to reduce the focus scope

cd apps

npm create hono@latest aserver
# select bun and wait for it to complete

cd ..

Then I replaced the index.ts of our new app with the following

// apps/aserver/src/index.ts

import { uselessAddFunctionForTheSakeOfExample } from "@local101/logic";
import { Hono } from "hono";

const hono = new Hono({});

hono.get("/", async (c) => {
  const ans = uselessAddFunctionForTheSakeOfExample(2, 2);
  return c.json({
    message: `Hey, did you know that 2 + 2 = ${ans}. I know right?`,
  });
});
hono.get("/ping", async (c) => c.json({ message: "pong" }));

export default {
  port: 9001,
  fetch: hono.fetch,
};

And now finally to ensure our bun dependencies are properly installed, modify the package.json of the newly setup app as follows:

{
  "name": "@local101/aserver",
  "version": "0.0.0",
  "private": true,
  "description": "Well, it's a... server",
  "scripts": {
    "postinstall": "bun install",
    "dev": "bun run --hot src/index.ts",
    "start": "bun run src/index.ts"
  },
  "dependencies": {
    "@local101/logic": "workspace:*",
    "hono": "^3.2.6"
  },
  "devDependencies": {
    "bun-types": "^0.6.2"
  }
}

The postinstall and start scripts are the only main requirements, but I put in a @local101/logic package for demonstration purposes.

And with that the core setup is complete!

Test running it all

Now from the root of the monorepo, run:

pnpm i
# you should see your bun's post install script running here

pnpm run dev

And you should get a:

Now testing out our bun application, if you followed this tutorial, you should get:

Now try running pnpm build and then pnpm start(if setup in the root turbo.json+package.json), it should all should work as expected. And so that concludes it all!

I hope you got something out of this article... if not then... go do something meaningful... like touch some grass.

Anyways, Click Here for the link to view the source code of the monorepo I used as the example in this article.

If you face any issues, then let me know and I'll try to help out.

I'll leave with a final note: If you're going to use bleeding edge technology, then bleed responsibly.

Friction-less dependency management with Python

Ahmad — Tue, 22 Mar 2022 04:46:18 +0000

Now... hear me out this combination might seem a bit weird to many of you but just have a read through to see if I can convince you that it's actually useful :)

So I got into an issue with managing dependencies a few weeks ago and thought of writing this quick little article on what I've been using and think is the best way to manage dependencies for your Python projects.

If you have no idea of what environments are or don't have familiarity with them, then the next section will give you an overview. Else you can just skip ahead

Environments and their reason to exist

When working in a project with python, you have to install many packages in order to complete what you are making, but if you are not careful then those packages can pile up in your python installation; which is not necessarily bad but when talking in the context of deploying to production, you will have to generate a requirements file so your code can run there. It's mostly done with a similar command:

pip freeze > requirements.txt

which dumps all your installed packages in the requirements.txt file. But if you take a look in that file, it will include not only the packages you installed while working but every single one which you have installed in the past. And it's basically a pain to deal with.

To solve this, environments are used in Python. Which simply putting it are:

An Isolated container/place for having a separate dependency tree for each project that you are working on.

The benefit of using such a tool is that you don't have to work with all those unnecessary dependencies which can build up overtime if you keep working with python in a single environment, you can already imagine the difficulty in the management.

Hope that clears up whatever question you had. If you still have question about environments in python & in general, then Google and YouTube are your best friends :)

Now with that aside, let's dive into what we actually are here for

Understanding

Now, both conda & poetry have a very similar reason to exist, to help manage environments. But they both work a bit differently. Now the technique can be used without conda completely, but the reason I include it is because of conda's capability to allow you to use whichever python's version you want to in an environment. And it's done with a simple command like:

conda create -n envname python=3.10

With this I can have like 100 environments, each with a different python version and it's own set of dependencies. And my system installed python can stay clean and simple like a fresh installation.

Now, conda is in of itself is good enough for a lot of people as it has it's own package manager and quite a few handy tools (inclined more towards Data Scientists), but the reason I wrote this article is because I use python a lot in development, and I want to keep my production environment as minimal as possible and it's simply not possible to have separate dev & prod dependency sets ( like in NodeJS for example, if you've worked with it ) with a simple requirements.txt without some nifty hacks

Which is where poetry comes in. Poetry is responsible for managing the packages. It's somewhat new (v1.1 as of writing this article) and is being updated relatively frequently. Poetry has a really cool management system which is quite similar to NodeJS, where you can manage normal and dev. dependencies separately.

The upside of this is when you let's say: package your application with poetry for use in a production environment; it won't matter even if you had like a 1000 packages installed with pip in your conda environment, you'll only be passing ahead the packages that are marked in the pyproject.toml file manged by poetry, which already reduced most of the pain with having a clean final requirements file. On top of that you can also exclude out the kinds of dependencies like linters and formatters (black I'm looking at you) which are not used in a production environment. And that's basically the gist of it, if you're sold on the idea and want to give it a try then keep reading on other wise this is all the wisdom I wanted to share for today and hopefully you got something out of that ~

A Quick Setup

If you haven't already setup conda and poetry, then for conda you should first go and install it from here

After downloading, if on Windows just run the executable, if on mac it's pretty similar but as for you linux chads out there, you should know what you're doing, but either it's just 2 commands to set it up after the install:

chmod +x <your anaconda installation filename here>.sh
/bin/bash <your anaconda installation filename here>.sh

and just follow their steps and it'll set it up for you.

Now restart/open your terminal and typing in conda will show a similar output

usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

Options:

positional arguments:
  command

... more output

by default conda puts you in a base environment which you can optionally disable it if you don't like it by typing in:

conda config --set auto_activate_base false

Now setup an environment with the command

conda create -n my_awesome_project python=3.10

Then activate it with conda activate my_awesome_project

Now there are many ways to setup poetry, but the quickest way is to just run

pip install poetry

And just like that you're ready to start working on whatever you're wanting to with minimal hassle :^]

For a simple FastAPI server setup, create a folder and run poetry init and follow the prompt. It'll generate the pyproject.toml file. Now type in

poetry add fastapi

following right after with everyone's faviourite formatter black

poetry add black -D ~ you can skip this if you want to, I'm just showing how you're gonna separate your prod/dev dependencies. and you'll endup with

[tool.poetry]
name = "my awesome project"
version = "0.1.0"
description = ""
authors = ["username <email@abc.xyz>"]

[tool.poetry.dependencies]
python = "^3.10"
fastapi = "^0.75.0"
uvicorn = "^0.17.6"

[tool.poetry.dev-dependencies]
black = "^22.1.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Now you're pretty much set and ready to go 👍

Conclusion

So, hopefully that convinced you that why this setup is useful over the environment management setups.

Hope you got something out of reading the article, and if so then do leave some feedback as this is my first article and I'm not really used to doing this :^]