Have you ever pushed a change to a project, only to find out later that it broke something else? Or spent hours setting up your development environment, only to find that your teammate has a different setup? These issues are common in growing software projects, and thankfully, there are tools that help tackle both.
We can use Continuous Integration and Development Containers to help us deal with these. I'll talk about both of these in this blog post, and my experience implementing them into my own project Codeshift:
uday-rana
/
codeshift
A command-line tool that translates source code files into a chosen programming language.
codeshift
Codeshift is a command-line tool to translate and transform source code files between programming languages.
Features
- Select output language to convert source code into
- Support for multiple input files
- Output results to a file or stream directly to
stdout - Customize model and provider selection for optimal performance
- Supports leading AI providers
Requirements
- Node.js (Requires Node.js 20.17.0+)
- An API key from any of the following providers:
- OpenAI
- OpenRouter
- Groq
- any other AI provider compatible with OpenAI's chat completions API endpoint
Installation
-
Clone the repository with Git:
git clone https://github.com/uday-rana/codeshift.git
- Alternatively, download the repository as a .zip from the GitHub page and extract it
-
In the repository's root directory (where
package.jsonis located), runnpm install:cd codeshift/ npm install -
To be able to run the program without prefixing
node, runnpm install -g .ornpm linkwithin the project directory:npm install -g
…
Continuous Integration
Continuous Integration is the practice of integrating source code changes frequently and automatically running checks against all changes to maintain code quality and ensure nothing accidentally breaks.
There are many providers for Continuous Integration, like GitHub Actions, CircleCI, Travis CI, or Jenkins. For the purposes of this blog post, since my project's source code is hosted on GitHub, we'll use GitHub Actions as an example.
GitHub Actions allows us to spin up a machine and run tasks. To implement CI with GitHub Actions, we need to define a series of steps that run tools that ensure our code meets our standards. Chances are you're already doing this manually with tools that check for errors and check code formatting, but thanks to CI, we can automate this process.
Let's define what tools we want to use. My project is written in JavaScript and uses the following tools to check source code:
You can use whatever tools your language supports - for example for Python you could use PyLint, Flake8, or Black.
On top of this, we'll also want to run tests on our source code. My project uses Jest, but again, you can use whatever testing framework you'd like.
To implement CI in GitHub Actions, we create the file .github/workflows/[workflow_name].yml in our project. This is a file written in YAML, which will describe the tasks we want to run.
Here's what it looks like in my project:
name: CI
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
jobs:
lint-format:
name: Check formatting and lint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [20.x]
steps:
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
- run: npm ci
- run: npx prettier --check .
- run: npx -y oxlint@0.10.3 --deny-warnings
- run: npx eslint
This might look confusing (or it might not), but let's walk through it.
// Give our workflow a name. This will be shown in the GitHub UI.
name: CI
// Define when we want to run our workflow. In this case, we want to run it on pushes and pull requests to the main branch.
on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
// Define a job. A job contains series of steps to be run in sequence.
jobs:
// We'll call this job 'lint-format'. This won't show up in the UI but can be used to refer to this job within the workflow file, like a variable name.
lint-format:
// Give the job a name that does appear in the UI.
name: Check formatting and lint
// Pick an operating system to run our tasks on.
runs-on: ubuntu-latest
// Define what version of Node.js we want to use.
strategy:
matrix:
node-version: [20.x]
// Define a series of steps to be run in sequence.
steps:
// Check out our code on to the machine running the CI job
- uses: actions/checkout@v4
// Setup Node.js using the version we defined earlier
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
// Define a series of commands to run:
// Install dependencies from package-lock.json, then run our tools one by one.
- run: npm ci
- run: npx prettier --check .
- run: npx -y oxlint@0.10.3 --deny-warnings
- run: npx eslint
In the list of steps, you'll notice both the uses and runs keys.
-
useslets us use pre-defined steps from the GitHub Actions Marketplace, so we don't have to write our own commands. This greatly simplifies steps like checking out our code from our repository onto the machine running the CI workflow, and setting up our Node.js runtime environment. -
runslets us manually write commands to run. This is great for running simple commands such as scripts or tasks defined in your project, like inpackage.jsonfor a Node project orbuild.gradlefor a Gradle project.
Also notice we're defining a strategy matrix. This lets us run our tests against different combinations of variables. For example, different versions of Node.js on different operating systems.
jobs:
example_matrix:
strategy:
matrix:
version: [10, 12, 14]
os: [ubuntu-latest, windows-latest]
For this example though, we're only using one version of Node.js on one operating system.
Now that we've defined this file, we can go ahead and push it to our repository. Navigate to the Actions tab to see currently running and past workflows.
Click on a workflow run and then select your job. You'll be able to see each step execute in order.
Last week, I added unit tests to my project with Jest. So let's go ahead and add a second job for unit testing. Jobs run in parallel to each other, meaning they can run at the same time.
jobs:
lint-format:
// ...
unit-tests:
name: Unit tests
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [20.x]
steps:
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
// Install dependencies and run tests
- run: npm install-ci-test
We do the same thing again, but at the end, instead of running our code checking tools, we run a command that installs our dependencies and runs our tests.
CI can save a lot of trouble if implemented well. There's good reason open source projects use CI all the time. It's so quick and easy to set up and it pays off in dividends.
Development Containers
Development Containers or Dev Containers is a standard that allows us to define a container to use as a development environment. We can automatically set up any tools required for the development environment to improve the developer experience. I use Visual Studio Code with the Dev Containers extension for my project, which automatically generates a nice configuration file.
I used the Node.js 20 with JavaScript base image, set it up to run npm install after the container is created, and automatically install recommended VSCode extensions.
{
"name": "Node.js",
"image": "mcr.microsoft.com/devcontainers/javascript-node:1-20-bookworm",
"postCreateCommand": "npm install",
"customizations": {
"vscode": {
"extensions": [
"dbaeumer.vscode-eslint",
"streetsidesoftware.code-spell-checker",
"esbenp.prettier-vscode",
"oxc.oxc-vscode"
]
}
}
}
I got one of my classmates to test it out and he said the experience went smoothly.
Testing Out My Classmate's CI Workflow
To test each other's CI workflows, me and my classmate added unit tests to each other's projects. My classmate's project is a tool called DialectMorph.
DialectMorph
DialectMorph is a powerful code transpilation tool designed to convert source code from one programming language to another. It leverages the Groq API to perform intelligent code translations while maintaining the original functionality and logic.
Features
- Supports transpilation between Python, JavaScript, Java, and C++
- Command-line interface for easy usage
- Automatically creates a
transpiledFilesdirectory for output - By default, Utilizes Groq's language model for accurate code conversion, option to use Gemini can be specified
- User can also provide their own API-Key to be used in this CLI Tool
- User can also request the the list of models available in the Groq-API
- User can also specify the model that they want to use for their use-case
Installation
-
Clone the repository:
git clone https://github.com/Kannav02/DialectMorph.git cd DialectMorph -
Install dependencies:
To install
bun, please follow this installation guide.bun installTo skip steps 3 and 4 , you can run the following…
This project used Bun and TypeScript, and Jest for testing. I've used all of these before so it was easy getting set up. When I opened the project directory, I noticed the tsconfig.json was throwing out tons of errors, because the root directory was set to src/ but there were TypeScript files in tests/. I also noticed a couple of dependencies were missing from the dependency list when I ran npm run test and it told me it couldn't find them. I made pull requests to fix these.
Then I got into writing the test. I ran npm run test:coverage to see what still needed testing. I found that successful responses from the Google Gemini AI client were not being tested, so I wrote a unit test for that. I based it on the error test but modified the client's mock implementation to return valid data. I ran into a weird TypeScript bug here where I wanted to use jest.fn().mockResolvedValueOnce(), but it kept throwing invalid type errors.
I found this issue on GitHub where somebody suggested the workaround of using jest.fn.mockImplementation(asyc ()=>{Promise.resolve()}), so that's what I did.
Argument of type X is not assignable to parameter of type 'never'
#2610
I am trying to use ts-jest to mock one of my dependencies. This dependency (auth0.ManageClient) has multiple definitions for a same method and overrides it based on the params you pass it:
// Roles
getRoles(): Promise<Role[]>;
getRoles(cb: (err: Error, roles: Role[]) => void): void;
getRoles(params: GetRolesData): Promise<Role[]>;
getRoles(params: GetRolesData, cb: (err: Error, roles: Role[]) => void): void;
getRoles(params: GetRolesDataPaged): Promise<RolePage>;
getRoles(params: GetRolesDataPaged, cb: (err: Error, rolePage: RolePage) => void): void;
When trying to mock the getRoles method, I get the following message: Argument of type X is not assignable to parameter of type 'never'.
I think this is because ts-jest is expecting me to override the getRoles(cb: (err: Error, roles: Role[]) => void): void; which is not what I am trying to do. I want to mock its result with .mockResolvedValue.
import { ManagementClient } from 'auth0';
import { MockedObjectDeep } from 'ts-jest/dist/utils/testing';
import { mocked } from 'ts-jest/utils';
jest.mock('auth0');
describe('UsersService', () => {
let manageClient: MockedObjectDeep<ManagementClient>;
beforeEach(() => {
jest.clearAllMocks();
manageClient = mocked(
new ManagementClient({
domain: 'DOMAIN',
clientId: 'CLIENT_ID',
clientSecret: 'CLIENT_SECRET',
}),
true
);
});
describe('updateUserRoles', () => {
beforeEach(() => {
manageClient.getRoles.mockResolvedValue([]); // error will show here
});
it('should test something', async () => {
// ...
});
});
});
Typings should support method overriding.
The intended solution seems to be to use jest.mocked(), but that would require modifying all of the other unit tests to use the returned mocked value, and I didn't want to make such a drastic change when the one above worked perfectly fine.
With that dealt with, I ran the test and generated the coverage report. Everything looked good, so I made a pull request.
Add test for successful chat completion generation
#31
Adds a test case which checks that a successful chat completion generation matches the expected format.
Coverage before:
Coverage after:
My classmate ran the checks in CI, which passed, and the PR was approved.
Conclusion
I think Continuous Integration and Dev Containers are indispensable tools for developers, especially when working in a team. They make things so much easier that I can't imagine not using them. I highly encourage anyone not already using them to check them out.
That's it for this post. Thanks for reading.
Top comments (0)