Does your merge-request pipeline clone and build more than it reads?

Originally published at omerbinsaleem.com by Omer Bin Saleem (Omer Saleem).

Last week our app's merge-request pipeline started cloning only what each job reads, skipping the build when no app code changed, and running related specs in four shards. A big merge request's tests now finish in about a third of the time.

How to do it

1. Write down what each pipeline job actually opens. Change its clone so it fetches only those paths. A lint job for one app does not need the whole monorepo on disk.
2. Make the build job look at the diff first. When no app code changed, for example only a spec or a pipeline file, skip the build and let lint and tests carry the merge request.
3. Run only the specs related to the changed files, and split them across shards that run side by side. Our pipeline uses 4 shards. Match the count to the runners you have.
4. Stop downloading tools the pipeline does not use. Our merge-request pipeline no longer downloads Cypress at all; the unit-test jobs ran fine without it.
5. Give main the same pipeline as staging: lint, build and tests side by side, plus the nightly scripts. One pipeline definition is easier to trim than two that drift apart.
6. Tell the team when a pipeline starts and when a shared branch is green again. In our merge-request channel each new merge request is announced as soon as its pipeline starts, even when its tests fail, and a follow-up says when main, staging or dev is green again.

Prompt to copy

Read my CI pipeline config at <path to .gitlab-ci.yml or workflow file>. For each job, list what files it reads, what it clones or downloads, and whether it still runs when only tests or pipeline files changed. Then propose four changes: a narrower clone per job, a rule that skips the build when no app code changed, a way to run only the specs related to the diff split across N shards, and the removal of any download no job needs. Show the config diff for each change and say in one line what it saves and what it risks.

---

I am Omer Bin Saleem, a frontend chapter lead who builds with AI every day. One practical AI hack a day at omerbinsaleem.com. LinkedIn: omer-bin-saleem · Instagram: @iomerbinsaleem.

Written with AI assistance, from my own work notes.