Wen Ping

Posted on Oct 29 Originally published at anguskit.com

After the repository accepts pushes: how to write the built-in pipeline YAML

#cicd #angusgit

This article explains how to write a .anguscicd.yaml that builds and tests on every push, after the repository already accepts pushes to AngusGit. After reading it, you can fill in the top-level fields, chain several jobs in order, choose when the pipeline runs, and reference a credential (a token or password saved on the platform) from a script without writing the token into a repository file.

Three facts come first. This file uses AngusGit’s own syntax. It is a different file from GitLab CI’s .gitlab-ci.yml and from a GitHub Actions workflow. Pipelines are available on Team and Enterprise. Community has no CI/CD menu. A runner is the program on a build machine that picks up a job and runs the commands. Connect one before the first run, or the job stays pending after the push. Sign in to AngusGit, open CI/CD → Runners in the sidebar, and click Add Runner. Change Control plane URL to an address the build machine can reach. Name the machine, for example build-linux-01. Click Generate token. The token lasts 1 hour and the plaintext is shown only once. When the job should run in a container, open the Run with Docker tab and set the labels to linux,docker. When the job should run directly on the build machine, open the Zip install tab, leave docker out of the labels, and install JDK 21 on the machine first. Copy the command onto the build machine and run it. When the list shows that runner as Idle, write the YAML. A job is one group of steps in a pipeline.

The YAML below comes from the example repository xlliu/test-cicd-example. The clone URL is https://git.anguskit.com/xlliu/test-cicd-example.git. Two files in that repository are used here: .anguscicd/01-basic-build.yaml (name: basic-build-test) is shown in the console as test-cicd-example · 01 Basic Build, and .anguscicd/04-monorepo-change.yaml (name: monorepo-change-analysis) is shown as test-cicd-example · 04 Monorepo Change.

1. Where the file lives

The pipeline file lives in the repository with the code. There are two places:

.anguscicd.yaml in the repository root, when the repository has one pipeline.
Any .yaml or .yml file under .anguscicd/, for example .anguscicd/ci.yaml or .anguscicd/release.yaml, when the repository has several pipelines. Each file is one pipeline.

The file name has to be exact. A name such as .gitlab-ci.yml or .angus-ci.yml is not treated as a pipeline config.

2. A small file that already runs

.anguscicd/01-basic-build.yaml in the example repository is a config that runs all the way through on a runner. Whenever a commit is pushed to main or develop, it does three things in order inside a node:20-alpine container: print the environment, run apps/hello-node, and run that module’s unit test. The file has no checkout step. The platform checks the code out before the job starts.

version: "1"
name: basic-build-test
triggers:
  - on: push
    branches:
      - main
      - develop
env:
  TZ: Asia/Shanghai
  APP_NAME: AngusCICD
pipeline:
  build:
    runner:
      mode: DOCKER
      image: node:20-alpine
      labels: ["linux", "docker"]
    env:
      MODULE: hello-node
    steps:
      - name: Show environment
        run: |
          echo "TZ=$TZ APP_NAME=$APP_NAME MODULE=$MODULE"
          node -v
          pwd
          ls -la
        timeout: 60
      - name: Run app
        run: node apps/hello-node/src/index.js
        timeout: 60
      - name: Unit test
        run: node --test apps/hello-node/src/index.test.js
        timeout: 120

Each field:

Field you write	What to put	What it does
`version`	`"1"`	Config format version. Leaving it out produces a warning
`name`	For example `basic-build-test`	Pipeline name. Required
`triggers`	A list of events	When the pipeline runs on its own, for example `on: push` matching the `main` branch
`pipeline`	A map of jobs	Every job. Required. 1 to 100 jobs
`build`	A job id	A job name you choose. `needs` refers to this id
`runner.mode`	`DOCKER` or `NATIVE`	Execution mode: inside a container, or directly on the host
`runner.image`	An image address	The container image. Required when the mode is `DOCKER`
`runner.labels`	An array of strings	An online runner takes this job when it has every one of these labels
`steps`	An array of steps	Run in order. Every step needs `name` and `run`
`timeout`	Seconds (1–86400)	How many seconds this step may run. Optional

Two fields need a longer note. First, name: the console list prefixes the repository name, so you see test-cicd-example · 01 Basic Build. Second, runner.mode has two values. DOCKER runs the commands in a container started from runner.image, so the image decides the tools. NATIVE runs the commands directly on the machine where the runner process lives, using tools already installed on that machine.

This sample uses the public image node:20-alpine, so the runner has to be able to pull it. If the internal build machine cannot reach the public network, use an image from your internal registry and write the full image name. apps/hello-java in the same repository is a Java module. Locally you can check it with mvn -B -ntp test. In a pipeline, change the image to one that includes a JDK and Maven.

The platform checks the code out for you. When the job starts, it checks out the commit that triggered this run into the working directory, and the commands run from the repository root. The checkout is a shallow clone: that one commit, with no history.

3. Several jobs: check first, then build by directory

Once one job is enough, split the flow. .anguscicd/04-monorepo-change.yaml in the same repository has three jobs. lint checks, and it runs every time. build-service-a and build-service-b build two services. Both write needs: ["lint"], and both use modules to name the directories they care about. This pipeline is triggered by a pull request whose target branch is main. On the branch feature/change-service-a, the only change is under apps/service-a.

version: "1"
name: monorepo-change-analysis
triggers:
  - on: pull_request
    branches:
      - main
smart:
  changeAnalysis: true
  cacheStrategy: aggressive
pipeline:
  lint:
    runner:
      mode: DOCKER
      image: node:20-alpine
    steps:
      - name: Lint all
        run: |
          node apps/hello-node/src/index.js
          echo "lint ok"
        timeout: 120
  build-service-a:
    needs: ["lint"]
    modules: ["apps/service-a/", "apps/shared-lib/"]
    runner:
      mode: DOCKER
      image: node:20-alpine
    steps:
      - name: Build service-a
        run: node apps/service-a/src/index.js
        timeout: 120
  build-service-b:
    needs: ["lint"]
    modules: ["apps/service-b/", "apps/shared-lib/"]
    runner:
      mode: DOCKER
      image: node:20-alpine
    steps:
      - name: Build service-b
        run: node apps/service-b/src/index.js
        timeout: 120

Three points in this file:

needs: ["lint"] sets the order. A job with no needs starts at the same time as the others. With needs, both builds wait until lint succeeds. If lint fails, they do not run.
Each job checks out the code on its own. The two jobs may land on different runners. They do not share a working directory, so a file written by one job is invisible to the other. The commands in this sample install nothing and run scripts that are already in the repository, so that split does not matter here.
The trigger is pull_request. Here branches: [main] is the pull request’s target branch. Every pull request that wants to merge into main runs this pipeline, and the result shows up in the status check on the pull request page (the row that shows whether the pipeline passed). The pull-request article linked at the end covers that in detail. modules works together with the top-level smart.changeAnalysis: when a change touches only one directory, the unrelated build is skipped.

4. How to write triggers

triggers decides when the pipeline runs on its own. It is a list. Each item uses on for the event, and it can add branches to limit the branch. Four events are the usual ones:

Value of `on`	Filter you can add	When it runs
`push`	`branches`	When `git push` sends commits to a matching branch
`pull_request`	`branches` (the target branch)	When a pull request is opened, when new commits are pushed to the source branch, and when the pull request is reopened
`tag`	None, or a tag rule	When `git push` sends a tag
`workflow_dispatch`	Input parameters `inputs`	When you click Trigger in the console and run it by hand

triggers can be left empty. On save, the message is “No triggers configured; the pipeline can only be triggered manually”.

on: push responds only to git push. Clicking the merge button on the pull request page to merge into main does not fire on: push. If main has branch protection (merge only through a pull request, with no direct push), a build on main has two paths. Use a pull_request trigger and finish the check before the merge. Or, after the merge, open this pipeline in the console and click Trigger to run it once by hand.

5. Passwords and tokens: create a credential, then declare it on the job

Publishing an artifact or calling an internal API often needs a token. A token written straight into the YAML is visible to anyone who can read the repository. Save it as a credential first, then reference it from the job.

Create the credential in the console

Open CI/CD → Settings, switch to the Credentials tab, and click New Credential. That opens Create Credential. Name is the name the YAML uses when it references this credential, for example NPM_TOKEN. The hint is “The name referenced from pipeline YAML secrets / credentials. Prefer UPPER_SNAKE_CASE. It cannot be renamed later.” Type defaults to Credential text, which fits a single token. Visibility defaults to Selected repositories, and you then pick the repository under Bound repositories. If you change it to All repositories, every pipeline in this tenant can use the credential. Paste the token into Value. After save, the plaintext cannot be viewed again. Fill in Expires at (optional) if you need it.

Declare which credentials the job uses

Under the job that needs the token, add a credentials field and list the credential names. A name that is not listed is not available to that job.

Use it in the command

A declared credential is handed to the job as an environment variable with the same name, so the command can use $NPM_TOKEN. The publish job below is written that way.

  publish:
    needs: [test]
    credentials: ["NPM_TOKEN"]
    runner:
      mode: DOCKER
      image: node:20-alpine
      labels: ["linux", "docker"]
    steps:
      - name: publish
        run: |
          echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/.npmrc
          npm publish

In env or in a condition, you can also write ${{ secrets.NPM_TOKEN }} or ${{ credentials.NPM_TOKEN }}. Both forms do the same thing, and both still require the name in this job’s credentials.

The picture is a filled-in Create Credential page: the name is NPM_TOKEN, the type is Credential text, and the visibility is Selected repositories bound to xlliu/test-cicd-example. Value has to be a real token before you save, because the plaintext cannot be viewed after save. If the token is not ready, click Cancel and return to the credential list.

6. Make the pipeline show up in the console

After the YAML is written, the pipeline still has to be created under CI/CD → Pipelines before it runs. There are two ways:

Push to the default branch. git push the commit that contains .anguscicd.yaml straight to the default branch. The platform picks it up and creates the pipeline.
Create it in the console. Click New Pipeline → Blank pipeline, fill in Pipeline Name, choose Repository, switch to YAML, paste the config, and click Create Pipeline. Before save, the platform runs a pre-run check. “All checks passed. Ready to save.” means you can save. An error shows “Fix errors first”.

If main has branch protection and you cannot push directly, the second way is the shorter path. A pipeline created in the console is stored on the platform and does not commit a file into the repository. If the repository later has YAML at the same path, the run uses the file on the branch that triggered it.

If you already have a GitLab CI or GitHub Actions config, the New Pipeline menu also has Import GitHub Actions and Import GitLab CI. They convert part of the syntax and produce a draft that still needs a person to check it. After import, go through the items one by one.

7. What you see when a field is wrong

The platform checks fields strictly. A field it does not recognize is an error. The usual cases:

What you see	Why	What to do
`Unknown field 'stages' in pipeline YAML. Please check for typos against the AngusCICD schema.` The same form appears for `variables` or `jobs`	The file uses GitLab CI or GitHub Actions fields	Put jobs under `pipeline`. Put global variables in the top-level `env`. Express order with `needs`
`Step 1 in job 'build' must have a name` (the numbers and the job id are the ones in your file)	Every step needs a name	Add `name` on every step
Writing `image` under the job is an error	The image belongs to the runner config	Write it under `runner.image`
The job stays queued and never starts	No online runner matches both `mode` and `labels`	Check the runner’s mode and labels
The job fails with `Credential not visible or expired`	The credential is missing, hidden from this repository, or expired	Check the credential name, visibility, and expiry
`$CI_COMMIT_SHA` and similar names in the command are empty	GitLab’s predefined variables are not injected	Use `GIT_COMMIT_SHA` and `GIT_BRANCH`

When a job stays queued, go back to CI/CD → Runners and check that the runner from the start of this article is Idle, and that its labels match runner.mode. To read the repository URL, the branch, or the commit id inside a script, use GIT_REPO_URL, GIT_BRANCH, and GIT_COMMIT_SHA, which the platform provides. Leave the GitLab CI_* names out.

Common questions

Can I try a run locally or in the console without really executing it?

There is no dry-run mode. The pre-run check when you create a pipeline in the console only validates the config. It does not run the commands. A steady approach is to create a pipeline whose only command is echo, confirm that a runner takes the job and finishes it, and then add the real steps.

How do jobs pass a file, for example a build output to a deploy job?

Jobs do not share a working directory, and the YAML has no field whose job is to pass an artifact (a build output) from one job to another. A common approach is to upload the output to an artifact repository from the build job, and download it from the deploy job. If build and deploy are already a few steps in a row, write them in the same job and run them in step order.

Dependencies are downloaded again every time, and that is slow. What can I do?

Set cache on the job. key separates one cache from another. paths lists the directories to cache, for example node_modules for a Node project. On the next run, when key is the same, the platform restores those directories before the steps run.

Summary

An AngusGit pipeline can start from a small file: fill in version, name, and triggers, then put one job under pipeline. Each job uses runner to choose the execution mode (DOCKER for a container, NATIVE for the host) and steps to list the commands. Every step needs name. The platform checks the code out. Several jobs run at the same time unless you set order with needs. For a token, create a credential in the console, list it in the job’s credentials, and use $name in the command. on: push responds only to git push. Merging on the pull request page does not fire it, so a protected main branch relies on a pull_request trigger for the check before merge.

Conclusion: In .anguscicd.yaml, write version: "1", name, triggers, and the jobs under pipeline (every job needs runner, and every step needs name). Use needs for job order and credentials for tokens and other secrets, and each push can build and test on its own.

DEV Community