Configuring a build plan

From an empty server to a repository that builds on every push. Every key of the pipeline document is in the reference; this page is the order to meet them in.

1. A project, and a plan in it

A project groups plans and is where permissions are granted once for all of them. It has a key — WEB — and a plan has its own, so a build is WEB-MAIN-42 wherever it is named: on a page, in a notification, in the search box.

A project groups plans once and permanently. Labels are the other grouping, and the only one that crosses a project: stick nightly on plans in four projects and every one of them is a link to the others.

2. Somewhere to build from

A token belongs to a forge, not to a repository. An administrator connects the forge once under Site administration, Forges, and every repository adopted from it clones with that credential — so rotating a token is one edit rather than one per repository.

Then Repositories, Add from a forge searches what that token can actually see and takes the repository's name, clone URL and default branch from the forge. It asks for a search term rather than opening on a list, because a token that can see thousands of repositories has no page anybody can read. A repository added by hand keeps a credential of its own, which is the path for a plain Git server.

Adoption is what makes commit status work. A build reports pending when it starts and its outcome when it finishes, keyed by a context naming the plan, so several plans building one commit each keep their own line. Setting a hand-added repository's type to Forgejo links it to nothing: the reporter needs the connection and the remote name, which only adoption writes, so such a repository builds happily and stays silent.

More than one repository

A plan's repositories are a list. The first is what the plan is built from: its branch, its pushes, its commit status. Each of the rest is checked out into a directory you name, at the branch you name, so a job can build against a shared library sitting beside its own source. Every job of one build sees the same commits, resolved once when the build was triggered.

What a checkout fetches

A checkout is one commit, shallow. Two things a repository may need beyond that are settings on the repository itself, under Add repository and the repository's own page, and both are off by default, so nothing about a build that already works changes.

They belong to the repository rather than to a plan because they are facts about its content: every plan checking it out needs the same answer, and the repository that matters most, the plan's own primary source, is not named through the per-plan list at all.

Fetch Git LFS objects. For a repository whose .gitattributes puts binaries in LFS. Without it the working tree holds pointer files, and the failure is silent: the clone succeeds, the compile succeeds, and what is packaged has none of its models, textures or binaries in it. With it, the agent runs git lfs fetch and git lfs checkout after the checkout, using the credential already in the remote URL. The agent's image must carry git-lfs; where it does not, the job fails and names the setting, rather than checking out the pointer files and building something with nothing in it.

Initialise submodules. git submodule update --init --recursive, at the checkout's own depth. Off by default for a second reason beyond size: a submodule is a clone from somewhere else, and the credential this repository carries may have no business there. A submodule named by a relative URL is the exception, as it is for git: the URL is resolved against origin, which is where the credential was folded in.

3. Where the pipeline lives

A pipeline is one YAML document describing everything the plan does. It is read from .dutest/pipeline.yml in the repository, or stored on the server for a plan that has no repository. Both go through the same parser and the same validator, so nothing works on one path and not the other.

By default every branch a plan builds is built by the document on the repository's default branch, so an edit takes effect when it is merged — the same rule the rest of your build already lives by.

4. A first pipeline

version: 1
name: Build and test
stages:
  - name: Build
    jobs:
      - key: BUILD
        runtime:
          type: docker
          image: eclipse-temurin:24-jdk
        resources:
          cpu: "2"
          memory: 4Gi
        steps:
          - name: Build and test
            script: ./gradlew build
        artifacts:
          - name: app
            paths: ["build/libs/*.jar"]
        tests:
          paths: ["**/build/test-results/**/*.xml"]
  - name: Verify
    jobs:
      - key: VERIFY
        consumes:
          - name: app
            from-job: BUILD
            into: incoming
        steps:
          - name: Check the artifact
            script: test -f incoming/build/libs/app.jar
triggers:
  - type: webhook
    branches: [main, "release-*"]

Property names are kebab-case, and an unrecognised key is an error rather than something to ignore: a mistyped key is almost always a mistake its author wants to hear about immediately.

5. What each part of that buys

Stages run strictly in order. Verify starts only once Build succeeded, which is what lets it consume what Build produced.

Jobs are the unit of scheduling. Jobs in a stage run in parallel on different agents, so a job cannot wait on a sibling — only on an earlier stage.

Runtime is per job, never per step: every step shares one workspace and one environment. A Docker job starts one container and execs each step into it.

Steps on the build page are not only the ones written here. The clone of each repository, the fetch of each artifact an earlier stage produced, the container's start, the report reading and each artifact's upload are listed with them, in the order they happen and each with what it cost — unnumbered, so the numbering still counts the steps in this document. It is where the minutes of a job that is slow before it compiles anything are to be found.

Resources. cpu and memory are held on the agent for as long as the job runs, written the way Kubernetes writes them. Memory is also a ceiling; CPU is a reservation and never one, because a throttled build is slow in a way nothing on the build page explains.

Artifacts are collected even from a failed job, because the partial output of a failure is usually what somebody needs.

Tests are read from JUnit XML. A job that declares tests is judged on them: any failed case fails the job whatever the steps exited with, which is what catches a runner that swallows its own failure.

Getting the job to an agent that can run it

requires is matched against what each agent advertises, and resources against what it has left. A job larger than anything in the fleet does not fail — it waits, and the queue page says it is waiting for an agent large enough rather than for a free one, because a busy fleet drains on its own and this one never will.

requires:
  os: linux
  dockerd: "true"

6. Deciding when it builds

Triggers are declared in the document, beside the pipeline they start, and applied whenever a valid pipeline is built. A webhook trigger carries as many conditions as you like and a push has to satisfy every one of them.

triggers:
  - type: webhook
    conditions:
      - type: branch
        include: [main, "release-*"]
      - type: commit-message
        exclude: ['\[skip ci\]']
  - type: schedule
    cron: "0 0 3 * * *"
    branch: main
  - type: after-plan
    plan: WEB-MAIN

A branch pattern is a glob matched against the branch as a path: feature/* matches feature/login and not feature/login/retry, while feature/** reaches every depth. A pattern the glob syntax rejects is compared exactly, which is what makes a plain list of branch names say what it looks like it says.

A webhook trigger can also be configured on the Triggers tab of the plan's settings page, and the document does not overwrite one configured there. That is how a plan whose pipeline lives in its repository gets its first push to build, since nothing reads the triggers: block until a build has resolved a pipeline. The repository page hands out the URL to paste into the forge — built from dutest.base-url, which is why that setting is worth getting right before this step.

Branches

Branches are discovered from what gets built, and retired when the forge says they are gone. Every one of them builds the pipeline on the default branch unless the plan allows a branch its own.

That is a setting, not the behaviour, because a pipeline names the commands an agent runs and the secrets they run with: allowing a branch its own pipeline lets anybody who can push a branch this plan builds run commands of their own choosing on an agent. On a repository whose branches only trusted people push it is worth having — a branch that adds a job or moves to a new image proves it on that branch and nowhere else. The branch's document applies to that branch alone, and the triggers it declares are not adopted as the plan's, so a schedule written on a branch cannot start builds of any other.

How many builds of one branch run at once

By default, as many as there are agents to run them. The plan's settings limit it, per branch: two branches building at once is what a build server is for, and what actually collides is two builds of the same branch sharing a deployment target or a database. A build over the limit either queues behind the one in front, or starts and cancels the older ones, which is what you want when only the newest commit is worth building.

7. Secrets

Secrets are never written in the pipeline. They are held on the server, scoped to a project, a plan or an environment, and merged into every step's environment beside variables. The document refers to one by name only:

steps:
  - name: Publish
    script: ./publish.sh --token "$REGISTRY_TOKEN"

A pipeline can also name which secrets it wants, and what to call them, with an environment: block.

They are encrypted at rest, never readable back through the API, and struck out of a job's output on the agent before it is sent anywhere, so one printed by accident reaches the log as ***.

8. What happens after it goes green

A deployment project belongs to a plan and holds environments and releases. A release names a build that succeeded, and deploying materialises an ordinary run of the environment's own pipeline against the commit the release was cut from.

Its jobs are ordinary jobs: each gets what it names in consumes: and nothing else, and here that resolves against the released build, so from-job is one of its job keys. See Consuming artifacts.

A plan can also ship itself: name a branch, and every build of it that passes cuts a release, which an environment set to deploy on release then has without anyone pressing anything.

The pipeline an environment runs is either stored on the server or read from the repository, the same choice a plan has. Read from the repository, it is the document at the path you name in the commit the release was cut from, so a change to how something ships lands with the commit that makes it, beside the scripts it calls.

All of that can be declared in the plan's own pipeline instead of set up on a page. A deployments: block names the deployment project, its environments, the file each one deploys with and the order a release travels in, and the server writes it down whenever a build resolves that pipeline — so a fresh install comes up from the repository rather than being clicked back together. See Deployments.

Two things it deliberately will not do. It never removes an environment, because that would take the environment's variables, secrets and permission grants with it. And it never sets a credential: a deployment project it creates is inert until somebody puts the secrets on it by hand, which is the one decision worth making in front of a person.

Notifications reach people by email or an HTTP webhook, on every run, on failures, or only when the outcome changes — which is the setting that survives a fortnight of a flaky test. A flaky test itself can be quarantined per plan, which stops its failures failing the build without stopping it being run or reported, and needs no commit.