Pipeline reference¶

A pipeline is one YAML document describing everything a plan does. It is read from .dutest/pipeline.yml in the repository, or stored on the server for a plan that has no repository. Both go through the same parser and validator.

Which commit it is read from is the plan's own setting. By default every branch a plan builds is built by the document on the repository's default branch, so an edit takes effect when it is merged.

It is read as one file at that commit, never as a checkout: through the forge's API for a repository adopted from a forge connection, and over Git with a blob:none filter for one added by its clone URL. A build refused because the commit has no document says so as "No pipeline at"; one refused because the repository could not be read says that instead, with the cause, since the document may be there and the fix is somewhere else.

A plan whose settings page allows its branches their own is built by the document in the commit being built, and that document applies to that branch alone, including its triggers:, which are not adopted as the plan's. Allow it only where everybody who can push a branch is trusted to choose the commands an agent runs.

Property names are kebab-case. An unrecognised key is an error rather than something to ignore, because a mistyped key is almost always a mistake its author wants to hear about immediately.

Document¶

version: 1
name: Build and test
variables:
  GRADLE_OPTS: -Xmx2g
stages: []
triggers: []
deployments: []

Key	Type	Default	Notes
`version`	integer	`1`	The only version this server understands
`name`	string	none	For display
`variables`	map	`{}`	Environment for every step, unless shadowed
`environment`	map	`{}`	Names the variables this pipeline wants, and what it calls them; see Environment
`stages`	list	`[]`	At least one is required
`triggers`	list	`[]`	See Triggers
`deployments`	list	`[]`	See Deployments

Variable names must be usable as environment variable names.

A project, a plan or the server can hold variables of their own, which every job in reach of them starts with and which this block shadows key by key. See Variables held on the server.

A pipeline says nothing about the checkout, and nothing here can. Which repositories a job's workspace holds is the plan's list; what a checkout of one of them fetches beyond a single commit — its Git LFS objects, its submodules — is a setting on the repository, since every plan checking it out needs the same answer. See what a checkout fetches.

Stages¶

Stages run strictly in order. A stage starts only once the one before it succeeded.

stages:
  - name: Build
    jobs: []
  - name: Deploy
    manual: true
    jobs: []

Key	Type	Default	Notes
`name`	string	required	Unique within the pipeline
`manual`	boolean	`false`	Waits to be released by someone holding BUILD
`jobs`	list	`[]`	At least one is required

Jobs¶

A job is the unit of scheduling. Jobs in a stage run in parallel, each on one agent, and every step of a job shares that agent's workspace and environment, so a step can rely on what the step before it installed or wrote.

jobs:
  - key: BUILD
    name: Compile and test
    runtime:
      type: docker
      image: eclipse-temurin:24-jdk
    requires:
      os: linux
      arch: amd64
    resources:
      cpu: "2"
      memory: 4Gi
    services:
      - name: postgres
        image: postgres:16
    variables:
      CHANNEL: stable
    timeout-seconds: 1800
    steps: []
    artifacts: []
    consumes: []
    tests:
      paths: []
    coverage: []
    metrics: []

Key	Type	Default	Notes
`key`	string	required	Uppercase letters, digits and underscores, starting with a letter. Unique across the whole pipeline
`name`	string	the key	For display
`runtime`	object	native	See Runtimes
`requires`	map	`{}`	Matched against what each agent advertises. `dockerd: "true"` for a job whose steps start containers
`resources`	object	none	See Resources
`services`	list	`[]`	See Services
`variables`	map	`{}`	Wins over the pipeline's
`environment`	map	`{}`	Wins over the pipeline's `environment`; see Environment
`timeout-seconds`	integer	`3600`	Must be positive
`steps`	list	`[]`	At least one is required
`artifacts`	list	`[]`	See Artifacts
`consumes`	list	`[]`	See Consuming artifacts
`tests`	object	none	See Tests
`coverage`	list	`[]`	See Coverage
`metrics`	list	`[]`	See Metrics

Job keys must be unique across every stage, not merely within one, because they identify a job's results and artifacts across the whole build.

Runtimes¶

Declared per job, never per step.

runtime:
  type: docker
  image: eclipse-temurin:24-jdk

`type`	Meaning
`native`	Steps run as subprocesses in a workspace directory on the agent host
`docker`	All steps exec inside one container started from `image`

image is required for docker and rejected for native, so a pipeline cannot name an image that will be silently ignored.

A container is started on an idle entrypoint and each step is exec'd into it, so the image's own entrypoint is not used and cannot be overridden.

A docker job can also name containers to run beside it — see Services.

A private image is pulled with a credential named beside it, and the pipeline holds the names rather than the values — see Registry credentials.

Resources¶

What a job needs held for it while it runs. A job that declares nothing is scheduled the way every job was before this existed: wherever a slot is free — unless the server sets dutest.build.default-job-resources, in which case a job that declares nothing is taken to ask for that, one key at a time, and is reserved, shown and held to exactly as if it had.

resources:
  cpu: "2"
  memory: 4Gi

Key	Type	Default	Notes
`cpu`	string	none	Cores (`2`), or thousandths of one (`500m`)
`memory`	string	none	Bytes (`536870912`) or a size (`512Mi`, `4Gi`, `2G`)

Quantities are written the way Kubernetes writes them. 4GB is not one of them and is rejected rather than read as four bytes — it is what somebody writes when they mean 4Gi, and a job read as asking for nothing runs wherever it does not fit.

Either key may stand alone. A job that only knows it needs four gigabytes says so and leaves CPU to whatever the agent would have given it.

What the reservation means. The job is only sent to an agent with that much capacity unreserved, counting what the jobs already on it reserved, and it holds that share for as long as it runs. An agent that has not declared its capacity, such as one running an older version, is never given a job that asks for a share of it, and is still given every job that asks for nothing.

Memory is also a ceiling. A Docker job's container is held to the memory it reserved, and a pod this server starts on Kubernetes is given it as both a request and a limit. A job taking more than it declared would be taking it from whatever else is on that machine, and the kill would land on somebody else's build.

CPU is a reservation and never a ceiling, which is what requests.cpu means on a pod. A throttled build is slow in a way nothing on the build page explains. An operator who wants a hard cap sets dutest.agent.sandbox.cpus on the agent.

A job larger than anything in the fleet waits rather than failing. The queue page says it is waiting for an agent large enough rather than for a free one, because a busy fleet drains on its own and this one never will.

A job that runs services beside it holds what they asked for as well, since that is what the agent is out of pocket while it runs. Each container is still held to its own figure.

Services¶

Containers the job runs beside: a database its tests talk to, a broker, a Docker daemon of its own. They are started before the first step and removed after the last one, whatever the job did in between.

services:
  - name: postgres
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: app_test
    ready: [pg_isready, -U, postgres]
    resources:
      cpu: 500m
      memory: 1Gi

Key	Type	Default	Notes
`name`	string	required	The hostname the job reaches it at. Lowercase letters, digits and dashes
`image`	string	required
`environment`	map	`{}`	This container's only environment; see below and Environment
`command`	list	the image's	Replaces the image's command, word for word
`ready`	list	none	Run inside the container until it succeeds; the job waits for that
`ready-user`	string	the container's own user	Who `ready` runs as
`required`	boolean	`true`	When false, a service that will not start is left out rather than failing the job
`resources`	object	none	Held for this container, over and above the job's own
`privileged`	boolean	`false`	Root on the agent's machine. Refused unless the agent allows it

Services need runtime.type: docker. The job's container and its services share a network of the job's own, and a native job has no container to put on it.

A name is an address. postgres:5432 from a test, with nothing published to the agent, so two builds of two branches can each run a database on one machine with no port to collide over. The network holds this job's containers and nothing else, which is also why one build cannot reach another's database.

Each service has a tab of its own under the build page's Logs tab, showing what that container printed, live while the job runs and stored with the build afterwards. Its own tab rather than lines in the build's log, because a database writes a line per connection and a daemon a line per layer: interleaved, neither log is readable. The job's own log still says what started, when, and at what name. /api/v1/builds/jobs/{jobRunId}/log?service=<name> is the same output for anyone reading a build without a browser.

A service's image is pulled the same way as the job's own, and names its own credential if it needs one. Each is its own pull, which is the ordinary shape of it: a private toolchain beside a stock postgres is two registries.

Nothing is passed to a service that it was not given — not the job's variables and not its secrets, because an image named in a pipeline is not something to hand a deployment key to. Anything a service needs goes in its own environment.

Say how it is ready. ready runs inside the container every half second until it exits zero, and the job's first step waits for that. It is worth writing: neither postgres nor docker:dind declares a HEALTHCHECK, so without it the job races a database that is up but not yet listening, producing an intermittent connection refused blamed on the test.

Where the image declares a HEALTHCHECK and the pipeline says nothing, that is waited on instead. Where there is neither, the job starts as soon as the container is running. A service that stops before the job starts, or never becomes ready, fails the job with its own last lines of output, since the reason a database would not start is on its stderr and nowhere else.

It runs as whoever the service is, which is not who its image says it is. A container started from the official rabbitmq image runs as root until its entrypoint drops to rabbitmq, and an image declaring no USER is exec'd into as root. So ready: [rabbitmq-diagnostics, -q, ping] wrote /var/lib/rabbitmq/.erlang.cookie owned by root half a second into start-up, and the server that had already dropped privileges died reading its own cookie. The probe broke the thing it was asking about. The command therefore runs as whatever the container's main process is running as, read out of the container rather than off its image, which is also the right answer for postgres and mysql. ready-user covers the image where it is not, such as one whose PID 1 is a supervisor the probe must not run as, and takes a name or a uid as docker exec --user does.

required: false is for a service the build can do without. Strictness is the default and stays it: a build talking to a broker that never came up fails half a minute later on a connection refused inside a test, and the reason is on a container thrown away by then. Where the tolerant answer is the true one, such as something the build reports to or a cache it can miss, required: false leaves the container out and says so in the job's log rather than failing the build. It never passes over it in silence, because a service that is not there is worth knowing about even when it is allowed not to be.

Resources are its own, written exactly as Resources are. The agent reserves the job and every service added together, so a build reserving 4Gi with a database reserving 1Gi only goes to an agent with five to spare. Each container is held to its own line, so a service cannot take the build's share and have the build killed for it. A service that reserves nothing is bounded by the agent's own settings, exactly as a job that reserves nothing is.

The workspace is mounted into each service at the same path the job sees it at. This is what makes a Docker daemon of the job's own useful: a bind mount is resolved by the daemon rather than by whoever asked for one, so a build running docker run -v $PWD:/src inside dind would otherwise get an empty directory.

A Docker daemon of the job's own¶

      - key: IMAGE
        runtime:
          type: docker
          image: docker:27-cli
        services:
          - name: docker
            image: docker:27-dind
            privileged: true
            environment:
              DOCKER_TLS_CERTDIR: ""
            ready: [docker, info]
        variables:
          DOCKER_HOST: tcp://docker:2375
        steps:
          - script: docker build -t app .

privileged: true grants root on the machine the agent is on, which is what a container runtime needs and equally what anyone who can push to the repository would need to take the agent. An agent runs one only where an operator has set dutest.agent.sandbox.allow-privileged-services, and a job that asks elsewhere fails naming that setting, rather than starting unprivileged and failing later on something inexplicable.

On Kubernetes there is a second way to the same thing that needs no privileged service at all: dutest.agent.kubernetes.docker-in-docker.enabled gives every agent pod a daemon beside it, and a job simply uses the one it already has.

Where a pod carries its own daemon, note what bounds a job's containers there. They are started by that daemon and live inside it, so the ceiling over all of them together — the job's own container and every service — is dutest.agent.kubernetes.docker-in-docker.memory-limit on the agent pod, whatever the individual reservations say. Each container is still held to its own line; it is their sum that meets a limit the pipeline did not set. Raise it where jobs run services.

Steps¶

steps:
  - name: Compile
    script: ./gradlew assemble
  - name: Test
    script: ./gradlew test
    working-directory: server
    environment:
      CI: "true"
    continue-on-failure: true

Key	Type	Default	Notes
`name`	string	`Step N`	For display
`script`	string	required	Run by the job's runtime
`working-directory`	string	workspace root	Relative to the workspace
`environment`	map	`{}`	For this step only; beats every variable but the built-ins. See Environment
`continue-on-failure`	boolean	`false`	Later steps still run; the job still fails
`inject`	block	none	Reads variables from a file instead of running a script

continue-on-failure is for a step whose report a later step collects. The job as a whole still fails.

A step either runs a script or has an inject block, never both and never neither. A document that sets both is refused, because the script would never run.

Environment¶

An environment: block names the variables a pipeline actually wants, and what it wants them called. It is written on the pipeline, on a job, on a step and on a service, and each entry is one of two shapes:

environment:
  CI: "true"                      # a value, written here
  REGISTRY_TOKEN:
    from-variable: HARBOR_TOKEN   # the variable holding the value

A scalar is a value. A mapping names a variable, and the only key it takes is from-variable. There is no type: to keep in step with the shape, because YAML already tells the two apart.

Naming a variable is an extra layer, not yet a gate. Everything a project, a plan, an environment or the server holds still reaches every step whether the document asks for it or not, exactly as it always has — an entry here adds a name rather than taking the others away. Writing them down is worth doing now because that is changing: a build handed every variable in scope hands each of them to every script, and to anything a script starts, so a deployment key set for one job is in the environment of the test runner two jobs away. Once documents name what they need, the wholesale merge goes.

Narrowest wins. What a job starts with runs: what the server holds, then the document's variables:, then a job's, then what an earlier stage injected, then the pipeline's environment:, then the job's. A step's own environment: goes over that for that step alone, and built-in variables beat all of it.

A service can be given a secret. Its environment: takes the same two shapes, so a database password is a variable named here rather than a literal written in the document beside the image — which matters because a service is given its own environment and nothing else.

A name nothing answers to fails the job, saying which, rather than arriving empty. A build running with an empty credential does not fail here; it fails later, against a registry or a cluster, with a 401 whose cause is a typo three files away. Whether a variable will be in scope is not a fact about the document — the server's variables can change between a commit and a build — so this is decided when the job is dispatched, and the job goes red with the variable named on its own page.

Variables from a file¶

A step with an inject block reads a properties file out of the workspace and hands what it finds to the steps after it.

stages:
  - name: Build
    jobs:
      - key: BUILD
        steps:
          - name: Work out the version
            script: |
              echo "version=$(git describe --tags)" > target/build.properties
              echo "channel=nightly" >> target/build.properties
          - name: Read the version
            inject:
              file: target/build.properties
              prefix: APP_
              scope: build
          - name: Package
            script: ./package.sh --version "$APP_version"

Key	Type	Default	Notes
`file`	string	required	Relative to the workspace root
`prefix`	string	none	Put in front of every name read from the file
`scope`	`job` or `build`	`job`	How far the variables reach
`required`	boolean	`true`	When false, a missing file is passed over

The file is a Java .properties file: key=value per line, # and ! comments, \ continuations, and the escapes that format defines. It is read as UTF-8. Every name, after the prefix, must be usable as an environment variable name and at most 128 characters. A file containing one invalid name fails the step rather than injecting the rest of it.

scope: job reaches the steps after this one, in this job. scope: build also reaches every job of every later stage. Not the jobs beside it in its own stage: those run in parallel on other agents and have already started, which is the rule consumes follows for the same reason.

Nothing read this way outlives the build that read it. There is deliberately no wider scope: a version read out of one run's workspace and surviving into the next is how something gets published under the previous build's name.

Injected variables take precedence over the variables a pipeline or job declares, and over what its environment: gave the job, from the step that read them onwards. A step's own environment: takes precedence over an injected one, since it is the narrowest thing the document says, and the built-in DUTEST_* variables take precedence over all of them.

The step is not run by the job's runtime, so it has no working-directory and no environment of its own, and a container image without a shell is no obstacle. It appears on the build page as an ordinary task, with the names and values it set in the log beneath it, and continue-on-failure behaves as it does elsewhere.

Artifacts¶

What a job produces.

artifacts:
  - name: app
    paths: ["dist/*", "build/libs/*.jar"]
    required: true

Key	Type	Default	Notes
`name`	string	required	Unique within the job
`paths`	list	required	Ant-style globs against the workspace root
`required`	boolean	`true`	When true, matching nothing fails a job that otherwise passed

A path may not be absolute or contain ... Artifacts are collected even from a failed job, because the partial output of a failure is usually what someone needs.

required catches a job that passed and quietly produced nothing, so it only applies to a job that passed. A job already failing matched nothing because the step that writes the artifact never ran, so that is noted in the log rather than becoming the job's verdict, which stays the step that failed.

Consuming artifacts¶

What a job wants from an earlier stage, placed in its workspace before the first step runs.

consumes:
  - name: app
    from-job: BUILD
    into: incoming

Key	Type	Default	Notes
`name`	string	required	An artifact an earlier stage produced
`from-job`	string	inferred	The producing job's key
`into`	string	`.`	Directory, relative to the workspace root

Only earlier stages qualify. Jobs within a stage run in parallel on different agents, so a job cannot wait on a sibling's output.

from-job may be left out when exactly one earlier job produces an artifact by that name. When two do, the pipeline is rejected rather than one being picked, because guessing would make the document's meaning depend on stage order nobody wrote down.

In a deployment¶

A deployment environment's document is the same document, with one difference here: a name no stage of it produces is resolved against the released build instead, and from-job is that build's job key.

stages:
  - name: Deploy
    jobs:
      - key: SHIP
        consumes:
          - name: app
            from-job: BUILD
            into: incoming

The document is asked first, so a deployment that packages something in an earlier stage of its own consumes its own. A job that consumes nothing starts with a plain checkout: nothing of the release is placed in a job that did not ask for it.

This is the one consumes: that is not checked when the document is saved. Which build will be released is not known then. It is checked when somebody deploys, and a name the release cannot answer fails that deployment before any job starts, saying what was asked for and what the release holds.

Tests¶

Where a job leaves its JUnit XML.

tests:
  paths: ["**/build/test-results/**/*.xml"]
  required: true

Key	Type	Default	Notes
`paths`	list	required	Globs against the workspace root
`required`	boolean	`true`	When true, producing no report at all fails the job

A job that declares tests is judged on them. Any failed case fails the job whatever the steps exited with, which is what catches a runner that swallows its own failure.

A step that failed before it could write its report keeps the blame. A missing report only fails a job whose steps all passed, since that is a job that claimed to run tests and did not.

JUnit XML is what every JVM, Python, Ruby and Node test runner can emit. External entities are never resolved, because a report is written by the build and is not trusted.

A test can be quarantined per plan, which stops its failures failing the build without stopping it being run or reported. That is managed on the plan rather than in the pipeline, so taking a flaky test out of the way does not need a commit.

Coverage¶

How much of the source this job's tests reached, and where it leaves the report.

coverage:
  - key: unit
    paths: ["build/reports/kover/report.xml"]
    required: true

Key	Type	Default	Notes
`key`	string	required	Letters, digits, dots, dashes and underscores, starting with a letter or a digit. Unique across the whole pipeline
`paths`	list	required	Globs against the workspace root
`required`	boolean	`true`	When true, producing no readable report for this key fails the job

Cobertura XML, JaCoCo XML and LCOV are read, and which format a report is in is determined from the file rather than declared here. Between them that covers every JVM, Python, Ruby, PHP, JavaScript, C and Rust coverage tool in ordinary use. External entities are never resolved, because a report is written by the build and is not trusted.

The key is what the numbers are added up against, build after build. A job measuring two things, such as a unit suite and a browser suite over different halves of the source, declares one entry each. Keys are unique across the whole pipeline rather than within a job, because a build's coverage is its keys added together and one key written by two jobs would count that part of the source twice.

What is recorded is lines covered and lines not covered. A percentage is not a quantity — two keys at 80% and 10% are not a build at 45% unless they are the same size — so the counts are stored and the percentage is worked out once, from their sum. Coverage is shown on the plan page, which is always of one branch, as a line over that branch's recent builds, with the keys behind the newest figure beneath it.

Several reports for one key are added together, and a file two of them both measured is counted once. A glob that catches an aggregate report as well as the per-module reports it was made from is a common mistake, and would otherwise report twice the code the project has.

Nothing is enforced. There is deliberately no threshold to fail a build on, because a floor gets met by testing whatever is cheapest to test. required is about the report arriving at all, not about what it says.

A step that failed before it could write its report keeps the blame, exactly as it does for tests: a missing report only fails a job whose steps all passed.

Metrics¶

Whatever else this project decided is worth watching build after build, and the variable each figure is left in.

metrics:
  - key: loc
    name: Lines of code
    value: $LINES_OF_CODE
    required: true

Key	Type	Default	Notes
`key`	string	required	Letters, digits, dots, dashes and underscores, starting with a letter or a digit. Unique across the whole pipeline
`name`	string	the key	What a reader is shown. Up to 128 characters
`value`	string or block	required	A variable: `from-variable: NAME`, or the older `$NAME`. A number written here instead is refused
`required`	boolean	`true`	When true, a job whose variable is unset or unreadable fails

Coverage answers one question every project asks the same way, and has a report format for it. Everything else a build knows about itself — lines of source, bundle size, benchmark duration, compiler warnings — is a number only that project's own steps can work out. So a metric names a variable rather than a file, and a step that already computes the figure only has to write it down.

The variable is read in the job that declares the metric, from the environment its steps left behind, so anything a step injected reaches it whichever scope it was injected at. A value that is not a variable reference is refused, because a literal would record the same figure for every build: a line across the page that looks like a measurement and is a constant somebody typed. from-variable: NAME is the spelling to write; $NAME is still read, and goes when the documents have moved.

What the variable holds is either a number or a map of names to numbers.

LINES_OF_CODE=42318
LINES_OF_CODE={"kt": 30102, "java": 1204, "sql": 7012}

Which of the two it is comes from what the build wrote rather than from anything declared here, so a metric grows a breakdown the day somebody teaches the step to produce one, with no edit to the pipeline. A map is JSON, since a step producing a breakdown will have used jq or a language's own serialiser and already holds one. Its values may be written as numbers or as quoted numbers, because half of what writes JSON in a build script writes the second. A plain number need not be JSON: +42 and 1e6 are what shells and awk produce, and both are read.

The figure is the map added up, and the parts are shown under it. Two metrics are never added together: lines of source and seconds of benchmark are not two halves of anything, and each is a line of its own.

A figure may be negative and may have decimals. It is stored as the decimal it was written as, to ten places, never as a floating point number, because a bundle size that gained a byte between the build and the page is indefensible.

Keys are unique across the whole pipeline rather than within a job, because a metric is one series drawn over a plan's builds and one key written by two jobs would be two figures drawn as one line.

Metrics are shown on a tab of the build page, with which job reported each and what each is made of, and on the plan page, which is always of one branch, as a line per metric over that branch's recent builds. Each line is drawn from nought to the highest reading rather than over the range the figure happened to move through, so a month in which the source grew by a thousandth is drawn flat.

Nothing is enforced. There is no threshold to fail a build on, exactly as there is none for coverage. required is about the figure arriving at all, not about what it says.

A step that failed before it could work the figure out keeps the blame, exactly as it does for tests and coverage: a missing metric only fails a job whose steps all passed.

Triggers¶

Declared here rather than configured separately, so a plan's triggers follow its pipeline. They are applied whenever a valid pipeline is built.

triggers:
  - type: webhook
    branches: [main, "release-*"]
  - type: schedule
    cron: "0 0 3 * * *"
    branch: main
  - type: after-plan
    plan: WEB-MAIN

`type`	Keys	Notes
`webhook`	`branches`, `conditions`	What a push has to satisfy. Neither means every push
`schedule`	`cron`, `branch`	Six fields: second minute hour day month weekday
`after-plan`	`plan`	The key of the plan this one follows
`manual`	none	Started by a person

A branch pattern is a glob of the kind Java's PathMatcher understands, matched against the branch as a path rather than as a shell string. A * covers one segment and stops at a slash, so feature/* matches feature/login and not feature/login/retry, while feature/** reaches every depth. A bare * matches main and nothing with a slash in it. An empty list matches every branch. A pattern the glob syntax rejects is compared exactly, which is what makes a plain list of branch names — [main, develop] — say what it looks like it says.

Conditions¶

branches filters on the pushed branch. conditions is that question in full: each one names what it reads and either the patterns it must match or the ones it must not.

triggers:
  - type: webhook
    conditions:
      - type: branch
        include: [main, "release-*"]
      - type: branch
        exclude: ["release-legacy"]
      - type: commit-message
        exclude: ['\[skip ci\]']

Key	Type	Notes
`type`	`branch` or `commit-message`	What the condition reads off the push
`include`	list	The subject must match one of these
`exclude`	list	The subject must match none of these

A push starts a build when it satisfies every condition on the trigger. Within one condition the patterns are alternatives: include passes when any matches and exclude when none does.

Exactly one of include and exclude is set on a condition. Saying both "only these" and "never those" about one subject is two conditions, which is why the example above has two branch entries rather than one.

A branch condition matches with the globs described above. A commit-message condition is a Java regular expression searched anywhere in the head commit's message and matched case-sensitively, so \[skip ci\] catches the convention wherever it appears and ^fix: is anchored to the start. A regular expression that will not compile is a validation error rather than a pattern that silently matches nothing.

A push whose message this server does not know is treated as one with an empty message: it fails an include and satisfies an exclude, so a build is neither started on a message nobody has nor withheld on a veto never shown to apply.

branches is shorthand for a branch condition with an include list. Writing both it and a branch condition on one trigger is refused rather than merged, since nothing in the document would say which of them won.

Conditions belong to a webhook trigger and are refused on the others. A schedule names the branch it builds, and a plan following another inherits one, so a condition there would filter a value the trigger itself chose, or ask about a commit nobody pushed.

A webhook trigger can also be configured on the Triggers tab of a plan's settings page. The document does not overwrite one configured there. That is how a plan whose pipeline lives in its repository gets its first push to build, since nothing reads the triggers: block until a build has resolved a pipeline.

An environment's pipeline declares no triggers. An environment runs when someone deploys to it, so a trigger there would be written expecting something that never happens.

Deployments¶

Where a plan's releases go, and in what order. Declared here so the path a release takes to production is in the repository beside the documents that describe each hop.

deployments:
  - key: ship-the-web
    name: Ship the web
    description: Server images, published to Harbor from a release
    release-branch: main
    environments:
      - key: harbor
        name: Harbor
        spec: .dutest/deploy-harbor.yml
        on-release: true
      - key: staging
        spec: .dutest/deploy-staging.yml
        after: harbor
      - key: production
        spec: .dutest/deploy-production.yml
        after: staging

Key	Type	Notes
`key`	string	Identity, unique within the plan. Lower-case letters, digits and hyphens
`name`	string	For display. Defaults to the key
`description`	string	For display
`release-branch`	string	A branch or glob whose successful builds cut a release here
`environments`	list	In the order they appear on the page

An environment takes:

Key	Type	Notes
`key`	string	Identity, unique within the deployment project
`name`	string	For display. Defaults to the key
`spec`	string	The environment's own pipeline, a path in the plan's primary repository
`on-release`	boolean	Deploy every release cut here as soon as it exists
`after`	string	The key of the environment this one follows

key is what this block is matched on, and it is the reason a name can be changed freely. Matching on the name would make a rename a second environment beside the first, with the variables, secrets and permission grants left on the one nobody deploys to any more.

spec names a file, never a document. The pipeline an environment runs is read from the commit the release was cut from, beside the scripts it calls, which is the whole reason that choice exists — see where the pipeline lives.

after names an environment declared beside it, and may name one written further down the list. Environments that follow each other in a circle are refused, as is an after naming nothing this block declares.

Nothing here is ever deleted. An environment whose key leaves the document goes on working exactly as it did, because removing one takes its variables, its secrets and its permission grants with it, and a key deleted by accident must not be able to do that. Remove an environment from its own page when you mean to.

Everything here is set from the document, including what it leaves out. Drop an after: and the environment stops following anything. Drop release-branch: and releases go back to being cut by hand. A block that could turn something on and never off would be a document that lies about where a release goes.

What it cannot do: set a variable, set a secret, or grant anybody a permission. A deployment project this block creates is inert until somebody sets the credentials on it, which stays a decision a person makes on the environment's page.

The block is applied whenever a build resolves a pipeline, which is how a plan whose pipeline lives in its repository gets its deployments at all — the same arrangement triggers: has, for the same reason. It is applied whole: if one environment cannot be written, none is, and the build carries on rather than failing.

A branch building its own document declares deployments for nobody. What it writes here is ignored entirely, since otherwise anybody who can push a branch the plan builds could point production at a document of their own.

An environment's pipeline declares no deployments:, for the reason it declares no triggers: an environment is somewhere a release goes, not something that decides where releases go.

Variables held on the server¶

Not everything a build needs belongs in its document. A value several plans of one project share has no single document to live in, and a credential must not be in a document at all. Both are held on the server, scoped from global down to one environment, and merged into every step's environment alongside variables:.

An environment: block is how a document says which of them it actually wants, and what to call them. That does not yet stop the others arriving — but writing it is what will, and it is the only way to give a held variable a different name inside the build.

They are set on a project's Edit project page, a plan's Settings page, an environment's own page, or through /api/v1/variables. Narrowest wins, so the order runs: the server, the project, the plan, the environment, then this document's variables:, then a job's, then a step's environment:, and built-in variables last of all.

A value may be several lines — a kubeconfig, a private key, a service-account document — and arrives at a job as it was written, with line endings normalised to LF.

# REGISTRY is registry.example.com on the project
variables:
  REGISTRY: staging.example.com   # every step of this pipeline
stages:
  - name: Publish
    jobs:
      - key: PUBLISH
        variables:
          REGISTRY: canary.example.com   # this job alone

A variable marked secret is encrypted at rest, is never returned by any page or endpoint, and is struck out of a job's output by the agent before it is sent — so a secret printed by accident reaches the log as ***. Values shorter than four characters are not masked, because a short value matches constantly and a log of asterisks hides more than it protects. A pipeline refers to one by name and never contains its value:

steps:
  - name: Publish
    script: ./publish.sh --token "$REGISTRY_TOKEN"

Nothing in the document says which variables are secret, and nothing needs to: a step reads $REGISTRY_TOKEN the same way either way. Shadowing crosses the two kinds, so a plan can hide a name its project holds in the open; the name arrives masked and the readable value never reaches the job.

Registry credentials¶

A registry that is not public needs a username and a password, and both are variables. The pipeline says which variables, beside the image they are for:

runtime:
  type: docker
  image: harbor.example.com/audrium/toolchain:latest
  credentials:
    username: $HARBOR_USER
    password: $HARBOR_TOKEN

Key	Type	Default	Notes
`username`	string or block	none	A variable: `from-variable: NAME`, or the older `$NAME`. Omit it for a registry that takes a token and no name
`password`	string or block	required	A variable, the same way

A service names its own, in the same block on its own entry, because each image is its own pull:

services:
  - name: db
    image: mirror.example.com/postgres:16
    credentials:
      password: $MIRROR_TOKEN

Both spellings are accepted, and from-variable is the one to write — it is how a variable is named everywhere else, and $NAME will be dropped once the documents have moved:

credentials:
  username:
    from-variable: HARBOR_USER
  password:
    from-variable: HARBOR_TOKEN

The document holds the name and never the value. A literal value here is refused rather than taken as a credential: password: hunter2 names no variable, and the refusal says which one it should have been. That is what makes this safe to have in a pipeline at all, since the document is read by everybody who can read the repository, stored on every build made from it, and shown on a page.

The names point at ordinary variables, set and rotated exactly as those held on the server are, at whichever of the four scopes suits: a company registry on the project, a customer's on one plan, a production one on the environment. Mark the password secret and it is encrypted at rest, returned by no page or endpoint, and masked in the job's output. Rotating it reaches the next job rather than the next pipeline change, because the document holds the name and the name does not change when the value does.

Leave the block out and the agent decides. An image with no credentials named is pulled with whatever the agent is configured with: its ~/.docker/config.json, or the registry environment its process was started with. That suits a fleet behind one company registry and needs no pipeline to say so. A public image needs neither.

Naming a variable that does not reach the job fails the job, naming the variable, rather than falling back to the agent's own credential. The two are different registries as often as not, and "pulled as somebody else" is a 401 whose cause is a typo three files away.

On Kubernetes this is not imagePullSecrets, which is what the kubelet pulls the agent's image with. A job's image is pulled later, by the Docker daemon inside that pod. See pulling a private image.

Built-in variables¶

Every step gets these, so a script can identify what it is building. They are applied last, so nothing a pipeline declares and nothing a step injects can replace one.

Variable	Notes
`DUTEST_PLAN_KEY`
`DUTEST_BUILD_NUMBER`
`DUTEST_JOB_KEY`
`DUTEST_BRANCH`	Absent when the plan has no repository
`DUTEST_COMMIT`	Absent when the plan has no repository