Staged rollouts by group
A rollout can go out in stages, and a stage can be a group of hosts. The usual shape
is two groups, staging and production: an update goes to staging first, soaks
there for a while, and only goes to production when somebody says so.
1. Create the groups
Section titled “1. Create the groups”On the Groups page in the portal, create a group called staging and one called
production. Admins and owners can create groups. Put hosts in each:
- by hand, by ticking them under Hosts, or
- by label, with a selector such as
env=staging. Every host carrying those labels is in the group, including hosts that enrol later.
Policies refer to groups by name, so create the groups before the policy. A
policy that names a group your organization doesn’t have is refused when you save it,
with the place it names it, for example
rollouts.staged.stages[0].hosts.groups: there is no group called "stagin".
2. Write the policy
Section titled “2. Write the policy”name: staging-then-productionpriority: 150
selector: groups: [staging, production]
rules: - match: { kind: security } action: auto_merge rollout: staged
- match: { kind: [patch, kernel] } action: propose approval: { required: true } rollout: staged
rollouts: staged: stages: - name: staging hosts: { groups: [staging] } soak: 24h - name: production hosts: { groups: [production] } approval: required halt_on: failed_jobs: 1 failed_health_checks: 1The portal has this as the Staging, then production example on the new-policy page.
rollouts.stagedis the strategy, and a rule uses it withrollout: staged.- Each stage takes the hosts its
hostsselector matches. A stage takes only hosts that no earlier stage took, so a host in both groups goes out with staging. soak: 24hwaits a day after the last staging host finishes before the rollout moves on.approval: requiredon the production stage means it does not start on its own. It waits for somebody to promote it.halt_onstops the whole rollout at the first failed job or failed health check, so a fix that breaks staging never reaches production.
What happens
Section titled “What happens”- The proposal is approved, or merges itself under an
auto_mergerule, and the staging hosts get their jobs. - When every staging host is done, staging soaks for 24 hours.
- When the soak ends, the rollout shows awaiting promotion. Nothing is sent to
the production hosts yet. A
rollout.awaiting_promotionnotification goes to any channel whose notification rules include it, and the rollout is listed first on the Rollouts page. - An operator, admin or owner opens the proposal and presses Promote production on its Rollout card. Production starts.
While a stage is soaking, Promote now on the Rollout card ends the soak early and starts the next stage straight away. If that stage needs approval, the person pressing it counts as the approval, so the portal asks first.
Halting, resuming and cancelling work as for any rollout. A rollout halted while it waits for promotion goes back to waiting when it’s resumed. It doesn’t start production.
Hosts in neither group
Section titled “Hosts in neither group”Nobody is left behind. Whatever the last stage selects, any host the proposal
covers that no earlier stage took goes out with the last stage. Without the
selector above, a host in neither group would be updated together with production.
The selector: { groups: [staging, production] } keeps the policy to the hosts in
those two groups, so hosts in neither are left to your other policies.
A stage whose group has no hosts is kept, empty, rather than dropped. If staging is
empty, production still waits for somebody to promote it.
Rules about the gate
Section titled “Rules about the gate”approval: requiredcan’t go on the first stage. Approving the proposal is what starts the first stage, so a gate there would ask twice.- Promoting needs the same role as halting and resuming a rollout: operator, admin or owner.
Renaming or deleting a group
Section titled “Renaming or deleting a group”Because policies name groups, renaming or deleting a group that a policy names is refused, with a list of the policies that name it. Change those policies first, then rename or delete the group.