kodebeat / papers / cost

Pricing CI toil in euros so it can compete for budget

A CI/CD toil analyser that prices the waste in euros.

Identifies six toil signals in CI history, manual re-runs, flaky recoveries, dispatches, queue stalls, failure triage and approval gates, assigns each an engineer-time cost at a loaded hourly rate, and prices wasted runner minutes separately. Every assumption is a CLI flag, because an audit is only as credible as its assumptions are defensible.

The problem

A team knows its CI is bad. Everyone knows. The evidence is a shared feeling that the pipeline is always red and someone is always pressing re-run.

The request to fix it fails, every time, in the same way. Fixing flaky tests is a week of work with no feature attached. It competes with a roadmap, and it loses, because the case for it is made in adjectives: it's frustrating, it slows us down, it's demoralising. All true, none budgetable. The person deciding hears a complaint rather than a proposal.

Meanwhile the cost is real and continuous, and it is invisible because it is distributed across a hundred small interruptions nobody records:

  • Someone presses re-run and waits — perhaps ten minutes of attention, most of it lost to the context switch rather than the waiting.
  • A build fails on a commit and passes on the same commit later with no new push. Somebody investigated first, and that investigation happened before anyone knew it was flaky.
  • A workflow is dispatched by hand because the automation does not cover a case.
  • A run queues for twenty minutes. The engineer switches to something else and switches back: two context switches, for one build.
  • A pull request is parked at an approval gate until someone clicks.

None of this appears in a metric. It appears in a person's day, and it is never totalled.

The method

Name six signals, and price each in engineer time:

Each is detected from a field that already exists in the API response. There is no instrumentation to add and no agent to install — the data has been accumulating all along.

Price it, and make the rates flags. Engineer cost uses a loaded rate, default €75/h (--rate). Wasted runner minutes (failed runs and repeat attempts) are priced separately at a GitHub-hosted runner rate, default €0.0074/min (--runner-rate). That default is about $0.008, the Linux two-core price before GitHub's January 2026 cut to $0.006/min, so set it to the current one. The README gives the reason: the audit is only as credible as its assumptions are defensible. A finance director who disputes €75/h can be handed --rate 85 and the same analysis recomputed in front of them. The per-signal minutes and the fifteen-minute queue threshold are constants in toilaudit/signals.py; detect_signals() accepts overrides, but the CLI does not expose them, so changing them means editing the module.

Count triage once per broken commit, and price it from the timeline. A commit that breaks eight parallel jobs produces eight red checks and one investigation. A naive model would count eight and inflate the total by a factor a sceptical reader would spot, and one inflated number discredits the whole audit. The one charge is the interval from the red run to the next push on that branch by a human, floored at one minute and capped at eight. A red run nobody pushed after, or one a bot pushed over, costs one minute. When the same commit subject lands in three or more repositories within thirty minutes, the copies after the first are billed one minute each: one diagnosis, applied several times.

Read the platform as it is, including where it is limited. GitLab has no run-attempt counter. A retry re-runs the failed jobs inside the same pipeline, and running the pipeline again creates a new pipeline on the same commit, so attempts are derived by grouping pipelines on (project, sha, ref) in creation order, which catches the re-runs but not the in-pipeline retries. (The README describes a retry as creating a new pipeline; the grouping catches that case either way.) The list endpoint also omits started_at/finished_at, so queue time reads as zero rather than being invented, with a pointer to the per-pipeline endpoint for real figures. Estimating it would inflate the number in a way nobody could audit.

Produce a document for a meeting. The output is Markdown: the bottom-line euro figure, a per-signal cost table, a monthly trend, the costliest workflows (and, across repositories, the costliest shared workflow files), and sample incidents to point at. A total plus three concrete examples is a different argument from a total alone.

The headline results

  • Signals are detected from fields the platform already records
  • Tests.
  • Two cost bases, kept separate.
  • What the defaults rest on.

The full results, with the tables and the reasoning behind each figure, are in the PDF.

What is in the PDF

  • The problem — what goes unanswered without it, and who notices first.
  • Why the obvious alternative falls short — stated plainly, including where it is the better choice.
  • How it works — the method, not a feature list.
  • Concrete use cases — with console output quoted from the repository, never reconstructed.
  • The methodology behind any number it emits — every term shown, so the figure survives a question.
  • What it deliberately does not do — the section most papers leave out.

Part of the Cost & FinOps theme.

Get the PDF

This paper already selected; the download starts when you send the form. No follow-up sequence.

Get the PDF