kodebeat / papers / greenops

Making CI carbon visible, then enforceable

Carbon-badge and sci-disclose's budget gate.

CI is the most easily optimised compute an organisation controls and the least examined. carbon-badge prices thirty days of job-minutes at each runner's power draw and labels every wattage measured, declared or guessed, because the gap between guessed and declared can be a factor of nineteen. sci-disclose's budget gate then fails a pull request when the carbon of a declared deployment exceeds a budget you set, and without the budget it reports and cannot fail anything.

The problem

A nightly integration job runs at 03:00 on a self-hosted 32-core builder. It takes ten hours, it has never failed, and nobody has looked at it since it was merged. It uses more energy than every pull-request check in the repository put together, and no report anyone reads says so.

CI is invisible because nobody owns it as infrastructure. A busy repository runs matrices, pull-request checks, scheduled scans and release pipelines. Cloud carbon dashboards cover the infrastructure you deployed, not the runners your workflows rented. The platform reports minutes used for billing, which is a number about money on a page nobody opens for environmental reasons. The nightly job goes unnoticed because it is quiet.

Deployments are invisible because drift has no enforcement point. A service is deployed with two replicas, 250m CPU and 512Mi. Six months later it runs eight replicas at 1000m and 2Gi. No single pull request did that. Each step was a reasonable response to a real pressure (a latency spike, a noisy neighbour, a launch), and each was approved by somebody with no visibility into the aggregate, because nothing in the review showed it. By the time anyone measures, the drift is a fact of the environment, and undoing it means proving retrospectively that each increase was unnecessary against a team that remembers why they made it. That argument is unwinnable, which is why resource footprints ratchet in one direction only.

And there is a failure that applies to both: a carbon figure without provenance cannot be checked. Somebody puts "1.2 kgCO2e/mo" in a README. A reader has no way to know whether that came from real measurements, a published hardware specification, or a fallback constant applied to a machine nobody recognised. All three render identically.

The method

Name the provenance of every wattage, in three words used everywhere. carbon-badge's central mechanism:

A recognised GitHub-hosted label (ubuntu-latest, macos-14, …-8-cores, …-arm, …-gpu) counts as declared, because its specification is published.

Say that declared is believed, not verified. The README states the trust model: "We trust you over any inference — you know your hardware and the API does not expose it. But nothing checks the number, so a wrong declaration produces a confidently wrong badge. Only measured closes that gap."

Make the cost of guessing concrete rather than warning in the abstract. Nine two-minute ubuntu-latest jobs plus one ten-hour job on an unrecognised my-builder:

                nine short jobs       the long one              total
guessed    0.3 h total × 9.4 W  +  10 h × 9.4 W  =   97 Wh  ->   46 gCO2e/mo
declared   0.3 h total × 9.4 W  +  10 h × 180 W  = 1803 Wh  ->  865 gCO2e/mo

Figure 1 — The same CI, nineteen times apart. The fallback does not introduce a small inaccuracy; it silently prices a big machine as a small one, which is why an unrecognised runner is named in the log with the exact flag that fixes it rather than quietly absorbed.

Report the share of energy, not the share of jobs. One long job on a big machine can outweigh a hundred short ones, so counting jobs would direct optimisation at the wrong workflow.

Keep the deployment model small enough to argue with. All of carbon-budget-action's constants live at the top of carbon_budget.py:

gCO2e = (cores × 4 W + GB × 0.4 W) × hours ÷ 1000 × PUE 1.2 × grid-intensity

A reviewer who disputes the figure can read the arithmetic in a minute and change it. The cost is accuracy: the model ignores utilisation, storage and network, and a more faithful one would be harder to audit.

Ship a mode that cannot fail the build. mode: report posts the estimate and always passes. Teams adopt in report mode, watch for a few weeks, learn what their footprint looks like, and then set a budget they believe in. A gate that arrives pre-configured with somebody else's threshold gets disabled in week one.

Make grid intensity an input. The default is 480 gCO2e/kWh, the world average; eu-north-1 is about 25. Same workload, a nearly twenty-fold difference, which makes region choice visible in the same check, and a team that moves a workload to a cleaner region sees the reward immediately rather than in an annual report. The figure can be typed in, or fetched live per run: ci-api-area reads the last hour from the keyless API described in Acting on grid carbon intensity, and em-zone with em-token reads Electricity Maps.

The headline results

  • carbon-badge: duration × watts, summed over thirty days.
  • carbon-budget-action: CCF-derived coefficients, reduced to what a manifest can supply

The full results, with the tables and the reasoning behind each figure, are in the PDF.

What is in the PDF

  • The problem — what goes unanswered without it, and who notices first.
  • Why the obvious alternative falls short — stated plainly, including where it is the better choice.
  • How it works — the method, not a feature list.
  • Concrete use cases — with console output quoted from the repository, never reconstructed.
  • The methodology behind any number it emits — every term shown, so the figure survives a question.
  • What it deliberately does not do — the section most papers leave out.

Part of the GreenOps theme.

Get the PDF

This paper already selected; the download starts when you send the form. No follow-up sequence.

Get the PDF