Reviewing an architecture against a dozen vendors' frameworks at once

The problem

An architect is asked to review a system before it goes live. There is a well-architected framework for this, and the trouble starts immediately: there are twelve of them.

AWS published the original in 2015 and everyone else followed. Microsoft has one for Azure and a separate one for Power Platform. Google, IBM, Oracle and Alibaba each have theirs. So do Salesforce, Snowflake, Databricks, HashiCorp and GitHub. At that point "well-architected" stopped being a cloud infrastructure idea and became a general form for publishing architectural guidance.

Nobody's estate matches one of them. A system might run on one cloud, ship through another vendor's CI, provision with a third vendor's tooling and keep its data in a fourth's warehouse. Reviewing it means reading four or five frameworks, and reading them in sequence produces three specific problems.

They overlap without agreeing. Every framework has a security pillar and they cover mostly the same ground in different words with different structure. Read five in a row and you answer the same question five times, phrased differently each time, without ever being sure whether the differences are substantive or editorial. There is no way to tell, from inside any one document, which of its checks are unique to it.

Their vocabulary diverges. One framework's Operational Excellence is another's Efficient Operations is another's Define and automate processes. A finding recorded under one name is invisible to somebody searching under another, and a review that spans two frameworks accumulates two parallel vocabularies with no defined relationship between them.

Assessment tooling is per-vendor and stays in the vendor's console. The AWS Well-Architected Tool is free, good, and produces a record inside AWS. It has nothing to say about the parts of the system AWS does not run, and there is no view that puts both in the same report. So the multi-vendor review, the kind most architects do, ends up in a spreadsheet somebody built themselves, with the frameworks retyped by hand and already drifting from their sources.

The result is predictable. The review is scoped down to the one framework whose vendor is dominant, or it becomes a fortnight of reading, or it does not happen. Most often it becomes a slide that says "we followed the Well-Architected Framework", which is a claim nobody can check.

The method

One binder, twelve frameworks, indexed like a reference work. Each framework gets a sheet code (WAF-01 … WAF-12) and carries the same fields: publisher, scope, structure, extensions, and how that vendor's own assessment works. Uniform metadata is what makes twelve heterogeneous documents comparable at a glance, and the sheet-code index is what makes a finding citable — WAF-06, reliability pillar is a reference somebody else can look up.

Seven universal themes, and one of them exists because the form outgrew the cloud. Every pillar is mapped to one of:

A pillar crosswalk, so the frameworks can be read against each other. The Crosswalk view groups every vendor's pillars by universal theme. "Read five security pillars in sequence" becomes "read the security theme once and see where the five differ". The places where frameworks agree can be answered once, and the places where one framework asks something the others do not are where the extra reading is worth doing.

A check-level cross-reference beneath it. Theme mapping says which pillars to read together; it does not say which individual checks ask the same thing. The page also carries 52 strict equivalence groups over the checks themselves, so ticking one member of a group can set the others and the unified checklist collapses each group to one row. That layer, its strictness rule and the propagation rules are the subject of the crosswalk paper.

A unified checklist that runs every in-scope framework in one pass. Rather than working through twelve sheets, the unified view presents all 395 checks organised by theme, so a review is one traversal instead of twelve. Each check carries its title and a one-line rationale explaining why it matters, so the checklist is readable by somebody who has not read the underlying framework.

Stamps are earned, not awarded. A pillar earns its stamp only when every check inside it clears. There is no partial credit and no score to average, because a percentage across a security pillar averages a missing control against a satisfied one and hides which is which. Most pillars therefore stay unstamped for a long time, and the radar looks discouraging. That is the accurate picture of most architectures under review.

The gap register is the deliverable. The gaps view lists open items by theme with a count per theme. This inverts the usual emphasis: a review tool that foregrounds what has cleared produces a document for reassuring people, and one that foregrounds what is open produces a work list. The register is what goes into the backlog.

A radar for the shape, a board for the estate. The readiness radar plots completion by universal theme and answers "where are we thin" in one glance. In the author's reviews so far it has shown the same shape each time: strong on security, thin on sustainability and operations. The review board covers multiple workloads at once, with per-framework progress across the in-scope estate, because an organisation reviews systems rather than a system.

Scope is a decision the tool asks you to make. The review view begins by selecting which frameworks are in scope for this workload. A system with no Salesforce should not be answering Salesforce's checks, and forcing all twelve through every workload would generate a gap register full of items that do not apply. That is the fastest way to make a register worthless.

Everything exports, and imports back. Review reports download as JSON and copy as Markdown; a review can be re-imported later, either merging into the current state or replacing it. Binder and per-workload exports exist separately. The merge path makes a review resumable across people and across quarters, and that resumability separates a control from an exercise.

Every pillar links to the vendor's own page, and the header disclaims the compilation. The status line says the binder was compiled from each vendor's published framework at a stated date and that sources should be verified before formal reviews. The links make verification a click, and the disclaimer says that a compilation is a secondary source. Where that single disclaimer falls short, and how the compilation grades its own rows, is the subject of the provenance paper.

The headline results

• How the counts were taken.
• The theme mapping is an editorial judgement and is visible as one.
• Checks carry rationale, not just titles.
• Trademark handling is explicit.

The full results, with the tables and the reasoning behind each figure, are in the PDF.