Code quality is a profile, not a score.

Trust here is layered, not asserted

A Facet score does not ask you to trust one thing. Four independent layers hold it up, and each stands on its own evidence even while a later one is still maturing:

• The measurement is reliable. Judges extract line-cited evidence; deterministic code scores it. The instrument reproduces at test-retest 0.94 on real code, and results are frozen per content hash.
• The weights are external. Where a dimension carries weights (security today), they are read off SonarQube severities and NVD CVSS, never learned from our own data.
• The weighting was adversarially defended. A 40-agent review attacked every security weight before shipping, and every audit-fatal defect it found was fixed.
• Security fails closed. A detected critical defect caps the score regardless of the good practice around it.

The rest of this page is the evidence behind each layer, limits included. The Security worked example shows all four applied to one dimension, in the open.

There is no universal best

Push runtime performance to its limit and readability usually suffers. Maximise portability and you tend to leave performance on the table. Optimise for prototype speed and you defer tests, observability, and hardening on purpose. None of that is failure. They are tradeoffs, and a good engineer makes them deliberately.

So “is this good code?” is the wrong question. The one you can actually answer is “is this code well-matched to what it is for?” A throwaway analysis script that skips tests is passing, not failing. A security boundary that skips input validation is failing, however fast it runs. Quality only means something once you know the intent.

Why 14 dimensions

Collapse quality into one number and you hide the part you actually need. Facet measures fourteen dimensions separately, so the tradeoffs stay visible instead of averaging into mush. Together they describe the shape of a piece of code, its profile. Each dimension is read as a two-pole bar, and learning to read one takes ten seconds:

From that fourteen-dimension fingerprint, Facet also suggests the profile your code most resembles (a hot path, a public library, a regulated core, a security boundary, and so on). That suggestion is a heuristic closest-match, not a verdict. It can over-fit small or single-purpose files, so you can declare the profile you intend, and the fingerprint is then read against your actual target. Where the code diverges from that profile’s priorities, that gap is the finding.

Three kinds of finding

Violations (harm pole). Negative-polarity indicators that fired on your code. A harmful pattern is actively exhibited, for example dynamic execution of untrusted input, disabled transport verification, or unbounded concurrent spawn. These are concrete, line-citable defects under any profile and are reported as a separate count, never subtracted from the capability score. Absence of a positive is not the same thing as presence of a negative, so the two are never netted.

Internal incoherence. Code that contradicts itself, where a higher-ladder practice sits over a missing lower-ladder foundation. Under compensatory scoring (the 13 formative dims) the satisfied higher rung is credited, and the diagnostic flags it as a possible cargo-cult pattern worth a human glance, not a penalty. Under cumulative scoring (D8 prototype economy only) the cap holds.

Fit gaps. Most other findings are conditional on intent. To reach what this profile asks for, address X. These are suggestions about fit, not universal verdicts. Code that scores low on a dimension your profile does not prioritise is not wrong. It is optimised somewhere else on purpose.

The dimensions are formative, not reflective

The most important methodological finding of 2026-06. In classical measurement theory, a reflective construct is a latent trait that causes its indicators (depression causes each symptom). A formative construct is constituted by its indicators (socio-economic status is composed of income plus education plus occupation, which need not correlate). They use different validity tests and different scoring models.

A within-dimension unidimensionality test on a real-code corpus (inter-feature correlation and first-eigenvalue share) showed 13 of 14 dimensions are formative composites. Features are genuinely independent properties that constitute the construct rather than symptoms of one latent trait. Only D8 (prototype economy) is unidimensional. The dimensions themselves are distinct. No inter-dimension pair has |r| ≥ 0.70 on real code, the ~6-factor scree shows no dominant general-quality halo (first factor only ~28% of variance), and the indicator overlaps a casual reader might predict (readability vs maintainability, robustness vs security) do not translate into co-moving scores.

Consequence for scoring. A cumulative Guttman ladder over independent features mis-scores (it caps at the first unsatisfied rung even when higher rungs are independently satisfied). A graded-response IRT model assumes the very unidimensionality the data deny. Compensatory counting is the theoretically correct model for a formative composite, not merely the cheaper one. The 13 formative dimensions use compensatory. D8, legitimately unidimensional, keeps cumulative.

How a dimension scores. Two sub-scores, never netted

Each dimension reports two numbers kept separate.

• Capability count. Positive indicators marked present. The good things the code does, for example safe boundary handling, reliable cleanup, or appropriately bounded public surfaces.
• Violation count. Negative-polarity indicators marked present. The harmful patterns the code exhibits.

A feature-rich-but-vulnerable file and a minimal-but-clean file are genuinely different and should not collapse to the same number. The five-rung level is preserved as a familiar overall readout (count of satisfied rungs under compensatory, contiguous cap under cumulative), and the two sub-scores are what make the level interpretable.

When every feature of a dimension is not applicable (for example security on a throwaway with no attack surface), the dimension reports no surface (N/A)and is excluded from profile-fit. A security-irrelevant script is not graded as “insecure” for lacking authorisation on code that has none.

Security (D6). Severity-weighted and fail-closed

Security is the first dimension we took beyond a flat count, because it has clean external criteria. Its indicators are weighted by external industry standards - SonarQube rule severities, corroborated by NVD CVSS class means from real CVEs - not by anything learned from our own data, and the two independent standards converge on the tier ranking. It is scored non-compensatory: a single detected critical defect (a SQL-injection or cross-site-scripting sink, a hardcoded secret, disabled TLS verification) fails the score closed regardless of the good practice around it, exactly as a real security review would treat it. Every weight is flagged in the report as confirmed, provisional or contested, so the score never claims more certainty than it has. We attacked the whole scheme with a 40-agent adversarial review before shipping and fixed what it caught.

What is confirmed today: the measurement is reliable and reproducible, the weights are externally grounded and adversarially defended, and on crafted single-issue items the severity-weighted violation ordering matches the independent CVSS ranking where a plain count does not. What is still maturing, and labelled as such: alignment to a panel of practising engineers, and significance of “weighting beats counting” on messy real-world corpora at scale. Read the full worked example to see one dimension investigated in the open, and judge the other thirteen by the same standard.

Reliability. The four-bar gate

A dimension is labelled reliable only when its judge clears four bars on a held-out minimal-pair bundle.

• Surface-weighted agreement ≥ 0.60. The judge tracks a defensible answer key rather than guessing.
• Minority-class recall ≥ 0.50. The rare degradation is caught, not majority-guessed away.
• Schema coverage ≥ 0.80. The judge returns a parseable, usable reading on the large majority of samples.
• Test-retest reproducibility ≥ 0.90. The same code profiled twice yields the same feature vector.

All 14 dimensions clear all four bars on the 2026-06-18 expanded-instrument requalification, each on an open-weight, on-prem-deployable judge. Reliability holds on real code, not just crafted items. A real-code test-retest probe (k=3 over a 50-file corpus) found mean feature-stability 0.938, in the crafted-bundle baseline range.

A minority-recall threshold is what most LLM-judge work omits and the one that matters most. A judge that says “present” to everything scores high agreement on healthy code yet never catches a degradation. Our gate rejected the highest-agreement judge on one dimension (minority-recall 0.40, below the 0.50 bar) in favour of a judge that caught the rare class (0.80). Reliability over raw agreement.

Construct validity. What we have, what we don’t

Reliability says the instrument is consistent. Construct validity asks whether the dimensions actually measure what they claim. We report this separately so the labels stay honest. The evidence today, Lane A, exploratory.

• Content validity. A per-dimension content census against ISO/IEC 25010 plus OWASP-CWE plus 12-factor plus FinOps plus linter rules produced ~127 candidate indicators. The highest-confidence subset (53) was materialised and each was individually qualified with a minimal-pair bundle for judge-discrimination before scoring real data. 53 of 54 cleared the gate. The one that failed was removed, not kept for completeness.
• Dimensionality. No inter-dimension pair |r| ≥ 0.70, ~6 weakly-correlated factors (no halo, first factor 28% variance). 14 distinct indicators that organise into about six families, not 14 fully-independent constructs and not one general-quality factor.
• Known-groups discrimination. Authored high-quality vs degraded exemplars separate monotonically under the production judges on the dimensions whose absolute level was least certain (D1 +2, D5 +4, D6 +2, D9 +5, 4 of 4).
• Convergent anchoring. D6 security features overlap the rules in static analysers (SonarQube, Bandit, Semgrep, CodeQL) and are cross-checked against their output; the D6 severity weights are read off SonarQube rule severities and corroborated by NVD CVSS class means. New D6 violations carry CWE/OWASP identifiers. See the Security (D6) worked example.
• Scoring-model fit. Compensatory plus cumulative is correct by direct evidence about the constructs, not by convention.

Open work (Lane B, confirmatory).Most Lane A studies used a single open judge and a modest real-code corpus (N ≈ 47 to 50). Confirmatory work needs a second independent judge, a larger held-out corpus, parallel analysis for factor retention, formal inter-rater κ, and formal DIF across languages and styles. None of these undercut the claims. They bound how strongly the claims may be stated today.

What this instrument does not do

Defensibility is as much about disclosed limits as positive results.

• It does not measure functional correctness. The judge extracts feature presence, never whether the code computes the right answer. Broken code with all the right machinery can score well on the structural dimensions. Correctness needs an oracle or test-execution gate, a different instrument. We state this plainly rather than pretend the rubric captures it.
• It does not authoritatively infer intent. The per-dimension measurement is the rigorous part. The single closest-match profile is a convenience heuristic over the fingerprint, and it can over-fit small or single-purpose files (a tiny, clean, security-hardened snippet can read as a throwaway because shortness and readability score high). Declare the profile you intend for the exact, profile-relative read; the auto-match is shown only for orientation. A more robust, re-validated inference is active work.
• One judge per dimension. A known, accepted simplification. We mitigate run-noise on the noisier judges with an ensemble (majority vote across N extractions), but we do not run multiple independent judges per dimension. Cross-judge convergent validity is on the Lane B list.
• Floor and ceiling effects are interpreted, not papered over. D10 (observability) and D14 (resource cost) score low on most real code. We verified that this is a true base rate (most code lacks observability instrumentation and paid-resource surfaces), not an instrument defect. The crafted bundles discriminate perfectly. D3 readability runs near-ceiling on real code for the same kind of reason (most code is readable on the standard features). The original D3 features sit at borderline crafted minority-recall, which is queued for ensemble extraction.
• An indicator that could not be measured reliably was removed. One change-amplification feature (cross-site dispersion) failed qualification. The judge marked the violation absent on the variant and present on the clean gold. After one reword it was dropped. We do not score what we cannot measure.

Why we don’t publish the predicates.The exact indicator predicates and per-feature criteria are not on this page. The rubric is the product. The value Facet adds is the curation of indicators across 14 dimensions, qualified for judge-discrimination and bounded by honest scope, and publishing it would mean publishing the product. The instrument’s integrity also depends on the predicates not becoming a gaming target. Independent third parties can audit the instrument under collaboration agreement (Lane B confirmatory programme, in progress). The published methodology, gate thresholds, dimensionality numbers, and known-groups discrimination scores are what we offer publicly. If you need predicate-level detail for academic or audit purposes, get in touch.

Three checks you can run without trusting us

The predicates are private; your ability to audit a report is not. On any report, today:

• Test the determinism. Profile the same file twice. The fingerprint comes back identical, because deterministic code does the scoring and results are frozen per content hash. A tool that quietly re-rolls its opinion cannot pass this check.
• Check the citations. Every finding names the lines it stands on. Read them and disagree in specifics; the per-dimension feedback controls exist for exactly that argument.
• Try to fool it. Name a function sanitize_input without sanitising, or wrap a harmful pattern in reassuring comments. “Claims are not evidence” is enforced structurally, and the judges were qualified on traps like these.

What you cannot verify from the outside, the indicator enumeration, the exact weights, the gate thresholds, is precisely what the under-agreement audit covers. The split is deliberate: everything a report asserts about your code is checkable by you; the instrument behind it is checkable by named auditors.

How the measurement works (the engine)

• The model finds, the code decides. A model reads your code and extracts atomic, line-cited features (present, absent, or not applicable, with a one-clause basis). It never assigns a score. Deterministic code, version-controlled and frozen per release, computes the score from the feature vector, so two runs of the same code agree; independent parties can audit that code under the collaboration agreement described above.
• Claims are not evidence. Every prompt enforces that names, comments, and docstrings are claims, not evidence. A function named sanitize_input that does not sanitise is absent. This is the explicit defence against the dominant LLM-grading failure mode, which is being fooled by reassuring surface language.
• One judge per dimension. No single model is best at everything. Each dimension is routed to the model that proved most reliable at measuring it.
• Chunked extraction where the prompt got too long. Five dimensions (D1, D7, D11, D12, D14) split their feature list into two extraction calls and merge the result, because the indicator expansion enlarged their prompts to the point where per-feature attention diluted. The chunking restored minority-recall on each of those dims. The fix was measured and validated before shipping, not assumed.
• Provider-pinned and seeded judge calls. Different OpenRouter backends of “the same” model can give different greedy outputs. We pin one provider and pass a fixed seed so the extraction is as stable as the provider allows, then majority-vote across N extractions to mop up residual flicker.
• Reliable, never silently degraded. All fourteen dimensions currently clear the four-bar gate before shipping to production. If a future model swap ever dipped a judge below the bar on real data, that dimension would be pulled from production and re-qualified before it shipped again - never kept in the profile and quietly mislabelled.
• Your code is never stored. We keep secret-scrubbed measurements and a content hash, scoped to your account. The source itself is never written down.

Profile your codeView your reports