Description
Most organisations' security posture was never in the code. It was in people. The engineer who knows tokens do not go in local storage. The architect who remembers why internal services sign their own requests. The threat model that was discussed once, in a room, and written down nowhere. That worked while every line passed through hands that had absorbed the unwritten rules through onboarding, review and proximity.
A model has none of that. It has no onboarding, it overhears nothing, and its familiarity with your conventions is whatever it can read. Research on what it can read is unkind: a company's internal conventions sit outside the distribution the model was trained on, so it produces code that looks plausible and does not match your patterns, your helpers or your architecture. If three approaches to the same problem are present in the codebase, one secure, one legacy and one half-migrated, all three are examples and none of them is marked.
The dangerous part is not the output. It is the confidence around it. In a controlled study participants with an assistant wrote less secure code on most tasks while reporting more confidence in it, and surveys since have found the same shape at scale: most developers do not fully trust generated code, and a minority actually verify it. Distrust is a belief. Verification is an action. Without a gate, the gap between them widens with volume.
So the useful exercise is uncomfortable and short. Pick a security assumption your team genuinely holds, then ask where in the codebase an agent would learn it. If the answer is nowhere, the vulnerability is not the tool. It is the silence, and this dimension scores how much of it there is.
The five levels
One question decides which rung you are on: does the system itself act on this, or does a human have to choose to? Each rung carries a tell: the thing that would settle it, so the score is something you checked rather than something you agreed with.
- 1
Blind
Security expectations exist as instincts. Nobody has written down what must be true, so there is no list to check code against, and different parts of the system take different approaches to the same question without anyone having decided which is correct.
The tellSearch your codebase for how a request gets authenticated and count the distinct patterns you find. More than one, with nothing anywhere saying which of them is correct, is the level.
- 2
Tacit
A few people hold the model. They know which endpoints handle sensitive data, which pattern is the approved one, and they catch deviations in review when they get to them. It is genuine expertise and it is entirely personal. Whatever they do not review keeps whatever pattern it was given.
The tellLook at who reviewed the last changes to your authentication or data-access code. If the same one or two names appear, and nothing routes work to them automatically, the security is a person.
- 3
Visible but inert
The rules are written: a security policy, a threat model, a page of approved patterns. Nothing enforces them. Code that contradicts the document passes review as often as not, several competing patterns remain in the codebase, and an agent reading that codebase finds no signal about which is the right one.
The tellSearch your codebase for the pattern your policy forbids. If you find live examples, and no build has ever failed over one, the policy is describing an intention.
- 4
Acted on
The critical assumptions are machine-enforced. There is one canonical way to authenticate, to reach data, to handle a secret, and something in the pipeline refuses a change that does otherwise. The rule sits where both a new joiner and an agent will meet it: they find it by breaking it.
The tellFind the last change that violated a critical security assumption. Something failed the build over it, and the failure named the rule, rather than a reviewer catching it on a good day.
- 5
Durable
Enforcement keeps up with the system. New services arrive inside the same checks, the superseded patterns are deleted rather than left in the codebase as tempting examples, and the assumptions are revisited when the threat model changes rather than at audit time.
The tellSearch for the pattern you deprecated two years ago. If there are no live examples left to copy, and new repositories start inside the same scans, the enforcement is keeping up.
What this dimension rests on
The five levels are observed, from client engagements, and no paper produced them. What the research below establishes is why this dimension is worth scoring at all. No links: the citations carry enough to find them, and they stay findable after a link would have rotted.
Perry, Srivastava, Kumar and Boneh, Do Users Write More Insecure Code with AI Assistants? (ACM CCS 2023)
Ran a controlled study and found participants with an assistant wrote less secure code on most tasks while reporting more confidence in it. Confidence and security moved in opposite directions.
Pearce, Ahmad, Tan, Dolan-Gavitt and Karri, Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions (IEEE Symposium on Security and Privacy, 2022)
Generated code across a wide set of security-relevant scenarios and found a substantial share carried known weakness classes. The model reproduces the patterns it was shown rather than reasoning about a threat model.
Where this came from
This dimension has a published article behind it, written from client engagements rather than from a greenfield repository.
Read the article on Security assumptions →
How to climb this dimension is not written yet. It is the next thing this page grows, and saying so is more useful than a paragraph of generic advice.