Skip to content

Your AI Security Maturity Score Is Probably Inflated

Your AI security maturity score is probably inflated: a score of 8.4 crossed out and replaced with 3.1
Contents

The Stage 3 Slide That Falls Apart

The AI security maturity slide is green across the board. Each pillar has a stage number beside it, and the leadership review is moving quickly. Then someone stops on one box and asks a simple question: “Can we see the evidence behind that score?”

What comes back is a vendor logo on a PDF, a policy that was drafted but never approved, or a control everyone believes is active but nobody can show firing. The number remains on the slide, but the room’s confidence in it disappears.

Rock Lambros describes this problem in his RockCyber review of the SANS AI Security Maturity Model. The failure is usually structural, not dishonest. Self-assessment rewards optimism. Your team remembers implementing a capability, so “we do that” quietly becomes “we have that.” Without a gate between performing a practice and proving it, your AI security maturity score rises one generous judgment call at a time.

An honest Stage 2 you can defend with evidence is a stronger security position than a claimed Stage 3 that collapses the first time someone asks for proof. In an evidence-based maturity model, the stage you can prove is the only stage you actually hold. Everything above it is a forecast, not a posture.

This is not an argument against maturity models. It is the logic of a good model working as intended. SANS makes the same point in its launch material: a smaller organization operating at a genuine, evidence-based Stage 2 is more secure than an enterprise claiming Stage 3 without documentation.

The standard is simple. Before a capability raises your score, you should be able to produce the artifact that proves it exists and works. If you cannot, it may be planned, partially implemented, or informally practiced. It is not yet a posture.

The Two Rules That Stop You Inflating

Good maturity models assume your self-score may be optimistic and make you prove otherwise. The SANS model builds evidence into its scoring design, while Rock Lambros’s RockCyber analysis sharpens that design into two practical rules for evidence-based security maturity.

Rule one: weakest-link capping. Your overall stage cannot float far above your weakest pillar. As Lambros reads the model, your score cannot sit more than one level above your weakest pillar or exceed your Govern score by more than one stage.

This makes maturity model scoring work like a floor rather than an average. Strong detection capabilities cannot compensate for governance that remains at Stage 1. Your real maturity is the weakest load-bearing wall, not the best-looking room. That prevents one advanced capability from masking a program that cannot support it consistently.

Rule two: the evidence ceiling. Lambros uses this term for the mechanism that caps capabilities you cannot document. On his reading, a capability claimed without supporting evidence cannot raise your score beyond the lower stages, regardless of how mature the underlying practice feels.

The evidence ceiling is a scoring rule that caps any capability you self-report without documentary proof. Without an approved policy, an audit log, or a dashboard export to back it, the capability does not count above the lower stages, no matter how confidently you assert it. Assertion is not evidence, and an evidence-based model writes that standard into the score so you cannot talk past it.

Anyone who has been through an audit recognizes the principle. What your team believes about a safeguard matters less than what you can produce, dated and current, showing that the safeguard worked on a real event. The evidence ceiling brings that discipline into a self-assessment, where optimism would otherwise go unchallenged.

The ceiling is a prompt, not a punishment. Before you claim a stage, you look for the document, log, or export that supports it. As Lambros’s analysis makes clear, the search often provides the honest answer. The artifact may not exist, may never have been approved, or may describe a system that has changed since the pilot.

You did not lose a stage when you discovered the gap. You learned which stage you could defend before a customer, regulator, or incident review asked you to prove it.

What Counts as Evidence, Domain by Domain

The evidence standard is easy to endorse and harder to apply. “Show documentary proof” is a principle. It becomes useful in an AI security maturity assessment only when you can identify, domain by domain, which artifact proves the capability and which assertion does not.

The table below applies that principle to agentic controls. One question governs every row: Can you produce the artifact right now, dated and current, without asking someone to build it first? If you cannot, you are not yet at the stage shown on the slide.

Control domainAssertion (does not count)Artifact that counts as evidence
Agent and MCP inventory“We know what agents and MCP servers are running.”A current, dated inventory of every agent and MCP server in production, with owner, purpose, and the tools each one can call, reconciled against what is actually deployed.
Least-privilege scopes per agent“Our agents only have the access they need.”The per-agent scope definitions showing which credentials, data, and tools each agent can reach, plus evidence the scopes are enforced, not just documented.
Tool-call logging and approval trail“High-risk actions require approval.”Logs of agent tool calls that show what was requested, what was approved or denied, by whom or by what policy, and that the approval gate actually fired on a real high-risk action.
Third-party agent tooling review“We review skills and tools before we install them.”A written review process with a record of specific tools assessed, what was checked, who signed off, and the decisions made, before those tools reached production.
Security review of AI-generated code“AI-written code goes through security review.”Evidence that a security review step ran on AI-generated changes: the findings it produced, what was remediated, and the gate that blocks merge when it fails.
Continuous posture re-assessment“We reassess our posture regularly.”A trail showing the posture was re-evaluated after real changes (a new agent, a model swap, a new tool), with dated results, not a single assessment from the day the program launched.

The pattern matters more than any individual row. Your assertion describes how you believe the program behaves. The artifact records it behaving that way. The distance between the two is what the evidence ceiling catches. Your inventory may live in three people’s heads, your access limits may have been correct when they were written, or your approval gate may never have been tested on a real action. Each practice can be true in spirit and still be impossible to defend.

The OWASP Top 10 for Agentic Applications identifies the risk categories these artifacts are meant to show you have addressed. For third-party tools and AI-generated code, a review produces usable evidence only when you record what was examined, what risks were identified, who approved the decision, and what happened next. That is why threat modeling belongs in the review record, not only in the team’s memory.

Most teams can produce two or three of these artifacts and need to build the rest. That is not a failed assessment. It is an honest picture of your current stage and a concrete list of what you need to create next.

Why Agentic Evidence Has to Be Continuous

For autonomous systems, evidence has a shelf life. An agent changes behavior with every model update, prompt change, and new tool it connects to, so an audit binder assembled in the spring can describe a system that no longer exists by the summer. Point-in-time proof of agentic security is not wrong, it is perishable, which means the evidence a maturity score rests on has to be produced continuously to stay true.

A conventional maturity artifact is a snapshot: an approved policy, a quarterly assessment, or a dashboard export captured on a particular day. That evidence holds up as long as the control and the system it describes have not changed. Snapshots are not the problem. The problem is that agentic systems can change faster than the assessment cycle designed to evaluate them.

SANS provides useful language for this problem with its Principle of Least Agency, the agentic counterpart to least privilege. Once a system can act on your behalf, the security question is no longer limited to what the system knows. You also need to know what it is permitted to do, which tools and data it can reach, what actions it actually took, and whether the expected controls fired.

That shift is central to agentic AI security. Imagine that your last assessment documented a support agent with permission to read tickets and draft responses. A month later, someone connects a new MCP server that lets the agent update customer records and issue refunds. The original assessment may have been accurate when it was completed, but it no longer proves the agent’s current posture. The system changed, while the evidence did not.

The same problem appears after a model update, prompt revision, new credential, modified approval policy, or additional tool connection. Each change can alter what the agent can access, how it behaves, or which safeguards apply. If your evidence is produced only during quarterly reviews, your AI security maturity score may describe a control environment that disappeared weeks ago.

Continuous evidence closes that gap. Your inventory should update when an agent or connector is added. Your permission records should reflect scope changes. Your tool-call logs should show whether approval gates fired. Reassess your posture after the change, with dated results that show what was tested and what changed.

Trent AI uses multiple agents in one continuous loop to scan, judge, mitigate, and evaluate your AI systems as they change, producing the evidence trail as a byproduct of running the security program instead of assembling it later for a review.

Earning a stage once is not enough if the system keeps changing. You stop defending the score you earned last quarter and start reporting the one you can prove today.

Make Your Evidence Continuous

Trent scans, judges, mitigates, and evaluates your agents on every change, so the evidence regenerates when the system does. New accounts are approved within about 48 hours.

Request access

How to Score Yourself Honestly This Week

Start with the test, not the framework. For each row in the table above, try to produce the artifact right now, from your own desk, without asking anyone to build anything. If you can put your hands on it, dated and current, you have evidence for that stage. If you cannot, you have an assertion, and the honest score drops. That hour is the most valuable one in any AI security maturity assessment you run this year.

None of this is a reason to beat yourself up. A defensible lower stage wins wherever the number gets tested: a customer security review, a regulator’s question, an incident post-mortem, a board slide with a skeptic in the room. The artifact-backed number survives all four. The optimistic one fails at the worst possible moment, when you are already explaining something else.

So score true, then close the gap on purpose instead of under pressure. A maturity score is only useful as a work queue, and the queue writes itself once you separate the assertions from the artifacts. The fastest way to raise your real maturity is to pick one row where you have the assertion but not the artifact, and build the artifact this week.

Reviewed by Eno Thereska, Co-founder & CEO at Trent AI

Score Your Program Against Evidence

The ASMM scores your AI security maturity across six NIST CSF 2.0 domains so you can see which stage you can actually defend.

Take the ASMM assessment

FAQs

What does “evidence ceiling” mean?

The evidence ceiling is a scoring rule that limits any capability you self-report without documentary proof. Rock Lambros uses the term in his RockCyber review of the SANS AI Security Maturity Model to express a simple standard: assertion is not evidence. Without an approved policy, audit log, dashboard export, or comparable artifact, the capability cannot support a higher maturity score.

Is a claimed Stage 3 worse than an evidence-based Stage 2?

Yes, wherever the score is tested. A provable Stage 2 can withstand a customer review, audit, regulatory question, or incident investigation because each claim has evidence behind it. An unsupported Stage 3 collapses when someone requests that evidence. SANS uses the same framing: a 30-person organization at an evidence-based Stage 2 is better positioned than an enterprise claiming Stage 3 without proof.

What counts as evidence for AI security maturity?

A record counts as evidence, not a belief about how your program operates. Examples include a reconciled agent inventory, enforced per-agent access scopes, tool-call and approval logs, documented reviews of third-party tools, security-review results for AI-generated code, and dated reassessments after material changes. See the evidence table for the complete assertion-versus-artifact comparison.

Why does AI security evidence need to be continuous?

Because agents can change with every model update, prompt revision, permission change, and new tool connection. A point-in-time artifact may accurately describe the system on the day it was produced and become outdated after the next change. Continuously produced evidence keeps your documented posture aligned with what agents can access, what they are allowed to do, and how they behave now.

By Trent AI Team

Trent AI is your AI Security Engineer: it maps what you're running, finds what's actually exploitable, and proposes prioritized fixes that fit your system. Our team writes about AI security, secure-by-design development, and building trust into software from day one. Learn more about who we are on our About Us page.

Related posts