Every AI security vendor is naming models. Parameter counts, frontier releases, benchmark scores. It’s the wrong argument. The model is the least differentiated part of an AI SecOps platform — and the part that has the least to do with whether the output is any good.
Bigger Models Don’t Make Better Analysts. Better Scaffolding Does.
There is a measurable gap between two deployments of the exact same model. Not two different models. The same weights, the same parameter count, wrapped in two different harnesses — the programmatic layer that decides what the model sees, what it’s allowed to do, how its output gets fed back in, and when it stops.
“You can take the same exact model, run two different scaffolds around it, and get very different results,” says Greg Scasny, CTO and Founder of IntelliThreat. “One harness lands around 46% accuracy. Change the harness, same model, and it jumps to 80%.”
That is the entire argument. A frontier model with no scaffold is a chatbot. A modest model inside a disciplined harness is a level-one analyst that works 24/7.
Four Eras of AI in Security, and Only One of Them Matters Now
- 2023 — the chatbot. Open a browser tab, paste in a log, ask a question. Most people still use AI this way.
- 2024 — the copilot. AI sitting next to your environment, answering questions about data you hand it. Still copy-and-paste. Still a human driving every step.
- 2025 — agentic AI. Agents that take actions somewhat autonomously. Real progress, and real risk, because autonomy without constraint is just a faster way to make a mistake.
- Now — the scaffold. The programmatic operations around what the agents and the models are permitted to do. This is where accuracy, safety, and cost actually get decided.
Most vendors are selling era three and naming era one’s models. The work that produces a trustworthy verdict happens in era four.
What a Harness Actually Does
Not a prompt. Not a system message. A program.
Bounds the input. The harness decides what context the model gets — environment baselines, known-good IP ranges, normal geography, documented business hours, prior activity for that user. A model reasoning without that context is guessing.
Chains the questions. One pass is rarely enough. The harness takes the model’s output, reformulates it into the next query, and keeps narrowing until it reaches a defensible conclusion — without a human writing each follow-up.
Constrains the output. The harness enumerates the actions the model may select from. It cannot invent a new one. This is what makes Autonomous Response safe to run in production rather than a liability.
Logs every step. Reasoning, root cause, action taken, timestamp. Stored for the full audit trail, not summarized away.
Why This Changes the Economics Too
If accuracy comes from the harness rather than the model, you don’t need a frontier model to do serious work on security data. IntelliThreat runs its own models on its own hardware — including small models on cyber threat analysis nodes at the edge of the customer environment — because the scaffold is doing the heavy lifting. That keeps Private AI genuinely private, keeps latency under control, and takes per-token API pricing out of the equation entirely.
Frontier models remain an option for customers who want them. It’s a setting. No IntelliThreat customer has turned it on, because the results don’t require it.
The Question to Ask Any AI Security Vendor
Not “which model do you use.” Ask what the model is allowed to do, what context it’s given before it decides, what happens to its output before a human sees it, and what it is programmatically prevented from doing. Vendors selling a model will struggle with those four questions. Vendors who built a harness will answer them in detail.
See the scaffold in your own environment — request a demo or start a 30-day free trial.