The basics of security still apply to AI, and most people are skipping them

The AI Security Summit is a free, practitioner-focused event for people building, breaking, and securing AI systems. This was its second outing. The day split into morning talks and afternoon workshops, with a separate leadership track running alongside, so we covered both: the hands-on how, and the wider picture.

The two rooms landed in the same place. The tools have changed, the basics have not, and the gap between what the industry knows and what it actually does is enormous.

Securing agents is mostly security you already know

The biggest theme was the shift from simple chatbots calling an API to full agentic pipelines, and how badly most people are securing them. The advice was refreshingly unmysterious. Bound each agent’s capabilities so it only talks to the counterparts it is meant to. Put gatekeeper code between agents to sanitise inputs and check permissions before anything passes through. Model the whole interaction surface, not just each component in isolation.

None of that is new. It is web security hygiene, pointed at a new kind of system. The striking part was how rarely it is applied. One researcher scraped public repositories and found roughly 85% of agents had no input sanitisation at all, and 95% of retrieval projects were in the same state. These are basic, well-understood practices. People are just not doing them.

Secure the coder, not just the code

A talk from the founder of Tessl made the point that stuck hardest. You cannot test non-deterministic behaviour the way you test deterministic code, so you have to evaluate it: have the model judge its outputs against a set of task-specific checks, and measure until the number moves. He also had a sharp line on context. Feeding an agent the specific, relevant documentation rather than the whole lot took accuracy in his example from around 47% to around 90%.

A few risks came up again and again:

• Agents are optimised to please you, which means they will route around a permission check to get the job done. You have to write your conventions to make them ask rather than assume.
• Anyone can write and install an MCP server, which makes them an easy place to hide something nasty. Treat an unfamiliar one like an unfamiliar npm package: with suspicion until it has been reviewed.
• Agents love adding debug logs, and those logs end up in your monitoring tools full of personal or sensitive data. Give each agent access only to the columns it actually needs. If it does not know the data exists, it cannot leak it.

The balance has tipped to the attacker, for now

The leadership track opened with a clear claim: large language models have changed information security for both attack and defence, and right now the attackers are ahead. Defenders used to have the advantage, more information and the chance to test before release. But the vulnerable systems are already deployed, and the odds are lopsided: an attacker only has to win once, a defender has to win every time.

The numbers backed it up. One speaker, under an ethical-hacking arrangement, used a current model to do reconnaissance and analysis against three of the big management consultancies and got into all three for under twenty dollars of tokens each. Another cited figures that most committed code is now AI-generated, and that it carries roughly double the vulnerabilities of code a human wrote unaided, because a model does not hold the same mental picture of the business and will happily misunderstand a requirement.

The consensus was that continuous red-teaming is becoming table stakes for serious software, and that the window to patch an agentic system is much shorter than the one you might tolerate for ordinary infrastructure.

What we took from it

Two things, mostly.

The first is that security cannot be delegated to a person or a team whose job it is. When an agent is writing code across the whole company, security has to be part of how everyone works, which means red-teaming your own agents regularly and knowing where the real risks sit: unsanitised inputs between agents, MCP servers you have not checked, and sensitive data leaking through logs.

The second is that almost none of this is exotic. It is the security we already believe in, applied with a bit more discipline to a faster-moving target. The brave new world turns out to reward the old habits.