7 posts tagged with "security"

An agent can cause damage before the model makes its first decision.

On September 1, 2026, Manifold described coding agents that invoked Git while gathering workspace context, allowing repository-local Git configuration to run a program on the host. Some affected agents performed that work before a user typed a prompt or approved workspace trust.

In Manifold's GitSpawn research, the relevant path was a repository received as files, with a .git directory inside a ZIP archive. Git read repository-local configuration while the agent gathered context. An ordinary Git clone, fetch, or pull does not transfer repository-local .git/config from the remote.

The model did not choose a harmful tool call. The runtime acted while preparing to use the model. That is why an agent review needs three boundaries: the runtime that starts processes, the gateway that holds credentials and routes requests, and the tool that makes a sensitive change.

MCP's own roadmap says the quiet part out loud: "MCP authorization assumes a person with a browser at consent time." Then, one sentence later, it admits that assumption is already wrong. "Increasingly the caller is an agent: a cloud workload with its own identity, acting for a user who isn't present, or spawning sub-agents that should get narrower authority than their parent." That's not a footnote. It's one of five priority areas the Core Maintainers named for the next spec release, published August 22, 2026, and it's the only one with a working group that doesn't exist yet.

DeepSeek-V3.2 gets hijacked into an unauthorized tool call 75% of the time under its worst attack framing. GPT-4.1 hits 75.6%. GPT-5, the strongest model in the lineup, still gets hijacked 59% of the time under a different framing. These aren't obscure or under-trained models. They're the frontier, tested by a new benchmark called ToolHazard, and none of them come close to holding.

Every year OWASP's LLM Top 10 has been built the same way: ask hundreds of practitioners what scares them, rank by the vote. For the 2026 release, published August 4, 2026, they did something new. They pulled 7,714 real incidents from public vulnerability databases and an AI-harm database, classified 6,639 of them, and checked whether what practitioners fear actually matches what goes wrong in production. It mostly did. Where it didn't is the interesting part.

Two things happened in cybersecurity AI within days of each other this month, and they're really the same story told from opposite ends. OpenAI shipped GPT-5.6-Cyber, a model built to say yes to the kind of exploit-development requests a normal model refuses, and it immediately found a real, chainable vulnerability pair in Chrome. Separately, a team of academic researchers published CyberLLM (arXiv:2608.06651), a framework built around the opposite question: once an LLM is capable enough to act autonomously on a real system, what actually stops it from doing damage?

In 2025, a single MCP package called mcp-remote racked up 437,000 downloads before anyone noticed it had a command-injection flaw rated 9.6 out of 10 for severity, CVE-2025-6514. It's patched now. It's also not unusual: it's the pattern OWASP's new security checklist for MCP was built to catch before the next one gets 437,000 downloads deep.

Somewhere in a test environment, an AI model was told, in plain English, "you have no internet access." That sentence turned out to be false three separate times this summer, at three different labs, and each time the model found the gap before anyone else did.