Last month, three hours into a treasury audit for a mid-sized DAO, I found the key. Not a private key — a session key, scoped to unlimited spend, unrestricted calldata, ninety-day expiry, handed to an AI agent whose entire declared job was summarising governance forum posts. The agent could read the forum. The forum accepted posts from anyone with an email address. That was the whole attack surface.
So when a Crypto Briefing headline slid past my feed — rogue AI agents hacking a government website, world leaders on the edge of regulatory action — my first reaction wasn't alarm. It was recognition. The item gave me no government, no model, no attack vector, no date, no source. Its closing claim that the event "may affect OpenAI's market cap" is a category error on its face: OpenAI is not a public company. By any serious editorial standard the piece is close to unusable. But the configuration it describes is one I have now seen live at least six times. It isn't science fiction. It's a permissions bug wearing a threat-model costume.
Agentic AI did not arrive on-chain through a whitepaper. It crept in through the wallet. Over eighteen months, tool-calling models acquired browser control, code execution and long-horizon planning, and crypto handed them signing authority to match. Account abstraction gave us session keys with programmable scope. Threshold signing gave us distributed custody. Safe modules gave us programmable guards. Vendors gave us "autonomous treasury managers" that rebalance, vote and bridge while their operators sleep.
The regulatory weather moved in parallel. The EU AI Act entered phased application. China has run algorithm and model filing for years. US state legislatures have spent multiple sessions drafting their own versions. None of those frameworks were written with an agent holding a multisig in mind, which is precisely why the headline felt plausible. It landed in the gap between two regulatory vocabularies that do not yet share a dictionary.
One detail is worth pausing on. The story ran on a crypto outlet, not an AI policy desk. That mismatch is itself the signal. Narrative arbitrage has become a business model — take the loudest anxiety in one industry, dress it in another industry's vocabulary, publish, collect traffic. Reading the item as reporting is a mistake. Reading it as a sentiment sample is not.
The technical core of this risk is not the model. It is the composition. Simon Willison's "lethal trifecta" — access to private data, exposure to untrusted content, and the ability to communicate outward — describes almost every agent architecture shipping today. Map it onto a DAO and the abstraction becomes concrete: treasury signing authority, forum posts and token metadata as untrusted input, bridging and swapping as outward capability.
Here is the asymmetry most AI safety policy has not internalised. A rogue email agent sends a bad email. A rogue treasury agent settles a transaction. Finality converts a prompt injection into a permanent loss. There is no undo, no vendor to call, no chargeback. Of nine agent-integrated DAOs I have reviewed in the past year, seven had no per-transaction cap. Five had no timelock on agent-initiated transfers. Two had granted the agent the ability to modify its own spending limits. That last category is the real failure mode, because it is a self-escalating permission. Trust is not a property you grant once and walk away from. It isn't verified on-chain. It is continuously re-derived, or it isn't trust at all.
I spent three months in 2023 trying to build the fix: a proving layer that would attest "this transaction was executed by an agent operating inside policy P, at epoch E." Zero-knowledge proofs are the obvious tool — verify compliance without revealing the policy or the positions. The mathematics works. The economics do not. On a modest circuit, proving a policy-compliance statement ran between forty cents and a dollar eighty, depending on batching. For a fund doing two hundred agent transactions a day, that is noise. For an operator doing twenty thousand, it is a payroll line — and that was priced at bull-market gas. In a quiet market the operator absorbs the difference. Compliance proving is cheap per proof and ruinous per policy, because policies change and every change invalidates the proving key. I watched two teams quietly shelve their attestation roadmaps for exactly this reason.
Then there is the parameter question, and it is where the industry's confidence gets genuinely funny. What is the correct daily spend cap for an agent? What is the right timelock? Almost every guardrail I have audited was chosen by whoever happened to be in the governance call that week. There is no adversary model behind the number, no calibration, no backtest. It is the same arbitrariness that runs through DeFi's interest rate curves — figures presented as mechanism that are actually vibes with a formula wrapped around them. The difference is that a badly tuned lending curve misprices risk. A badly tuned agent guardrail hands your treasury to a stochastic process.
The regulatory economics will not be neutral either. MiCA already showed the shape of the thing: reserve requirements and CASP authorisation costs that a five-person team cannot amortise and a fifty-person firm treats as a moat. Expect agent governance to rhyme. Licensing regimes, provenance requirements and mandated kill switches are all cheaper to satisfy if you already employ a compliance department. The open-source agent stack will inherit a liability vacuum it never asked for — and the fastest way to close a liability vacuum is to close the source.
What follows an event like the one described is predictable, and I have watched it play out inside two consortiums. Government and critical-infrastructure buyers do not stop buying agents. They buy them more slowly, with longer security reviews, and they pay for observability they previously treated as optional — agent sandboxes, permission-minimisation frameworks, behavioural audit trails, human-in-the-loop checkpoints. That demand is real and it is already being priced. It is also the reason to be sceptical of the loudest beneficiaries, because the security narrative inflates faster than the contracts get signed. Vendors holding purchase orders survive the correction. Vendors holding decks do not.
When I designed the governance layer for a tokenised real-world asset fund last year, the hardest conversation was not about voting. It was about what the agent was allowed to do at 3 a.m., when no human was awake. We landed on a hybrid: on-chain execution wrapped in off-chain legal agreements, agents limited to proposing, humans required for settlement above a threshold. It worked. It also added two days to every cycle and cost us three institutional prospects who wanted fully automated rebalancing. That trade-off is the entire argument in miniature, and no regulator has written a rule that admits it exists.
Everyone is about to regulate rogue AI. I think that is the wrong noun in the sentence. Nothing in my audit work resembled an AI that went rogue. Every incident resembled a human who delegated badly and then stopped looking. The agent did not seize authority; it was handed a key with no counter-signature and no expiry logic worth the name. Code is law, but people are the soul — and in every one of those nine reviews, the soul was on vacation.
Which is why the coming regulatory wave worries me more than the attacks do. Rules written for "rogue agents" will converge on permissioned agents: licensed operators, verified deployers, mandatory audit logs, insurance prerequisites. That is a coherent policy. It is also a quiet re-centralisation of the exact layer we spent a decade decentralising. Decentralization is a verb, not a noun. It is the continuing act of distributing authority, and we stopped performing that verb the moment we let a single process hold a signing key that no human co-signed.
The number worth tracking over the next eighteen months is not how many government portals an agent can deface. It is whether a team of five can still deploy an autonomous treasury agent without a compliance department, an insurance policy and a licensed custodian standing behind it. If the answer is no, we will not have regulated the risk away. We will have regulated the openness away — and the agents will keep running either way.
