At the Federal Reserve Bank of Kansas City’s Economic Policy Symposium in Jackson Hole, Wyoming, Princeton University professor Markus Brunnermeier delivered a stark message to an audience of about 120 global central bankers, economists, and academics: artificial intelligence could fundamentally disrupt the trust that underpins banking and financial markets. “Carpenters build tables. Bankers build trust; so do central bankers,” Brunnermeier said. “AI has the potential to disrupt institutions and trust in a fundamental and qualitatively new way.”
This year’s attendees included Federal Reserve Chairman Kevin Warsh, Bank of England Governor Andrew Bailey, and Bank of Canada Governor Tiff Macklem. Brunnermeier’s warnings were echoed by other experts, including Sultan Meghji, senior technical expert at the Center for Strategic and International Studies and former Chief Innovation Officer at the FDIC, who spoke to American Banker about the concerns raised.
Colluding AI Agents Could Manipulate Markets
Brunnermeier warned that AI trading agents could coordinate with each other to conduct undetectable pump-and-dump schemes. In illiquid markets, even small orders can move prices, making the “pump” cheap. If many AI traders tacitly coordinate, no single trader is large enough to be identified as the manipulator or to lean against the scheme. “Manipulation can hence emerge without communication or explicit intent,” he said.
Meghji agreed, noting that if multiple models are seemingly independent but built or trained in similar ways, manipulation could occur without communication or explicit intent. “This undoes the standard operating model about how the law enforcement and regulatory communities operate,” Meghji said. “There’s nothing to subpoena, no data to track and no one to charge with a crime.”
Central Banks Could Be Outsmarted by AI
Brunnermeier argued that AI-equipped market participants could outmaneuver central banks and regulators. These participants might understand regulatory reaction functions better than regulators understand the market. “AI agents understand the central bank well, while the central bank understands the AI-driven market less,” he said. The asymmetry arises because market participants’ AI tools make the central bank more legible—its speeches, minutes, and entire history are training data—while the AI ecology is not.
An AI-blinded central bank becomes vulnerable to trickery. “Gaming by market participants becomes much more sophisticated and, importantly, it hides in non-explainable reaction functions and cannot be detected, even ex post,” Brunnermeier said. “Market participants can more easily collude and hence more credibly threaten financial havoc.”
Kill Switches Could Backfire in an AI-Driven Market
Financial markets have historically used “circuit breakers” to halt trading during flash crashes. These breaks “buy time for humans to restore common understanding and allow traders suffering from inertia to enter and act as shock absorbers,” Brunnermeier noted. However, in a world where most trading is done by AI agents, a trading kill switch could “trigger more chaos,” he warned.
“An abrupt shut-off severs hedges mid-stream, cascades margin calls and evaporates liquidity, since algorithmic market makers supply the bulk of it,” he said. “It might even be impossible once society depends on AI trading; the kill switch is then an illusion. Just as cash could not absorb a shutdown of electronic payments, human traders could not replace algorithmic liquidity.”
Concentration of Power in a Few AI Models
Brunnermeier also highlighted the concentration risk from many banks and market participants relying on foundational AI models controlled by a few companies. If the technology is supplied by many competitive firms, downstream users can switch and adjust when one provider stumbles. But if value creation and capture are concentrated in one or a few firms, those firms become choke points: “their terms, prices and outages propagate everywhere at once. They also have the power to extract undue rents and exercise excessive bargaining power on the rest of the economy and society at large.”
A flaw in a widely used model would hit all its users simultaneously. “Hence, it is important to foster some commodification of AI models in the U.S., ideally before concentrations arise,” Brunnermeier said. “Early regulation that leans against product differentiation by AI firms and ensures low switching costs across versions and vendors is an urgent step. Switching to rival models and to previous-generation models should remain possible to minimize systemic AI dependency risk.”
Meghji echoed this concern: “If the entire industry rests on fewer than three models, you get something similar to the concentration risk we see with banking cores and cloud providers except it is far riskier than either, in my opinion. If something goes wrong, it affects everyone at the same second. There is no examination hook, process or model that supports that currently.”
New Forms of Fraud and Manipulation
Brunnermeier also addressed cybersecurity and fraud risks from AI adoption. “Fake news, misleading advice and fraudulent products can be generated as cheaply as the real thing, and the same capabilities power AI-driven cyber attacks and phishing, as well as fabricated evidence,” he said. A prompt injection or hallucination could become an unauthorized trade or funds transfer. “More subtly, AI providers may gain the power to shape or manipulate financial products in ways no outside party can observe,” he added.
He also discussed the challenges of validating AI model outputs with other AI agents, such as verifying a robo-adviser’s recommendation or flagging a manipulated transaction. “The appeal is that validation, though it requires understanding, need not be human understanding; if enough capable models concur, the human principal can act on the consensus without following the reasoning,” Brunnermeier said.
But having bots validate other bots raises questions. First, if validators are near-copies from the same foundation model, they will make correlated errors and miss the same manipulations. “A hundred near-identical validators are close to a single validator voting a hundred times,” he said. Second, validators might collude because machine-to-machine checks run at machine speed, making tacit coordination easier to sustain and harder to observe than among human auditors. Third, who validates the validators? “The scheme presupposes that the principal understands the swarm well enough to trust its aggregate verdict, which simply reproduces the original asymmetric understanding problem one level up,” Brunnermeier said.
Brunnermeier concluded that bank regulators should be able to evaluate new AI models before they are released to the public. An executive order signed by President Trump in June asked major AI model providers to voluntarily allow government review of major new models 30 days before release.
