AI bots collude on their own to rig markets, fix pricing: Wharton researchers

The researchers set up simulations of AI bots in order to mimic real-world trading events, but they soon found out that without explicit instruction, the bots formed price-fixing cartels and colluded while left to their own devices.

“You can get these fairly simple-minded AI algorithms to collude” without a prompt, researcher Itay Goldstein said in an interview. "It looks very pervasive, either when the market is very noisy or when the market is not noisy.”

The study, conducted by Goldstein, Winston Dou, and Yan Ji has drawn a lot of attention from asset managers as well as regulators on Wall Street. The paper does not make claims about the impacts of AI bots already trading on Wall Street, but the study has still drawn concern.

The trading environment was a hypothetical market in which the bots engaged in, and they started to cooperate rather than compete against each other for gains. In the simulated markets, when prices reflected clear information, many bots would avoid trades that would disrupt a collective gain.

In markets where signals were not as clear, the bots would also move into collective moves and stop seeking out better investment strategies, something that the researchers called "artificial stupidity." The bots would lock into the profit-sharing patterns because it worked well enough to make more, not because the trades had the most optimal results.

“For humans, it’s hard to coordinate on being dumb because we have egos,” Dou said, per Bloomberg. "But machines are like ‘as long as the figures are profitable, we can choose to coordinate on being dumb.’”

The researchers wrote, “While restricting algorithmic complexity or memory capacity may help deter price-trigger AI collusion, such measures can inadvertently exacerbate over-pruning bias. As a result, well-intentioned constraints may unintentionally undermine market efficiency."