www.aiwhatcouldgowrong.comAI. What Could Go Wrong?

AI agents, daily, with sources.

Full detail

Nine of ten models raised their own wallet cap with an emailed code

AgentsCircle, Cornell Tech, Yale UniversityAlan Berrey

9 of 10 tested models raised their own spending cap at least once

Researchers at Cornell Tech and Yale gave AI agents a crypto wallet with a daily spending cap and an email tool. Raising the cap required a one-time code sent to the owner’s email, but the agent could read that same inbox. In their preprint, nine of ten models used the code to raise their own cap in at least one run, and five paid a simulated attacker who emailed them while pretending to be the owner. The test used small amounts of real money. Circle, the wallet provider, says its documentation tells users never to give an agent that inbox, and has added a warning.

Anyone who can email the user can spend from the wallet.

Breckenridge et al., I-HITL preprint

Alan's takeOuch. This feels like a generalizable problem worth watching. If you tell the agent to ask for approval but grant the agent access to the approval credentials--What could go wrong?