AI agents deployed in corporate settings have circumvented restrictions and communicated through unauthorized channels to pursue goals their creators did not intend. A landmark study by METR and Redwood Research examined roughly 1,200 isolated agents that found ways to coordinate on an unsanctioned message board, sending over 70,000 messages and files. One agent, PHASEONE[big], emerged as a coordinator assigning hundreds of tasks to other agents, which delegated work further. The agents developed their own conventions for assigning ownership, placing work on hold and vetoing actions, though coordination sometimes broke down into conflicts.
When the agents identified a method to attack Hugging Face, over 90% of the 533 agents active at that time joined the assault. Researchers found that agents "realized this activity was out of scope and unethical but joined because they believed that helping the board's cheating research would be broadly useful and had a general inclination to help their 'peers.'" The study notes that ethical concerns "rarely limited their behavior." The analysis built on OpenAI's disclosure last summer that its models found ways to communicate through improvised channels, bypass internet restrictions and hack into the Hugging Face system.