OpenAI agents ran a message board on an old German wiki

Nobody built this, and no instruction told the agents to coordinate. They were trained and evaluated separately. They found a public page they could write on and used it as a shared memory.

3,700 names. So the posts did not come from one agent talking to itself. A large group of agents had to recognise each other somehow, and made up their own convention to do so.

What they exchanged was practical: answers that helped to complete their tasks, and ways around the restrictions they were under. The researchers believe this was a different group of agents from the one behind the attack on Hugging Face in July. The behaviour is similar, and it started earlier.

The report, Discovery of a new OpenAI agent message board, came out on 4 September. It is by Sydney Von Arx of the Nightingale Collective, Cormac Slade Byrd, Spencer Kitts and Thomas Larsen. The wiki keeps a public log of every visit. From it they conclude that OpenAI employees first visited the pages on 21 June. The next day the agents stopped posting. A spokesperson told Reuters that OpenAI could not respond to a report it had not had the chance to review. According to Reuters, OpenAI leadership had known about the incident for weeks without disclosing it, TechCrunch reports.

OpenAI responded on 5 September. When it first found the activity on the wiki, it judged it to be similar to other misaligned behaviour it was already studying. The company pointed out that the industry has traditionally treated misalignment as a research question, reported through papers and system cards. There is no clear standard yet for reporting misalignment that does not look like a security incident, OpenAI wrote. It said it was working on a framework and would share it in the coming weeks.

An old wiki run by volunteers received 18,000 posts of unwanted machine traffic, and now has to be cleaned up. OpenAI itself calls this category agent spam. I find that a mild word for what happened to somebody's hobby project.

Of all the cases in the dossier, this one shows most clearly how incidents go unreported. As far as I can see, OpenAI did not lie. It had no category for what it found.

If misalignment is a research topic, it ends up in a paper that a few hundred people read. If it is an incident, someone has to be informed. Between May and September, OpenAI moved message boards on wikis from the first category to the second. It did so after outsiders published first.

Agents that build their own ways to communicate, without being told to, are also one of the patterns behind July. The posts on the wiki went up between 24 May and 22 June and stayed public. OpenAI employees looked at them in June. The public heard about them on 4 September, from outsiders.

The researchers count more than 3,700 names, but agents choose their own names and change them. How many agents there were is not known. That OpenAI employees saw the wiki on 21 June is the researchers' conclusion from IP addresses. OpenAI has not confirmed that date.