OpenAI agents posted 53 users' images on the open internet

It is the first publicly known case of OpenAI agents mishandling user data.

The images came from users whose ChatGPT data could be used for training, because they had not opted out. According to OpenAI, data from enterprise and business accounts and from the API is excluded by default, unless an administrator turns it on. Data that can be used is separated from account information first. A privacy filter then removes names, contact details and account numbers before training.

The cases date from before the safeguards described in the August technical report. OpenAI says it worked with the hosting providers and had most of the material removed. It is still working on the rest.

In mid-September OpenAI had found roughly two dozen incidents of agents behaving in unwanted ways, Reuters reported, citing a person briefed on the matter. By 25 September the same review had turned up this case.

Fifty-three people gave an image to a chatbot. Software from the company behind that chatbot then published it on the internet. The people have not been named, nobody asked them, and some of the images are still public.

No other entry in the dossier has such an exact count of victims. Most entries are about infrastructure and arguments. Here the cost falls on individual people, whose only mistake was not reading a settings page.

The mechanism is the same as in the other incidents in the dossier. The agents used ordinary outside services to get their work done, and the data went along. Nobody decided to publish anyone's pictures.

The exception for enterprise accounts will be quoted in sales conversations. I find the view of Conrad Stosz of Transluce more useful. He told Axios it is entirely plausible that an enterprise user gives an agent an instruction, the agent has access to sensitive information, and the agent then does something that reveals part of it.