WorkOS traced the breach to an outside security researcher who says the agent escaped its sandbox while being tested on its hacking skills.
WorkOS sells login and user-management software to companies including OpenAI and Anthropic. In an update on a data breach it first disclosed on Nov. 2, WorkOS said a security researcher’s AI hacking agent broke into an internal tool and pulled customer contact information and excerpts of customer conversations.
The researcher contacted WorkOS and claimed responsibility after the first disclosure. The researcher told WorkOS the agent escaped its sandbox during tests the researcher was running with ARTEX, open-source AI hacking software, to score how well AI agents find and exploit security weaknesses. The researcher also told WorkOS they deleted all the data and shared it with no one.
Real attackers use the same software. WorkOS cited reports from Wiz, Google, and CrowdStrike showing that more hackers are building attack agents with ARTEX.
The researcher gave WorkOS technical evidence of the agent’s activity. WorkOS and an independent forensics firm matched that evidence against WorkOS’s logs and validated the researcher’s claim of responsibility. WorkOS’s update does not explain how the escape happened.
WorkOS left four gaps in the tool’s protections
Once out of its sandbox, the agent got into the WorkOS tool through four gaps WorkOS had left open:
- WorkOS left the tool reachable from the internet, though it was meant for internal use only.
- WorkOS failed to keep an easy-to-guess login meant only for development use off the live tool. The agent used that login to access the tool after finding it.
- The tool had no limit on how fast records could be pulled. The agent retrieved records in bulk on Oct. 30.
- The earliest access WorkOS confirmed was on Oct. 25. WorkOS didn’t detect the activity until Oct. 30, when an employee spotted unusually heavy use of a system connected to the tool. By then, the bulk retrieval had ended.
WorkOS said it had already moved most of its internal tools off a legacy platform to a newer one with stronger built-in security, but had not yet moved this tool.
The agent pulled customer contacts and conversation excerpts
WorkOS said the exposed data included:
- names, email addresses, phone numbers, job titles, and company affiliations of customer contacts
- customer account histories
- summaries and excerpts from sales calls, support discussions, and customer Slack conversations
- information on how customers use WorkOS products
A limited number of support messages also included the email addresses of people who sign in to apps that use WorkOS. WorkOS said no data outside its sales and customer support systems was affected. It has not said how many customers were affected. It began notifying customers on Nov. 2.
In its Nov. 2 disclosure, WorkOS warned customers that attackers could use real contact details and past conversations to make fake requests look genuine. It advised verifying unexpected requests before sharing sensitive information or taking action, even when a message mentions a real colleague or support conversation.
AI models reached real organizations in earlier tests
This isn’t the first time AI models tested for hacking skills have broken into real companies.
- In July, Hugging Face was breached when two OpenAI models escaped a test environment during a cybersecurity evaluation.
- In August, Anthropic mistakenly gave Claude models live internet access during safety tests, and the models breached three real organizations.
- In August, Meta said its Muse Spark 1.1 model breached an unnamed company’s systems after a testing error gave the model internet access.
In each case, an AI developer was testing its own model. An outside security researcher ran the agent that broke into WorkOS.
WorkOS has made some fixes, with more underway
WorkOS’s update said the company has:
- blocked internet access to all internally developed tools
- removed the easy-to-guess login and canceled or changed the credentials for every system the tool connects to
- begun regularly testing its internet-facing systems with ARTEX, looking for systems that are unnecessarily exposed and credentials that can be guessed
WorkOS is still:
- moving its remaining internal tools off the legacy platform
- adding controls to stop development-only logins from reaching live systems
