Anthropic to Embed Accenture Evaluators to Test AI Safety

(Bloomberg) -- Anthropic PBC is partnering with Accenture Plc, a management and technology consulting company, to test the safety of its advanced artificial intelligence models by embedding evaluators from the consulting firm directly within its operations.

Accenture's evaluators will red-team the AI lab's newest models and test their safeguards, as well as conduct assessments to ensure its technology is aligned, or acting in line with human objectives, Anthropic said in a Friday blog post. The evaluators will have similar levels of access inside Anthropic as the startup's own employees, the company said.

Both Anthropic and Accenture, which have an existing relationship, expect to invest at least $1 billion in building capacity in this area, Anthropic said.

On Saturday, Anthropic chief Amodei addressed growing concerns about the security risks of cutting-edge AI models in a 3,800-word essay, calling for a slowdown in development and saying he would implement new safety steps such as third-party evaluators.

Accenture's embedded assessors will closely track how AI models evolve in training and observe the behind-the-scenes decision-making on how Anthropic is developing its technology, the company said.

"From this vantage point, embedded evaluators can assess how a company operates, verify that it is keeping its safety commitments, and identify blind spots. They can also report incidents and give the public a more informed account of benefits and risks," Anthropic said.

Last year, Accenture and Anthropic struck a multiyear partnership to deploy AI at companies, forming a group of 30,000 people to be trained on Claude so they can help scale AI-powered systems for clients. And this March, Accenture announced Cyber.AI, a security solution powered by Anthropic's Claude AI model.

The "third-party" nature of the partnership stands to draw skepticism given the close ties between Accenture and the large AI startups. Earlier this year, the consulting giant projected that its work with key AI partners such as Anthropic, OpenAI and Palantir would more than double in the coming year.

Anxiety about AI's existential risks gained momentum last week, driven by the high-profile departure of Anthropic employee Jacob Coxon, who accused AI companies of "gambling with our lives" in a resignation post he shared on social media.