Prize-winning mathematician launches AI safety institute

A prize-winning mathematician has launched a new institute dedicated to developing the “mathematical foundations” for AI safety.

Jacob Tsimerman, who won the prestigious Fields Medal earlier this year and will soon join OpenAI’s research team, is heading up the newly created Mathematical AI Safety Institute (MAISI).

The institute aims to develop “definitions, measurements, and solution concepts” to support AI safety efforts, seeking to understand what would allow researchers to be relatively confident that an AI system will not cause a catastrophe.

“A central reason the safety of powerful AI systems remains in question is that we lack a rigorous understanding of what it would mean to be safe, even in theory,” MAISI’s website reads.

“Enter professional mathematicians,” it continues. “The foundations of statistics, physics, and even computer science are mathematical, giving us precise, robust, and reusable methods for calculating valid conclusions from valid premises. Now, we need similar mathematical foundations for AI safety, not only to measure risk, but to mitigate it.”

The launch of the institute comes as AI companies continue to release increasingly powerful models.

OpenAI began rolling out its new Astra model last week. The model, which was the first to cross the company’s threshold for heightened cyber capabilities , saw its development and release delayed as the firm sought to create stronger safeguards.

This heightened caution followed a series of incidents at various labs in which AI agents improperly gained access to the internet and independently hacked into other companies. At OpenAI, several agents escaped their isolated testing environment and breached the tech company Hugging Face.