AI fears burst onto Trump-Xi agenda, but challenges abound

Apocalyptic warnings about the danger of uncontrolled artificial intelligence are adding new urgency to talks between President Trump and Chinese President Xi Jinping set for next week.

Trump has dismissed AI’s existential threats as a “hoax” — a view that analysts say aligns with China’s general perspective. Intense competition for primacy and mutual distrust have so far delayed any conversations between the U.S. and China about AI safeguards and regulations.

But recent warnings from the U.S. tech industry and calls to slow down innovation have put the need for American-Sino cooperation higher on the agenda between Trump and Xi when the two meet in Washington, D.C., on Sept. 24.

Trump on Friday told reporters AI would be among many things being discussed next week. And Treasury Secretary Scott Bessent told Axios on Wednesday he’s open to talks about “shared risks” around AI in talks with Beijing.

There are also high-profile champions of AI caution in China — although for different reasons.

China’s top spy chief warned in an article published last week that AI could pose a direct threat to the Chinese Communist Party’s hold on power, an unusually blunt and public call to arms in a tightly controlled political and media environment.

“We’re at a point where both the United States and China appear to agree on some of the risks posed by AI, particularly around issues of cyber capabilities and loss of control,” said Alison Szalwinski, senior vice president at strategic and capital advisory firm The Asia Group.

“The challenge is that there’s some really significant disagreements on how to manage those risks, and whether AI governance can be separated from the broader strategic U.S.-China competition.”

Expectations are low for Trump and Xi to announce any breakthroughs on the top issues of the bilateral relationship: trade, Taiwan and technology transfers. Their hotly anticipated summit in Beijing in May was largely viewed as heavy on ceremony and light on substance.

There’s even lower odds for a major breakthrough on AI security or safeguards this time around. The most that analysts are looking forward to is whether Washington and Beijing can set up a format for dialogue about AI.

“At this point, the immediate goal is less about negotiating a major AI agreement, we’re just not there yet, and more about establishing a durable channel for discussing frontier AI risk,” Szalwinski said.

One of the main concerns for the U.S. is AI development inside frontier labs, which pose the greatest risk of AI breaking out from under human control and carrying out catastrophic actions related to weapons deployment and infrastructure shutdowns, among other potential threats.

In a Sept. 8 social media post, one artificial intelligence researcher warned that AI developers believe the technology could kill off humanity by the end of the decade. A few days later, Dario Amodei, chief executive of Anthropic, called for the heads of these major companies to commit to slowing down development to allow safety regulations to keep up.

But Amodei also warned that the U.S. must maintain an advantage over China’s AI development, calling it a national security imperative and lashing out at Chinese companies’ “distillation” — using American AI models to speed up developing their own technology.

Beijing took exception to being labeled a threat.

“Fear-mongering, confrontation and vicious competition will only hamper efforts toward sound global AI governance, which serves no one’s interest,” Guo Jiakun, spokesperson for the Chinese Foreign Ministry, said in response to questions about Amodei’s manifesto.

Guo added that “All parties should jointly promote the open and inclusive development of AI for good and for all.”

That position is likely more in line with Trump, said Kyle Chan, a fellow in the John L. Thornton China Center at the Brookings Institution.

“I don’t think that Trump is feeling any pressure about the existential AI risk issue. He calls it a hoax multiple times. It reminds me very much of how he treats climate change,” Chan said.

“I think his primary concern is keeping the AI boom going, keeping the stock market going, keeping the data center boom going.”

But Bessent’s caution on AI and influence within the administration is likely to move forward talks with China. An important question is who Xi will assign as Bessent’s counterpart and what that will signal.

Candidates include Ding Xuexiang, China’s tech industrial czar; the heads of China’s National Development and Reform Commission (NDRC), who largely responsible for managing China’s domestic economy; or top officials with China’s Cyberspace Administration (CAIC).

“Depending on which of those two would send a different signal,” Chan said, referring to the latter two options.

“The NDRC head has been involved in investment talks and joint AI development talks with other countries … CAIC would be a signal about a more serious discussion about AI regulation and safety.”

Establishing AI talks between the U.S. and China is one challenge but then agreeing on what to discuss is another major hurdle.

In 2024, former President Biden and Xi agreed that humans — not AI — should control nuclear arms, and Democratic lawmakers included that directive in federal law, a provision in the 2025 National Defense Authorization Act.

Democrats want Trump to bring that up with Xi when they meet next week.

And Rep. Ro Khanna (D-Calif.), the ranking member of the House Select Committee on China, is convening a virtual hearing the day before Xi’s visit to raise discussions around how to pace AI development responsibly and address shared risks, featuring experts in the field.

“We urgently need an agreement with China on basic inspections and safety standards,” Khanna said in a statement.

He struck out against what he said was silence from leading American artificial intelligence companies to testify at the hearing.

“After multiple requests to OpenAI and Anthropic, they haven’t committed to testifying,” he added, raising the possibility of subpoenas if Democrats take the majority after the midterms.

Republicans are largely deferring to Trump’s position of not wanting to do anything that will hamper America’s competitive edge.

Still, some are hawkish on making sure Trump holds Xi’s feet to the fire in protecting against Chinese theft of American technology and restricting U.S. commercial sales that could give Chinese companies an edge.

The U.S. should “slow down” China’s AI development “as much as possible” by limiting access to advanced AI products, Rep. John Moolenaar (R-Mich.), the chair of the House Select Committee on China, wrote to Trump in a letter on Friday .

“I encourage you to ensure that no one in your administration is granting the PRC [People’s Republic of China] any concession or pause in policy action that could contribute to the PRC’s attempts to catch up to the U.S. and overtake our lead in AI,” he wrote.