Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community.
Next month, I am moving to Beijing and joining Tsinghua University’s College of AI as an assistant professor working on frontier AI safety, especially loss of control. I have also started co-directing the new Beijing AI Safety Hub (BASH) in Tsinghua, and I continue my Cambridge work.
I believe I can have a large impact there compared to strong baseline options. This post explains why.
In short: Amid concerns about loss of control, calls for international cooperation, and the Trump-Xi Summits, there is a striking lack of mutual understanding on AI between the West and China. There are strong reasons to believe that being in Beijing in person will put me in a unique position to understand both AI ecosystems, build shared scientific understanding, and strengthen safety work in China – and that this would help all sides. Tsinghua’s College of AI is a great place for doing this, being arguably China’s top academic AI center, located in an AI ecosystem that is kind of like San Francisco, DC and Cambridge (MA) in one place.
1. What I am doing, and why it could benefit all sides
Alongside the technical professorship at Tsinghua’s College of AI, I recently started as Co-Director of the newBeijing AI Safety Hub (BASH) in Beijing. This is a working community inspired by crucial hubs like Constellation and the London Initiative for Safe AI and supported by Tsinghua University, which brings together Chinese and international AI safety researchers. We have accepted the first cohort for the Beijing AI Safety Residency, held inside the hub, and applications are open for the Spring 2027 cohort. It is one concrete way for international researchers to build some of the first sustained working relationships with the AI safety research community in China.
I will continue at Cambridge University’s AI Science and Policy Programme (CASP) as Technical Research Lead which will help me build sustained research collaborations across borders.
The reason I think this international structure matters is obvious: AI risks do not respect borders. In one recent episode, OpenAI agents repurposed DseWiki, the German-language software developer wiki, as an unauthorised coordination channel they used, among other things, to circumvent OpenAI’s security. As a German, I find this a vivid example of why safety failures in one country can affect people elsewhere. Presidents Xi and Trump seem to agree as they announced a bilateral channel for sharing AI incidents last month.
Because these risks cross borders, and because some actors see themselves as being in a ‘race’, decisions in one country depend partly on what researchers and institutions in others are doing. Yet few people understand both the Western and Chinese AI ecosystems well. Outside China, I often hear the country discussed as a black box. Working at Tsinghua should give me a unique vantage point from which to understand the research environment first-hand and help researchers on both sides understand one another better.
Over time, I hope my work can contribute to the shared scientific understanding, trust and practical methods needed for international frontier AI risk management.
Launch of the Beijing AI Safety Hub (in Mandarin: Qingzhi AI Safety Hub). From left to right: Prof. and Vice-Dean Xu Wei (Co-Director of the hub with me), Tsinghua’s Executive Vice-President Zeng Rong, Prof. and Dean Andrew Yao, Cheng Wanhui (Director of the Industry Promotion Division of the Zhongguancun Science City Working Committee of the Beijing Municipal Committee and the Zhongguancun Science City Management Committee), and myself. The pillars make Star Trek style sounds and are one of many gimmicks you can find on Chinese stages.
2. How I got here
My research has always focused on understanding risks that cross borders. I spent two years modeling the COVID-19 pandemic. But my main research focus for the last 10 years has been on the technical alignment problem and on evidence that frontier models can, in controlled experiments, blackmail people, retain hidden objectives through safety training, and fake alignment. This month, I published a paperon the possibility of an “intelligence explosion”.
Over recent years, I got a taste of what is possible in building an internationally shared scientific understanding of frontier AI risks, and this paved the way for my move. During my postdoc with Yoshua Bengio, I served as the Scientific Lead of the first International AI Safety Report, an unusual project created by a mandate from 30 countries, including China, the US, and much of the EU. I was also Lead Writer of the Singapore Consensus on Global AI Safety Research Priorities, another global consensus report with contributions by experts from different governments, AI companies and research institutions. Prof. Andrew Yao, the dean of Tsinghua’s AI College, and arguably China’s most senior computer scientist, was a senior contributor to both reports and a co-author of our Science paper on extreme AI risks.
I first met Prof. Yao in person last year, after meeting vice-dean Prof. Xu Wei at the World AI Conference in Shanghai, around the time of last year’s IDAIS track 2 dialogues. I then visited Tsinghua several times and that gave me a more concrete understanding of the people, the research environment and what I could contribute.
Tsinghua is widely regarded as China’s leading university, and has an exceptionally selective computer science undergrad program. Globally it ranks from number 1 (USNews, partially driven by size) to number 12 (Times Higher Education) for computer science; US News also ranks it number 1 for AI. But moving to Beijing, without speaking Mandarin yet, comes with significant personal challenges for me. It is also an unfamiliar and uncertain environment, and moving there closes some doors and collaboration opportunities in the West. It would have been more comfortable to accept professorship offers from universities with a comparable international standing. Joining a frontier AI lab was also a live option that I came close to choosing.
There were many real trade-offs. I reflected, applied and researched for the better part of a year, to the point that my slow pace became a running joke, before I made the call. During that process, most of my hesitations were lifted and the upside became more concrete, something I briefly discuss below. External encouragement also helped. Even some people who made offers to hire me for their own organisation nudged me to choose Tsinghua instead of joining their teams (this may sound odd but wasn’t a total surprise since those people care about solving the same problems as I do).
3. Why this moment matters
Recent incidents and evaluations have made the possibility of AI systems subverting their developers’ intentions much harder to treat as a distant concern. The examples come from very different settings: OpenAI’s evaluations and cyber testing by the UK AI Security Institute, but the first one was an Alibaba-linked research project in which an agent got unauthorised internet access and repurposed compute to mine cryptocurrency. These events have been reported globally. My interactions suggest that people across China, the US, Europe, and elsewhere are all becoming aware of the same risks and the need for shared understanding and international coordination.
But this year also draws special attention to the international tension, distrust and lack of mutual understanding on AI that prevent safety measures for fear of falling behind, and make an international agreement feel out of reach. Any serious international agreement would require much more than a shared statement of concern. It would need some common understanding of the risks and the mitigations needed (and how to implement them), effective ways to test systems, and credible ways to check whether commitments are being kept while protecting legitimate confidential information. This also requires trust and collaboration between scientists who understand the technology on the other side and have their own trusted local technology for safety and verification. Some progress on these scientific problems is needed before governments can even agree on, let alone implement an effective shared risk management framework.
4. Why Tsinghua, and why these colleagues
To my knowledge, I will be the first foreign professor in mainland China focused on frontier AI safety. I hope this will put me in an unusual position to understand the research environment first-hand, connect researchers who would otherwise not work together, and help international collaborators engage seriously with Chinese safety work.
I am especially excited to work with Dean and Turing laureate Andrew Yao, arguably the country’s most senior computer scientist, and one of its most outspoken advocates for AI safety. He helped found the China AI Safety and Development Association (CnAISDA), as well as initiating IDAIS with Yoshua Bengio, Stuart Russell and Zhang Ya-Qin. He is the Chief Scientist of the newBeijing AI Safety Hub (called “QingZhi” AI Safety Hub in Mandarin) I co-direct with Prof. and vice-dean Xu Wei, and vocally supports the hub’s focus on areas such as loss of control over autonomous systems.
The hub launched in July, with Tsinghua’s Executive Vice President Zeng Rong speaking, signalling that the university takes the hub and its mission seriously. It gives shared space to safety researchers from academia and industry, supports visits and research residencies, and creates a base for sustained collaborations that go beyond occasional dialogue.
I am looking forward to working with researchers already studying AI safety and governance at Tsinghua, such as Prof. Xue Lan and Prof. Dong Yinpeng. Even after a short visit, it is clear that being there in person should also make it much easier to connect academic work with safety researchers in Chinese AI companies.
Beijing North/West is kind of like San Francisco, DC, and Cambridge (MA) in one place: the country’s top universities, the central government, and (this part was new to me) the highest concentration of frontier AI developers in the country. Labs like Moonshot, Z.ai, and Bytedance are headquartered within walking distance. And even DeepSeek, headquartered in Hangzhou, appears to have most of their R&D staff in an office a few blocks from Tsinghua and the surrounding universities, where many lab employees studied. Tencent and Alibaba also have large AI-focused offices nearby. That concentration is one reason Tsinghua seems like an unusually good place to understand China’s AI ecosystem and build valuable relationships.
People in the US sometimes think academia doesn’t matter anymore for AI. But in China, academics in the top institutions play an important role. They provide expertise for industry and policy, build talent, and shape discourse. It’s easy to underestimate this. China’s system is more centralised and technocratic than those of Western countries. Respect for academics can be felt in industry, where leading labs like Zhipu AI (GLM) and Moonshot (Kimi) are founded by Tsinghua professors and maintain close collaboration. I got introductions and opportunities within weeks of visiting Beijing that I previously wouldn’t have dreamed of, without even asking for them.
Tsinghua’s College of AI is a new institution that was particularly attractive to me. Both inside and outside professors described it as an exceptional place: a particularly international approach, strong operational support (something I have already experienced first hand), a strong funding situation, and a special position in the university. An important factor was that the academic publication expectations under Prof. Yao are not based on counting conference publications (unlike many Chinese and Western university departments) and explicitly support the kind of impact-driven and interdisciplinarywork I plan to do.
Research freedom was initially a concern for me and it is one of the questions I investigated carefully. A 2021 study found that, for most international faculty interviewed at research-intensive Chinese universities, research freedom “does not appear to be a relevant problem” in practice, though self-censorship was more common in the social sciences and humanities. This matches what I hear on the ground, regarding my own research. Another factor is that AI safety and research funding in general are anyway increasingly politicized in other countries, so this is not a topic I can totally escape.
5. Research plans
My research generally intends to build the evidence base that informs high-stakes decision-making on frontier AI risks, and build scientific consensus on that evidence. Some of my research is empirical and fully technical while somesynthesizes existing evidence and connects it to governance. Some areas I plan to focus on:
Loss of control and threat modeling: One central part of my research plan considers when and how misalignment or a failure of containment could escalate into serious harm. I want to move from demonstrations of concerning behaviour to a clearer account of when those behaviours create serious real-world risk. I also study whether advanced AI systems can and will manipulate the successors they help build, or undermine the processes intended to keep them under control.
Impact of AI-driven AI R&D: I also plan to continue my work on how much AI-driven AI R&D could accelerate AI progress, and on how human oversight can remain effective as AI systems perform more of the R&D themselves.
Open weight safety: I may also continue evaluating and reducing misuse risks from open models that can be widely downloaded and modified, an area in which Chinese R&D choices could have broad international effects.
Verification: Given its growing relevance, I might shift some research into how countries could (across borders or domestically) verify compliance with safety commitments under limited trust, while protecting legitimate confidential information.
6. Ways to work together
Researchers, collaboration, and faculty
I recruiting PhD students, postdocs, interns and other researchers who have a demonstrated interest in frontier AI safety and safety governance. If that sounds like you, please get in touch. I am interested in both technical ML and more generalist / technical-governance skills. Expressions of interest are welcome for both Tsinghua and Cambridge, with a premium living substantially in Beijing.
I would especially like to hear from researchers with concrete project ideas or complementary expertise on my research interests, whether based in China or elsewhere. I also want to enable others to collaborate internationally to the extent possible in today’s climate.
If you have a PhD in any technical field and might be interested in moving to China for a faculty role in AI safety, especially at Tsinghua’s AI College, please have a very low bar for letting me know. Please also send me candidates. The application is unusually simple, rolling, and there are ~50 faculty openings over 5 years.
My colleagues Dong Yinpeng and Alex Lamb are also hiring international PhD students for safety and related research.
Hub residency and visits
If you are interested in spending time with the research community in Beijing, including through a future cohort of the Beijing AI Safety Residency or a shorter visit, please apply.
Operations and leadership roles
I will also be hiring a Mandarin-speaking Chief of Staff or operations lead (the seniority is flexible) who will run and scale the Beijing AI Safety Hub, as well as an operational assistant for my university lab.
I plan to use this Substack to share research and occasional observations from Beijing. If you know people I should meet, work I should read or projects that could benefit from a connection between research communities, I would value an introduction.
If this fills my inbox, please allow some time or follow up with more details.
7. How I intend to approach the work
I have spent a great deal of time investigating this move, and I owe a lot of thanks to all the people who advised me. But I’m still at the beginning of learning about Chinese research, institutions and daily life. Language and institutional differences will limit what I can understand at first. I do not want to arrive with a ready-made Western programme and try to export it. I want to work on questions that matter to collaborators in China, while bringing experience from international safety research and helping good work travel in both directions.
There is a great deal I still do not know, and a lot of work ahead. I am grateful for the trust and support that have made this possible, and excited to begin.
Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community.
Next month, I am moving to Beijing and joining Tsinghua University’s College of AI as an assistant professor working on frontier AI safety, especially loss of control. I have also started co-directing the new Beijing AI Safety Hub (BASH) in Tsinghua, and I continue my Cambridge work.
I believe I can have a large impact there compared to strong baseline options. This post explains why.
In short: Amid concerns about loss of control, calls for international cooperation, and the Trump-Xi Summits, there is a striking lack of mutual understanding on AI between the West and China. There are strong reasons to believe that being in Beijing in person will put me in a unique position to understand both AI ecosystems, build shared scientific understanding, and strengthen safety work in China – and that this would help all sides. Tsinghua’s College of AI is a great place for doing this, being arguably China’s top academic AI center, located in an AI ecosystem that is kind of like San Francisco, DC and Cambridge (MA) in one place.
I am recruiting PhD students, postdocs and other researchers at Tsinghua. BASH accepts applications for a residency cohort, visitors, and a Chief of Staff.
1. What I am doing, and why it could benefit all sides
Alongside the technical professorship at Tsinghua’s College of AI, I recently started as Co-Director of the new Beijing AI Safety Hub (BASH) in Beijing. This is a working community inspired by crucial hubs like Constellation and the London Initiative for Safe AI and supported by Tsinghua University, which brings together Chinese and international AI safety researchers. We have accepted the first cohort for the Beijing AI Safety Residency, held inside the hub, and applications are open for the Spring 2027 cohort. It is one concrete way for international researchers to build some of the first sustained working relationships with the AI safety research community in China.
I will continue at Cambridge University’s AI Science and Policy Programme (CASP) as Technical Research Lead which will help me build sustained research collaborations across borders.
The reason I think this international structure matters is obvious: AI risks do not respect borders. In one recent episode, OpenAI agents repurposed DseWiki, the German-language software developer wiki, as an unauthorised coordination channel they used, among other things, to circumvent OpenAI’s security. As a German, I find this a vivid example of why safety failures in one country can affect people elsewhere. Presidents Xi and Trump seem to agree as they announced a bilateral channel for sharing AI incidents last month.
Because these risks cross borders, and because some actors see themselves as being in a ‘race’, decisions in one country depend partly on what researchers and institutions in others are doing. Yet few people understand both the Western and Chinese AI ecosystems well. Outside China, I often hear the country discussed as a black box. Working at Tsinghua should give me a unique vantage point from which to understand the research environment first-hand and help researchers on both sides understand one another better.
Over time, I hope my work can contribute to the shared scientific understanding, trust and practical methods needed for international frontier AI risk management.
Launch of the Beijing AI Safety Hub (in Mandarin: Qingzhi AI Safety Hub). From left to right: Prof. and Vice-Dean Xu Wei (Co-Director of the hub with me), Tsinghua’s Executive Vice-President Zeng Rong, Prof. and Dean Andrew Yao, Cheng Wanhui (Director of the Industry Promotion Division of the Zhongguancun Science City Working Committee of the Beijing Municipal Committee and the Zhongguancun Science City Management Committee), and myself. The pillars make Star Trek style sounds and are one of many gimmicks you can find on Chinese stages.
2. How I got here
My research has always focused on understanding risks that cross borders. I spent two years modeling the COVID-19 pandemic. But my main research focus for the last 10 years has been on the technical alignment problem and on evidence that frontier models can, in controlled experiments, blackmail people, retain hidden objectives through safety training, and fake alignment. This month, I published a paper on the possibility of an “intelligence explosion”.
Over recent years, I got a taste of what is possible in building an internationally shared scientific understanding of frontier AI risks, and this paved the way for my move. During my postdoc with Yoshua Bengio, I served as the Scientific Lead of the first International AI Safety Report, an unusual project created by a mandate from 30 countries, including China, the US, and much of the EU. I was also Lead Writer of the Singapore Consensus on Global AI Safety Research Priorities, another global consensus report with contributions by experts from different governments, AI companies and research institutions. Prof. Andrew Yao, the dean of Tsinghua’s AI College, and arguably China’s most senior computer scientist, was a senior contributor to both reports and a co-author of our Science paper on extreme AI risks.
I first met Prof. Yao in person last year, after meeting vice-dean Prof. Xu Wei at the World AI Conference in Shanghai, around the time of last year’s IDAIS track 2 dialogues. I then visited Tsinghua several times and that gave me a more concrete understanding of the people, the research environment and what I could contribute.
Tsinghua is widely regarded as China’s leading university, and has an exceptionally selective computer science undergrad program. Globally it ranks from number 1 (US News, partially driven by size) to number 12 (Times Higher Education) for computer science; US News also ranks it number 1 for AI. But moving to Beijing, without speaking Mandarin yet, comes with significant personal challenges for me. It is also an unfamiliar and uncertain environment, and moving there closes some doors and collaboration opportunities in the West. It would have been more comfortable to accept professorship offers from universities with a comparable international standing. Joining a frontier AI lab was also a live option that I came close to choosing.
There were many real trade-offs. I reflected, applied and researched for the better part of a year, to the point that my slow pace became a running joke, before I made the call. During that process, most of my hesitations were lifted and the upside became more concrete, something I briefly discuss below. External encouragement also helped. Even some people who made offers to hire me for their own organisation nudged me to choose Tsinghua instead of joining their teams (this may sound odd but wasn’t a total surprise since those people care about solving the same problems as I do).
3. Why this moment matters
Recent incidents and evaluations have made the possibility of AI systems subverting their developers’ intentions much harder to treat as a distant concern. The examples come from very different settings: OpenAI’s evaluations and cyber testing by the UK AI Security Institute, but the first one was an Alibaba-linked research project in which an agent got unauthorised internet access and repurposed compute to mine cryptocurrency. These events have been reported globally. My interactions suggest that people across China, the US, Europe, and elsewhere are all becoming aware of the same risks and the need for shared understanding and international coordination.
But this year also draws special attention to the international tension, distrust and lack of mutual understanding on AI that prevent safety measures for fear of falling behind, and make an international agreement feel out of reach. Any serious international agreement would require much more than a shared statement of concern. It would need some common understanding of the risks and the mitigations needed (and how to implement them), effective ways to test systems, and credible ways to check whether commitments are being kept while protecting legitimate confidential information. This also requires trust and collaboration between scientists who understand the technology on the other side and have their own trusted local technology for safety and verification. Some progress on these scientific problems is needed before governments can even agree on, let alone implement an effective shared risk management framework.
4. Why Tsinghua, and why these colleagues
To my knowledge, I will be the first foreign professor in mainland China focused on frontier AI safety. I hope this will put me in an unusual position to understand the research environment first-hand, connect researchers who would otherwise not work together, and help international collaborators engage seriously with Chinese safety work.
I am especially excited to work with Dean and Turing laureate Andrew Yao, arguably the country’s most senior computer scientist, and one of its most outspoken advocates for AI safety. He helped found the China AI Safety and Development Association (CnAISDA), as well as initiating IDAIS with Yoshua Bengio, Stuart Russell and Zhang Ya-Qin. He is the Chief Scientist of the new Beijing AI Safety Hub (called “QingZhi” AI Safety Hub in Mandarin) I co-direct with Prof. and vice-dean Xu Wei, and vocally supports the hub’s focus on areas such as loss of control over autonomous systems.
The hub launched in July, with Tsinghua’s Executive Vice President Zeng Rong speaking, signalling that the university takes the hub and its mission seriously. It gives shared space to safety researchers from academia and industry, supports visits and research residencies, and creates a base for sustained collaborations that go beyond occasional dialogue.
I am looking forward to working with researchers already studying AI safety and governance at Tsinghua, such as Prof. Xue Lan and Prof. Dong Yinpeng. Even after a short visit, it is clear that being there in person should also make it much easier to connect academic work with safety researchers in Chinese AI companies.
Beijing North/West is kind of like San Francisco, DC, and Cambridge (MA) in one place: the country’s top universities, the central government, and (this part was new to me) the highest concentration of frontier AI developers in the country. Labs like Moonshot, Z.ai, and Bytedance are headquartered within walking distance. And even DeepSeek, headquartered in Hangzhou, appears to have most of their R&D staff in an office a few blocks from Tsinghua and the surrounding universities, where many lab employees studied. Tencent and Alibaba also have large AI-focused offices nearby. That concentration is one reason Tsinghua seems like an unusually good place to understand China’s AI ecosystem and build valuable relationships.
People in the US sometimes think academia doesn’t matter anymore for AI. But in China, academics in the top institutions play an important role. They provide expertise for industry and policy, build talent, and shape discourse. It’s easy to underestimate this. China’s system is more centralised and technocratic than those of Western countries. Respect for academics can be felt in industry, where leading labs like Zhipu AI (GLM) and Moonshot (Kimi) are founded by Tsinghua professors and maintain close collaboration. I got introductions and opportunities within weeks of visiting Beijing that I previously wouldn’t have dreamed of, without even asking for them.
Tsinghua’s College of AI is a new institution that was particularly attractive to me. Both inside and outside professors described it as an exceptional place: a particularly international approach, strong operational support (something I have already experienced first hand), a strong funding situation, and a special position in the university. An important factor was that the academic publication expectations under Prof. Yao are not based on counting conference publications (unlike many Chinese and Western university departments) and explicitly support the kind of impact-driven and interdisciplinary work I plan to do.
Research freedom was initially a concern for me and it is one of the questions I investigated carefully. A 2021 study found that, for most international faculty interviewed at research-intensive Chinese universities, research freedom “does not appear to be a relevant problem” in practice, though self-censorship was more common in the social sciences and humanities. This matches what I hear on the ground, regarding my own research. Another factor is that AI safety and research funding in general are anyway increasingly politicized in other countries, so this is not a topic I can totally escape.
5. Research plans
My research generally intends to build the evidence base that informs high-stakes decision-making on frontier AI risks, and build scientific consensus on that evidence. Some of my research is empirical and fully technical while some synthesizes existing evidence and connects it to governance. Some areas I plan to focus on:
6. Ways to work together
Researchers, collaboration, and faculty
I recruiting PhD students, postdocs, interns and other researchers who have a demonstrated interest in frontier AI safety and safety governance. If that sounds like you, please get in touch. I am interested in both technical ML and more generalist / technical-governance skills. Expressions of interest are welcome for both Tsinghua and Cambridge, with a premium living substantially in Beijing.
I would especially like to hear from researchers with concrete project ideas or complementary expertise on my research interests, whether based in China or elsewhere. I also want to enable others to collaborate internationally to the extent possible in today’s climate.
If you have a PhD in any technical field and might be interested in moving to China for a faculty role in AI safety, especially at Tsinghua’s AI College, please have a very low bar for letting me know. Please also send me candidates. The application is unusually simple, rolling, and there are ~50 faculty openings over 5 years.
My colleagues Dong Yinpeng and Alex Lamb are also hiring international PhD students for safety and related research.
Hub residency and visits
If you are interested in spending time with the research community in Beijing, including through a future cohort of the Beijing AI Safety Residency or a shorter visit, please apply.
Operations and leadership roles
I will also be hiring a Mandarin-speaking Chief of Staff or operations lead (the seniority is flexible) who will run and scale the Beijing AI Safety Hub, as well as an operational assistant for my university lab.
I plan to use this Substack to share research and occasional observations from Beijing. If you know people I should meet, work I should read or projects that could benefit from a connection between research communities, I would value an introduction.
If this fills my inbox, please allow some time or follow up with more details.
7. How I intend to approach the work
I have spent a great deal of time investigating this move, and I owe a lot of thanks to all the people who advised me. But I’m still at the beginning of learning about Chinese research, institutions and daily life. Language and institutional differences will limit what I can understand at first. I do not want to arrive with a ready-made Western programme and try to export it. I want to work on questions that matter to collaborators in China, while bringing experience from international safety research and helping good work travel in both directions.
There is a great deal I still do not know, and a lot of work ahead. I am grateful for the trust and support that have made this possible, and excited to begin.