Isn't the real concern that people will misuse AI?
Misuse is a problem with every powerful technology. Fire can be used to cook food or to burn down a village, and nuclear reactions can be used to produce electricity or city-devastating explosions.
Misuse of AI has the potential to cause harm at the individual, societal, and global levels by enabling:
One of the aims of AI governance is to find ways to prevent such harm.
However, misuse is not the only risk from AI. Even when used with the best of intentions, technology can cause accidents: a fire can spread to burn down a city, and a reactor can melt down.
Accidents with advanced AI could be particularly bad when AI can pursue its own goals, since we currently don’t know how to specify safe goals and have highly advanced systems follow them. Future AI could be powerful enough to outmaneuver humanity, and accidents resulting from misalignment could threaten human civilization as a whole.
In other words, for advanced AI to go well, we will need to both:
Coordinate to prevent bad actors from misusing AI.
Ensure, by solving the alignment problem, that powerful AI can be safely used at all, without causing harms that no human intended.
Deepfakes are synthetic media (usually images, but sometimes video or audio) that have been digitally manipulated with AI tools.
A deepfake of Pope Francis in a Balenciaga-inspired white puffer coat.
The generation of synthetic media has been possible for years due to image editing software. The recent improvement in AI capabilities has transformed this task from a slow task, requiring familiarity with tools like Photoshop, to one that can be done cheaply with no special skill. While such manipulation was previously mostly restricted to images, AI tools also have made it possible to create audio and video deepfakes. As of 2024, some major categories of harmful deepfakes are propaganda, pornography, and impersonation.
Deepfakes can be generated for use in propaganda campaigns aimed at large audiences. A typical case would be a doctored image of a politician intended to make them look bad.[1] There are concerns that such media could flood social media and become a major influence on political discourse.
Pornographic deepfakes[2] are photorealistic media that depict people unclothed or in sexualized positions. The subjects are predominantly women[3] and sometimes minors. They may be superstars, ordinary people undressed by people who know them[4], or people who never existed.[5] Such images are often generatedby fine-tuned versions of open-weight AI models.
Finally, there are scams of impersonation. A person being impersonated could be a famous person that is made to support some shady product. Scams could also be more customized to their targets, such as your employer calling you with a request to transfer a sum of your company’s money, or a loved one with a desperate plea for financial help.
Conversely, the prevalence of deepfakes could lead to authentic pieces of media being labeled as fake, e.g., by people who find them inconvenient.
All of these cases present new challenges that will need to be addressed both at the societal and individual levels.
Isn't the real concern technological unemployment?
Technological unemployment through AI, while not the kind of life-and-death risk for humanity that we focus on, is a real concern.
In the past, technology has often been blamed for causing unemployment within certain sectors. Weavers, buggy whip makers, and bank tellers lost their jobs when they were automated, but technology also produced new kinds of jobs or transformed the jobs that were made obsolete. These new and transformed jobs have required workers to learn new skills, which was possible when changes were somewhat slow.
Some fear that the rate of job automation through AI could be much higher than in the past, and could affect a much larger proportion of jobs.[6]In particular, AGI might replace humans in most white-collar jobs quickly without offering many replacement jobs, which would lead to a brutal transition and possibly social unrest.
Nevertheless, soon after AGI could automate these jobs, assuming humanity is not wiped out, we might enter a post-scarcity economy which would eliminate the need to work for subsistence. We might still expect people to want to work for personal fulfillment or to do tasks that we prefer be done by humans (e.g., tasks related to caring for other humans), but not needing to work to survive would probably help the transition.
We do not concentrate on technological unemployment because we fear that a superintelligence with the power to outcompete many of us in the labor market might also be able to kill us all, and we concentrate on the latter risk.
Isn't the real concern AI-enabled surveillance?
AI-enabled surveillance and control might be a dangerous form of AI misuse.
Some AI ethics-focused groups and thinkers have raised concerns about contemporary AI applications such as facial recognition and predictive policing being used to exert social control, especially targeting marginalized communities. These capabilities are expected to increase in the future, and some civil liberties organizations such as the EFF have been reporting on these uses of AI in both democracies and autocratic regimes.
Security expert Bruce Schneier has argued that AI is enabling a shift from general surveillance (e.g., pervasive use of CCTV) to personalized surveillance of any citizen. For instance, AI can quickly and cheaply search all phone calls and CCTV footage in a city to form a detailed profile on one individual, which was previously only possible through laborious human effort.[7] Furthermore, traditional spying could only gather information after the target was identified as a person of interest, whereas the combination of AI and mass recording allows for the inspection of a target’s behavior in the past.
As of early 2024, access to the most advanced models is moderated through API access by Western corporations, which allows these corporations to restrict uses of their models that they do not condone. These corporations are incentivized not to collaborate with totalitarian governments, or to authorize use of their models by projects perceived as authoritarian, lest they face public backlash. As capabilities increase and powerful models become more accessible to smaller actors[8], this state of affairs might change.
A number of prominent researchers who mainly focus on risks from misalignment (rather than misuse) nevertheless view AI-enabled surveillance as one of the most salient risks that could arise from near-term AI:
Daniel Kokotajlo has speculated that LLMs could be used as powerful persuasion tools to disproportionately aid authoritarian regimes.
Nick Bostrom has discussed the potential incentives for widespread surveillance systems augmented by AI, based on state responses to concerns about living in an extremely risky and “vulnerable” world.
Richard Ngo has claimed that outsourcing the task of maintaining control (e.g. through surveillance) to AI makes it easier to consolidate power, which in the limit leads to authoritarianism.
While it is important to undertake measures to mitigate these kinds of risks of AI misuse, it's not sufficient; even well-intentioned actors have the potential to accidentally pose an existential riskwhen deploying AGI because of misalignment.
Isn't the real concern with AI that it's biased?
Bias and discrimination[9] in current and future AI systems is one concern among a number of real issues, each of which deserves attention.
Bias in AI refers to systematic errors and distortions in the data and algorithms used to train AI systems that cause those systems to treat people inequitably. Note that this use of the term bias is different from the one used in statistics, which refers to a failure to correctly represent reality but not necessarily in a way that particularly affects certain groups.
The forms of bias in AIs today that are most discussed by the media are the ones that lead to racism[10][11][12][13] and sexism[14][15]. Other biases[16][17][18][19][20] have also been identified. These biases in AI are often a reflection of which societies are most heavily represented in the training data (such as English-speaking communities) as well as the biases within these societies.
That being said, we choose to concentrate on existential risk because we perceive the dangers of superintelligence to be both imminent and of the greatest importance.
Admittedly, there are also uses of such software to make images that are photorealistic but clearly parody. For instance, in this humoristic but informative video, the voices of recent US presidents are cloned in a discussion about ranking AI alignment agendas.
A 2019 report by Deeptrace found that 96% of deepfake videos were non-consensual pornography and that virtually all of these videos featured women as subjects. In contrast, non-pornographic deepfake videos on Youtube mostly depicted men. Another report from 2023 reported 98% of deepfake videos as pornographic and 99% of the subjects being women.
Isn't the real concern that people will misuse AI?
Misuse is a problem with every powerful technology. Fire can be used to cook food or to burn down a village, and nuclear reactions can be used to produce electricity or city-devastating explosions.
Misuse of AI has the potential to cause harm at the individual, societal, and global levels by enabling:
One of the aims of AI governance is to find ways to prevent such harm.
However, misuse is not the only risk from AI. Even when used with the best of intentions, technology can cause accidents: a fire can spread to burn down a city, and a reactor can melt down.
Accidents with advanced AI could be particularly bad when AI can pursue its own goals, since we currently don’t know how to specify safe goals and have highly advanced systems follow them. Future AI could be powerful enough to outmaneuver humanity, and accidents resulting from misalignment could threaten human civilization as a whole.
In other words, for advanced AI to go well, we will need to both:
Isn't the real concern deepfakes?
While it’s not one of the risks this website concentrates on, the proliferation of deepfakes is one of the risks from AI.
Deepfakes are synthetic media (usually images, but sometimes video or audio) that have been digitally manipulated with AI tools.
The generation of synthetic media has been possible for years due to image editing software. The recent improvement in AI capabilities has transformed this task from a slow task, requiring familiarity with tools like Photoshop, to one that can be done cheaply with no special skill. While such manipulation was previously mostly restricted to images, AI tools also have made it possible to create audio and video deepfakes. As of 2024, some major categories of harmful deepfakes are propaganda, pornography, and impersonation.
Deepfakes can be generated for use in propaganda campaigns aimed at large audiences. A typical case would be a doctored image of a politician intended to make them look bad.[1] There are concerns that such media could flood social media and become a major influence on political discourse.
Pornographic deepfakes[2] are photorealistic media that depict people unclothed or in sexualized positions. The subjects are predominantly women[3] and sometimes minors. They may be superstars, ordinary people undressed by people who know them[4], or people who never existed.[5] Such images are often generated by fine-tuned versions of open-weight AI models.
Finally, there are scams of impersonation. A person being impersonated could be a famous person that is made to support some shady product. Scams could also be more customized to their targets, such as your employer calling you with a request to transfer a sum of your company’s money, or a loved one with a desperate plea for financial help.
Conversely, the prevalence of deepfakes could lead to authentic pieces of media being labeled as fake, e.g., by people who find them inconvenient.
All of these cases present new challenges that will need to be addressed both at the societal and individual levels.
Isn't the real concern technological unemployment?
Technological unemployment through AI, while not the kind of life-and-death risk for humanity that we focus on, is a real concern.
In the past, technology has often been blamed for causing unemployment within certain sectors. Weavers, buggy whip makers, and bank tellers lost their jobs when they were automated, but technology also produced new kinds of jobs or transformed the jobs that were made obsolete. These new and transformed jobs have required workers to learn new skills, which was possible when changes were somewhat slow.
Some fear that the rate of job automation through AI could be much higher than in the past, and could affect a much larger proportion of jobs.[6] In particular, AGI might replace humans in most white-collar jobs quickly without offering many replacement jobs, which would lead to a brutal transition and possibly social unrest.
Nevertheless, soon after AGI could automate these jobs, assuming humanity is not wiped out, we might enter a post-scarcity economy which would eliminate the need to work for subsistence. We might still expect people to want to work for personal fulfillment or to do tasks that we prefer be done by humans (e.g., tasks related to caring for other humans), but not needing to work to survive would probably help the transition.
We do not concentrate on technological unemployment because we fear that a superintelligence with the power to outcompete many of us in the labor market might also be able to kill us all, and we concentrate on the latter risk.
Isn't the real concern AI-enabled surveillance?
AI-enabled surveillance and control might be a dangerous form of AI misuse.
Some AI ethics-focused groups and thinkers have raised concerns about contemporary AI applications such as facial recognition and predictive policing being used to exert social control, especially targeting marginalized communities. These capabilities are expected to increase in the future, and some civil liberties organizations such as the EFF have been reporting on these uses of AI in both democracies and autocratic regimes.
Security expert Bruce Schneier has argued that AI is enabling a shift from general surveillance (e.g., pervasive use of CCTV) to personalized surveillance of any citizen. For instance, AI can quickly and cheaply search all phone calls and CCTV footage in a city to form a detailed profile on one individual, which was previously only possible through laborious human effort.[7] Furthermore, traditional spying could only gather information after the target was identified as a person of interest, whereas the combination of AI and mass recording allows for the inspection of a target’s behavior in the past.
In the future, more powerful AI surveillance, along with other AI-enabled technologies like autonomous weapons, might allow authoritarian or totalitarian states to make dissent virtually impossible, potentially enabling the rise of a stable global totalitarian state.
As of early 2024, access to the most advanced models is moderated through API access by Western corporations, which allows these corporations to restrict uses of their models that they do not condone. These corporations are incentivized not to collaborate with totalitarian governments, or to authorize use of their models by projects perceived as authoritarian, lest they face public backlash. As capabilities increase and powerful models become more accessible to smaller actors[8], this state of affairs might change.
A number of prominent researchers who mainly focus on risks from misalignment (rather than misuse) nevertheless view AI-enabled surveillance as one of the most salient risks that could arise from near-term AI:
While it is important to undertake measures to mitigate these kinds of risks of AI misuse, it's not sufficient; even well-intentioned actors have the potential to accidentally pose an existential risk when deploying AGI because of misalignment.
Isn't the real concern with AI that it's biased?
Bias and discrimination[9] in current and future AI systems is one concern among a number of real issues, each of which deserves attention.
Bias in AI refers to systematic errors and distortions in the data and algorithms used to train AI systems that cause those systems to treat people inequitably. Note that this use of the term bias is different from the one used in statistics, which refers to a failure to correctly represent reality but not necessarily in a way that particularly affects certain groups.
The forms of bias in AIs today that are most discussed by the media are the ones that lead to racism[10][11][12][13] and sexism[14][15]. Other biases[16][17][18][19][20] have also been identified. These biases in AI are often a reflection of which societies are most heavily represented in the training data (such as English-speaking communities) as well as the biases within these societies.
Work to reduce existential risk is sometimes presented as opposed to work addressing bias in current systems, but the AI safety community's focus on existential risk doesn’t mean it’s unsympathetic to concerns about bias. Yoshua Bengio, who has worked on AI ethics for many years, rhetorically asks: “should we ignore future sea level rises from climate change because climate change is already causing droughts?” Humanity can address both classes of problems if it decides to prioritize them both. Furthermore, some research areas such as interpretability are useful toward both goals. On the governance side, there is some overlap in the techniques and institutions to make AI fair and to make AI safe.
That being said, we choose to concentrate on existential risk because we perceive the dangers of superintelligence to be both imminent and of the greatest importance.
Admittedly, there are also uses of such software to make images that are photorealistic but clearly parody. For instance, in this humoristic but informative video, the voices of recent US presidents are cloned in a discussion about ranking AI alignment agendas.
This category is sometimes called “non-consensual intimate deepfakes” or “non-consensual intimate imagery”.
A 2019 report by Deeptrace found that 96% of deepfake videos were non-consensual pornography and that virtually all of these videos featured women as subjects. In contrast, non-pornographic deepfake videos on Youtube mostly depicted men. Another report from 2023 reported 98% of deepfake videos as pornographic and 99% of the subjects being women.
This can also be an instance of revenge porn.
In particular, the prevalence of generated child sex abuse material (CSAM) is overwhelming law enforcement departments that are responsible for such reports.
Others argue that this could be slower, for instance that a large portion of the work in tech jobs is not automatable by large language models (LLMs) even if they can program as well as a human.
Schneier calls this a shift from “mass surveillance” to “mass spying”, although other authors do not separate these two categories.
This could happen for instance when powerful models are open-sourced.
These fit within the larger concepts of AI Ethics and FATE (fairness, accountability, transparency, ethics).
Google Photos would tag black people as gorillas in 2015, and the problem was fundamentally hard to fix.
Racial discrimination appears in face recognition technology.
AI-assisted predictive policing and bail setting exhibits racial biases (among others).
Facial recognition is more prone to false-positives with Black faces, leading to wrongful arrests.
Most of the data used to train AIs comes from men's lives.
Women with good credit scores can get lower limits on their credit cards.
AI-assisted hiring practices might discriminate in problematic ways.
Data on older adults is sometimes excluded in training datasets for health-related AI applications.
AI correctly tags western bridal dresses as such, but not brides from other cultures.
AI-powered speech and movement tracking software used in hiring could be prejudiced against disabled people.
AI-powered gender and sexual orientation recognition might discriminate against people who don't fit well in the gender binary or might be used to actively discriminate LGBTQ people.