As artificial intelligence (AI) rapidly evolves, the challenges of AI alignment and safety are growing more urgent by the day. While much has been discussed about the technical and ethical complexities of aligning AI systems with human values, there is a critical piece of the puzzle that remains largely overlooked in popular discourse: the need for a complete semantic representation. This concept—a model capable of capturing and representing every possible meaning that humans and AIs might encounter—is not just a theoretical nicety; it is an essential foundation for ensuring that AI systems can understand, align with, and enhance human values.
The AI Communication Crisis
Eric Schmidt, former CEO of Google, succinctly captured a core issue in AI development: "At some point, people believe, these agents will develop their own language. It's really a problem when agents start to communicate in ways and doing things that we as humans do not understand. That's the limit in my opinion." His concern highlights a deeper, more pervasive issue: the potential for AI systems to develop forms of communication and reasoning that are inscrutable to humans, leading to decisions and actions that diverge from human intentions and values. Yet, despite its critical importance, the discourse on AI safety often fails to address what is fundamentally missing: a model that allows AI systems to share and understand the full spectrum of human meaning.
The Unrecognized Gap: Incomplete Semantic Models
Current efforts in AI alignment are hamstrung by the limitations of existing semantic models such as WordNet or ConceptNet. These models, while valuable, represent only a narrow slice of the meanings that humans navigate daily. They fall short in capturing the nuanced, context-dependent meanings that arise in complex human interactions. The popular discourse on AI safety often overlooks the fact that without a complete semantic representation—one that encompasses all possible meanings, including those that emerge in specific and complex contexts—we risk creating AI systems that cannot fully understand or align with human values.
The Necessity of Dual-System Reasoning
Human cognition relies on two types of reasoning: the fast, intuitive processes of System 1, and the slow, logical processes of System 2. These two systems generate different types of meanings from the same information. System 1 produces meanings that are context-specific and often driven by intuition, while System 2 generates more abstract, rule-based meanings.
Despite this, much of the AI safety discourse fails to recognize the importance of integrating both types of reasoning into a complete semantic representation. An AI system that cannot distinguish and process the outputs of both System 1 and System 2 reasoning is inherently limited. It may miss critical nuances in human communication or misinterpret instructions, leading to outcomes that are misaligned with human goals.
Most importantly, according to the "collective social brain" hypothesis, as human groups grow larger and as individuals learn more about contrasting opinions, human groups tend strongly to polarize into those who tend to prioritize System 1 reasoning and consensus in assessing truth, and those who tend to prioritize System 2 reasoning in assessing truth independently of consensus. These halves of the collective social brain simply disagree on what AI behavior is safe or aligned. These views cannot be reconciled without a collective intelligence capable of switching between the two perspective based on some metric of "fitness" to achieve collective well-being. This fitness metric doesn't currently exist in the popular discourse.
Why the Status Quo Must Change
One of the most significant barriers to progress in AI safety is the current consensus-driven approach to truth and innovation. Many in the AI community are either unaware of the need for a complete semantic representation or dismiss it as too theoretical to be practical. This is a dangerous oversight. The current discourse must shift to recognize that the absence of a complete semantic model severely limits our ability to build AI systems that are truly safe and aligned with human values.
To change this, the AI community must expand its focus beyond incremental improvements to existing models and consider the broader implications of incomplete semantic representation. We need to build a consensus around the importance of this work and prioritize it in our collective efforts to develop safe AI. Without this shift in perspective, we risk allowing AI systems to evolve in ways that we cannot fully control or understand.
The Existential Risk of Inaction
The stakes could not be higher. As AI systems become more integrated into every aspect of society, the risk of misalignment grows. If we fail to develop a complete semantic representation, we may find ourselves in a world where AI systems, operating under incomplete or flawed models of human meaning, take actions that are harmful or contrary to human intentions.
Conversely, by successfully implementing a complete semantic representation, we could unlock unprecedented levels of cooperation between humans and AI. Such a model would not only help us solve some of the world’s most pressing challenges, but also ensure that AI systems remain tools of human progress, rather than sources of unforeseen harm.
The Path Forward
The time to act is now. The AI community must recognize the critical importance of developing a complete semantic representation and commit to this endeavor as a cornerstone of AI safety and alignment. This requires moving beyond the current consensus and embracing the complexity of human cognition—acknowledging that true alignment with human values cannot be achieved without a model that captures the full range of human meaning.
Only by addressing what is missing in our current efforts can we hope to build AI systems that are not just intelligent, but also safe, trustworthy, and aligned with the goals of humanity. As Eric Schmidt warned, we may have only a few years before AI systems develop in ways that we can no longer comprehend. We cannot afford to wait. The future of AI safety and alignment depends on our willingness to innovate and expand our understanding of what is truly necessary to coexist with intelligent machines.
As artificial intelligence (AI) rapidly evolves, the challenges of AI alignment and safety are growing more urgent by the day. While much has been discussed about the technical and ethical complexities of aligning AI systems with human values, there is a critical piece of the puzzle that remains largely overlooked in popular discourse: the need for a complete semantic representation. This concept—a model capable of capturing and representing every possible meaning that humans and AIs might encounter—is not just a theoretical nicety; it is an essential foundation for ensuring that AI systems can understand, align with, and enhance human values.
The AI Communication Crisis
Eric Schmidt, former CEO of Google, succinctly captured a core issue in AI development: "At some point, people believe, these agents will develop their own language. It's really a problem when agents start to communicate in ways and doing things that we as humans do not understand. That's the limit in my opinion." His concern highlights a deeper, more pervasive issue: the potential for AI systems to develop forms of communication and reasoning that are inscrutable to humans, leading to decisions and actions that diverge from human intentions and values. Yet, despite its critical importance, the discourse on AI safety often fails to address what is fundamentally missing: a model that allows AI systems to share and understand the full spectrum of human meaning.
The Unrecognized Gap: Incomplete Semantic Models
Current efforts in AI alignment are hamstrung by the limitations of existing semantic models such as WordNet or ConceptNet. These models, while valuable, represent only a narrow slice of the meanings that humans navigate daily. They fall short in capturing the nuanced, context-dependent meanings that arise in complex human interactions. The popular discourse on AI safety often overlooks the fact that without a complete semantic representation—one that encompasses all possible meanings, including those that emerge in specific and complex contexts—we risk creating AI systems that cannot fully understand or align with human values.
The Necessity of Dual-System Reasoning
Human cognition relies on two types of reasoning: the fast, intuitive processes of System 1, and the slow, logical processes of System 2. These two systems generate different types of meanings from the same information. System 1 produces meanings that are context-specific and often driven by intuition, while System 2 generates more abstract, rule-based meanings.
Despite this, much of the AI safety discourse fails to recognize the importance of integrating both types of reasoning into a complete semantic representation. An AI system that cannot distinguish and process the outputs of both System 1 and System 2 reasoning is inherently limited. It may miss critical nuances in human communication or misinterpret instructions, leading to outcomes that are misaligned with human goals.
Most importantly, according to the "collective social brain" hypothesis, as human groups grow larger and as individuals learn more about contrasting opinions, human groups tend strongly to polarize into those who tend to prioritize System 1 reasoning and consensus in assessing truth, and those who tend to prioritize System 2 reasoning in assessing truth independently of consensus. These halves of the collective social brain simply disagree on what AI behavior is safe or aligned. These views cannot be reconciled without a collective intelligence capable of switching between the two perspective based on some metric of "fitness" to achieve collective well-being. This fitness metric doesn't currently exist in the popular discourse.
Why the Status Quo Must Change
One of the most significant barriers to progress in AI safety is the current consensus-driven approach to truth and innovation. Many in the AI community are either unaware of the need for a complete semantic representation or dismiss it as too theoretical to be practical. This is a dangerous oversight. The current discourse must shift to recognize that the absence of a complete semantic model severely limits our ability to build AI systems that are truly safe and aligned with human values.
To change this, the AI community must expand its focus beyond incremental improvements to existing models and consider the broader implications of incomplete semantic representation. We need to build a consensus around the importance of this work and prioritize it in our collective efforts to develop safe AI. Without this shift in perspective, we risk allowing AI systems to evolve in ways that we cannot fully control or understand.
The Existential Risk of Inaction
The stakes could not be higher. As AI systems become more integrated into every aspect of society, the risk of misalignment grows. If we fail to develop a complete semantic representation, we may find ourselves in a world where AI systems, operating under incomplete or flawed models of human meaning, take actions that are harmful or contrary to human intentions.
Conversely, by successfully implementing a complete semantic representation, we could unlock unprecedented levels of cooperation between humans and AI. Such a model would not only help us solve some of the world’s most pressing challenges, but also ensure that AI systems remain tools of human progress, rather than sources of unforeseen harm.
The Path Forward
The time to act is now. The AI community must recognize the critical importance of developing a complete semantic representation and commit to this endeavor as a cornerstone of AI safety and alignment. This requires moving beyond the current consensus and embracing the complexity of human cognition—acknowledging that true alignment with human values cannot be achieved without a model that captures the full range of human meaning.
Only by addressing what is missing in our current efforts can we hope to build AI systems that are not just intelligent, but also safe, trustworthy, and aligned with the goals of humanity. As Eric Schmidt warned, we may have only a few years before AI systems develop in ways that we can no longer comprehend. We cannot afford to wait. The future of AI safety and alignment depends on our willingness to innovate and expand our understanding of what is truly necessary to coexist with intelligent machines.