Bogdan Ionut Cirstea

[Linkpost] Personal and Psychological Dimensions of AI Researchers Confronting AI Catastrophic Risks

This is a linkpost for https://yoshuabengio.org/2023/08/12/personal-and-psychological-dimensions-of-ai-researchers-confronting-ai-catastrophic-risks. Yoshua Bengio: > For most of these years, I did not think about the dual-use nature of science because our research results seemed so far from human capabilities and the work was only academic. It was a pure pursuit of knowledge, beautiful, but mostly detached from society until about a decade ago. I now believe that I was wrong and short-sighted to ignore that dual-use nature. I also think I was not paying enough attention to the possibility of losing control to superhuman AIs. > [...] it started to dawn on me that my previous estimates of when human-level AI would be reached needed to be radically changed. Instead of decades to centuries, I now see it as 5 to 20 years with 90% confidence. > And what if it was, indeed, just a few years? > I started reading more about AI safety and came to a critically important conclusion: we do not yet know how to make an AI agent controllable and thus guarantee the safety of humanity! And yet we are – myself included until now – racing ahead towards building such systems. > It is painful to face the idea that we may have been contributing to something that could be greatly destructive. Human nature will lead us towards brushing aside these thoughts or finding comfort in reassuring arguments rather than face the full horror of such possibilities. Bringing the benefits of AI to the table is not sufficient to compensate if the possible negative outcomes include catastrophic misuses of AI on par with nuclear war and pandemics, or even existential risk. > As scientists, we should avoid making claims we can’t support; but as decision-makers we also ought to act under uncertainty to take precautions. In spite of our differences in points of view, it’s time for our field of AI to seriously discuss the questions: what if we succeed? What if potentially dangerous superhuma

42Aug 12, 2023

Bogdan Ionut Cirstea

Message

Automated / strongly-augmented safety research.

1775

483

Densing Law of LLMs

Authors: Chaojun Xiao, Jie Cai, Weilin Zhao, Guoyang Zeng, Xu Han, Zhiyuan Liu, Maosong Sun. Abstract (bolding mine): > Large Language Models (LLMs) have emerged as a milestone in artificial intelligence, and their performance can improve as the model size increases. However, this scaling brings great challenges to training and...

Dec 8, 20249

LLMs Do Not Think Step-by-step In Implicit Reasoning

Author: Yijiong Yu. Abstract: > It has been well-known that Chain-of-Thought can remarkably enhance LLMs’ performance on complex tasks. However, because it also introduces slower inference speeds and higher computational costs, many researches have attempted to use implicit CoT, which does not need LLMs to explicitly generate the intermediate steps....

Nov 28, 202411

Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?

Authors: Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, Mor Geva. Abstract: > We evaluate how well Large Language Models (LLMs) latently recall and compose facts to answer multi-hop queries like "In the year Scarlett Johansson was born, the Summer Olympics were hosted in the country of". One major challenge...

Nov 26, 20249

Disentangling Representations through Multi-task Learning

Authors: Pantelis Vafidis, Aman Bhargava, Antonio Rangel. Abstract: > Intelligent perception and interaction with the world hinges on internal representations that capture its underlying structure ("disentangled" or "abstract" representations). Disentangled representations serve as world models, isolating latent factors of variation in the world along orthogonal directions, thus facilitating feature-based generalization....

Nov 24, 202414

Reward Bases: A simple mechanism for adaptive acquisition of multiple reward type

Authors: Beren Millidge ,Yuhang Song, Armin Lak, Mark E. Walton, Rafal Bogacz. Abstract: > Animals can adapt their preferences for different types of reward according to physiological state, such as hunger or thirst. To explain this ability, we employ a simple multi-objective reinforcement learning model that learns multiple values according...

Nov 23, 202411

A Little Depth Goes a Long Way: the Expressive Power of Log-Depth Transformers

Authors: Anonymous (I'm not one of them). Abstract: > Most analysis of transformer expressivity treats the depth (number of layers) of a model as a fixed constant, and analyzes the kinds of problems such models can solve across inputs of unbounded length. In practice, however, the context length of a...

Nov 20, 202416

The Computational Complexity of Circuit Discovery for Inner Interpretability

Authors: Federico Adolfi, Martina G. Vilas, Todd Wareham. Abstract: > Many proposed applications of neural networks in machine learning, cognitive/brain science, and society hinge on the feasibility of inner interpretability via circuit discovery. This calls for empirical and theoretical explorations of viable algorithmic options. Despite advances in the design and...

Oct 17, 202411

Load More (7/38)

LESSWRONG
LW

LESSWRONG
LW

Bogdan Ionut Cirstea

Bogdan Ionut Cirstea

Bogdan Ionut Cirstea

[Linkpost] Personal and Psychological Dimensions of AI Researchers Confronting AI Catastrophic Risks

[Linkpost] Large Language Models Converge on Brain-Like Word Representations

[Linkpost] Scaling laws for language encoding models in fMRI

[Linkpost] Concept Alignment as a Prerequisite for Value Alignment

Bogdan Ionut Cirstea

Densing Law of LLMs

LLMs Do Not Think Step-by-step In Implicit Reasoning

Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?

Disentangling Representations through Multi-task Learning

Reward Bases: A simple mechanism for adaptive acquisition of multiple reward type

A Little Depth Goes a Long Way: the Expressive Power of Log-Depth Transformers

The Computational Complexity of Circuit Discovery for Inner Interpretability

[Linkpost] Personal and Psychological Dimensions of AI Researchers Confronting AI Catastrophic Risks

[Linkpost] Large Language Models Converge on Brain-Like Word Representations

[Linkpost] Scaling laws for language encoding models in fMRI

[Linkpost] Concept Alignment as a Prerequisite for Value Alignment

Densing Law of LLMs

LLMs Do Not Think Step-by-step In Implicit Reasoning

Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?

Disentangling Representations through Multi-task Learning

Reward Bases: A simple mechanism for adaptive acquisition of multiple reward type

A Little Depth Goes a Long Way: the Expressive Power of Log-Depth Transformers

The Computational Complexity of Circuit Discovery for Inner Interpretability