LESSWRONGTags
LW

Transformers

EditHistorySubscribe

Help improve this page

EditHistorySubscribe

Help improve this page

Contributors

Posts tagged Transformers

5

37Striking Implications for Learning Theory, Interpretability — and Safety?

11mo

4

3

131How LLMs are and are not myopic

1y

15

2

219Modern Transformers are AGI, and Human-Level

8mo

88

2

87Google's PaLM-E: An Embodied Multimodal Language Model

2y

7

2

76Residual stream norms grow exponentially over the forward pass

StefanHex, TurnTrout

2y

24

2

62Tracr: Compiled Transformers as a Laboratory for Interpretability | DeepMind

2y

12

2

56Concrete Steps to Get Started in Transformer Mechanistic Interpretability

2y

7

2

53How fast can we perform a forward pass?

2y

9

2

33AGI will be made of heterogeneous components, Transformer and Selective SSM blocks will be among them

11mo

9

2

27How Do Induction Heads Actually Work in Transformers With Finite Capacity?

2y

0

2

7If I ask an LLM to think step by step, how big are the steps?

2mo

1

1

411Transformers Represent Belief State Geometry in their Residual Stream

7mo

100

1

89An Analogy for Understanding Transformers

CallumMcDougall

2y

6

1

77Attention SAEs Scale to GPT-2 Small

Connor Kissane, robertzk, Arthur Conmy, Neel Nanda

10mo

4

1

70Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream

Diego Caples, rrenaud

2mo

7