LESSWRONG
Wikitags
LW

1

Information theory

1

Written by So8res, RobertM last updated 19th Feb 2025

The study (and quantification) of information, and its communication and storage.

Parents:

Children:

Fractional bits

and 13 more

Posts tagged Information theory

11

86Entropy, and Short Codes

Eliezer Yudkowsky

17y

27

10

206The Second Law of Thermodynamics, and Engines of Cognition

Eliezer Yudkowsky

17y

76

9

168Six (and a half) intuitions for KL divergence

CallumMcDougall

3y

27

9

66Searching for Bayes-Structure

Eliezer Yudkowsky

17y

49

8

237Introduction to abstract entropy

2y

78

3

214Utility Maximization = Description Length Minimization

4y

44

2

61You Can’t Predict a Game of Pinball

Jeffrey Heninger

2y

13

1

58Deconfusing Landauer's Principle

3y

15

1

44What's the Right Way to think about Information Theoretic quantities in Neural Networks?

Dalcy, johnswentworth

3mo

13

1

42Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs

Kola Ayonrinde, Michael Pearce, Lee Sharkey

8mo

8

1

17Law without law: from observer states to physics via algorithmic information theory

7y

6

1

13Revisiting the Manifold Hypothesis

2y

19