LESSWRONG
LW

All of zookini's Comments + Replies

Redundant Attention Heads in Large Language Models For In Context Learning

each previous example confirming the pattern attends to the $i$ th example, and adds a constant update term (equivilant to $log (c) + n o r m$ ) to the $i$ th example.

Nit: should be \log(c) - norm

1skunnavakkam11d

norm \in \mathbf{R}, doesn't matter

The Best Tacit Knowledge Videos on Every Subject

zookini1y20

Tacit knowledge videos for CAD modelling:
https://www.youtube.com/playlist?list=PLzMIhOgu1Y5fwotlIEKNnuIXcEbVIZ7Qm

1Parker Conley1y

Thanks! Added.