x

LESSWRONG
LW

Zachary Robertson — LessWrong

Zachary Robertson

Top postsTop post

Zachary Robertson

Message

PhD @ Stanford | Auditing LLM evaluation, incentives, and measurement without ground truth

Blog/About - https://zrobertson466920.github.io/about/

441

4

114

8y

Zachary Robertson

PhD @ Stanford | Auditing LLM evaluation, incentives, and measurement without ground truth

Blog/About - https://zrobertson466920.github.io/about/

Some Remarks on Regulator Theorems No One Asked For

Epistemological Status: Correct, but lazily written. Basically, a while ago John made a post attempting to fix the Good Regulator Theorem. I've refrained from writing this simply because I know it may come off as "why does anyone care isn't this trivial"? However, more recently John has set up a...

Nov 5, 2021•19

Minimal Maps, Semi-Decisions, and Neural Representations

Epistemological Status: I attempt to distill the minimal map argument and provide a natural way to chain a sequence of queries together. I argue that this definition, together with reasonable symmetry constraints on the minimal map, provides an intuitive explanation for the appearance of neural networks. Minimal Maps Say we...

Dec 6, 2020•30

You Can Probably Amplify GPT3 Directly

Epistemological Status: The idea is un-original and the implementation is the first I'm aware of. I'm reasonably confident anyone (patient) with an hour to burn could reproduce these results. You can probably amplify GPT3 directly. Let me cut the suspense. I can get GPT3 to sum the items in a...

Jul 26, 2020•34

Replication Dynamics Bridge to RL in Thermodynamic Limit

Epistemological Status: I ran a basic simulation and this checks out. Specific details need refinement. Probably just a curiosity, but I think there's a chance there's a deeper connection here. I opted to post in the earlier stages incase I'm missing something big. In biology, the quasispecies equation is used...

May 18, 2020•6

Top postsTop post

You Can Probably Amplify GPT3 Directly

Epistemological Status: The idea is un-original and the implementation is the first I'm aware of. I'm reasonably confident anyone (patient) with an hour to burn could reproduce these results. You can probably amplify GPT3 directly. Let me cut the suspense. I can get GPT3 to sum the items in a list using recursion. I have a notebook here and Dungeon instances for the parts: factoring, amplification1, and amplification2. I'm suggesting I can combine different instances of GPT3 to implement primitive recursive functions. Moreover, I can get GPT3 to suggest approaches for implementation. Combined, this is a toy-example showing how to amplify GPT3's capabilities. At the moment, this method is slow, but does provide proof-of-concept. Obvious extensions include: prompting GPT3 to factor questions, ask questions, generate examples, induce belief states that encourage learning. Factoring In the following I don’t re-roll unless I say so. Anything omitted here is available in the links at the top. Mistakes have to be dealt with in script. This simplifies reproducibility. I'll bold my prompts and italicize for emphasis. Occasionally, GPT will get ahead of me and auto-complete questions. Before hand I decided that I'd cut off at the end of an answer, but that doesn't mean that I can't ask the same question if I think it was good. First, I state the problem, > Q: Say I want to sum the items in a list. How would I do this recursively? The answer involves two steps. > > A: > > 1. Take the item from the first step and put it into the second step. > 2. Repeat until you run out of items! B: You can't use recursion here because there are no more items to add, so you have to start over with Step 1 again. > Q: Could you explain step 1) in more detail? Here GPT3 fails to explain step one properly, > A: I will refer to the first step as the "base case". In Step 1, the base case is an empty list. In order to put an item into a list, you have to have a list in the first place! >

Minimal Maps, Semi-Decisions, and Neural Representations

Some Remarks on Regulator Theorems No One Asked For

Replication Dynamics Bridge to RL in Thermodynamic Limit