Nate Showell

What actions have you been taking to prepare for the possibility that the AI industry will experience a market crash, something along the lines of the dotcom bust of the early 2000s? Also, what actions would you take if a crash like that occurred? For example: * If your career...

Goodhart's Law refers to the tendency that when someone sets a performance metric for a goal, the metric itself becomes a target of optimization, often at the expense of the goal it's supposed to measure. Some metrics are subject to imperfectly-aligned incentives in ways that are easy to identify, such...

Edit: I no longer agree with most of this post, due to the arguments given in Reward is not the optimization target. Epistemic status: still early in the process of exploring this. Goal-content integrity is frequently listed as one of the convergent instrumental goals that any AGI is likely to...