Good summary. But concerning your final point :

For this approach like the others, it seems important to make the most progress toward learning human values in a way that doesn't require a very good model of the world.

I suspect this is impossible in principle, because human values are dependent on our models of the world.

The key is to develop methods that scale; where values become aligned as the world model approaches human level of capability.