All of Erik Istre's Comments + Replies

I should clarify that we only intend to design pathways that either complement/augment/supplement existing resources. As you rightly point out, safety-oriented ML is more saturated with much better material from experts. I don't plan to replace that, only to provide a pathway through it some day. I see my role at least as providing tools for someone to verify their knowledge, to challenge it, and to never be lost about where they need to go next.

I have the analogy in mind of a mountain range and different approaches to guiding people through that mou... (read more)

What's currently present is mostly a sketch of a part of what we intend to do with it. We do eventually plan to extend into machine learning as well.

The limitation at the time was that my academic background was purely in foundations of mathematics research, and so the MIRI approach was a more natural starting point. I am working on remedying these gaps in my knowledge :)

5Raemon
It also seems to me like it's easier to find resources on ML stuff (since it's an actual existing field) moreso than the agent-foundations paradigm. (I'm not very confident in this, maybe safety-oriented ML is just as ambiguous as agent-foundations?) It also seems like this is a pretty hard thing to do well in the first place, and probably benefits from do-one-thing-well. If Safety-Oriented ML needs better introductory texts too, I'd guess it's better for someone(s) with the relevant skills to focus more deeply on that, and for toon/erik et al to focus on iterating on their current project for awhile.