But the "strong convergence of values" thesis hasn't really been argued, so I remain unclear as to why Eliezer finds it plausible.
Hasn't been argued and seems quite implausible to me.
I find moral realism meaningful for each individual (you can evaluate choices according to my values applied with infinite information and infinite resources to think), but I don't find it meaningful when applied to groups of people, all with their own values.
EY finesses the point by talking about an abstract algorithm, and not clearly specifying what that algorithm actually implements, whether my values, yours, or some unspecified amalgamation of the values of different people. So that the point of moral subjectivism vs. moral universalism is left unspecified, to be filled in by the imagination of the reader.To my ear, sometimes it seems one way, and sometimes the other. My guess was that this was intentional, as clarifying the point wouldn't take much effort. The discussions of EY's metaethics always strike me as peculiar, as he's wandering about here somewhere while people discuss how they're unclear just what conclusion he had drawn.
I find moral realism meaningful for each individual (you can evaluate choices according to my values applied with infinite information and infinite resources to think),
I can how that could be implemented. However, I don't see how that would count as morality. It amounts to Anything Goes, or Do What Thou Wilt. I don't see how a world in which that kind of "moral realism" holds would differ from one where moral subjectivism holds, or nihilism for that matter.
...but I don't find it meaningful when applied to groups of people, all with their own v
There seems to be a widespread impression that the metaethics sequence was not very successful as an explanation of Eliezer Yudkowsky's views. It even says so on the wiki. And frankly, I'm puzzled by this... hence the "apparently" in this post's title. When I read the metaethics sequence, it seemed to make perfect sense to me. I can think of a couple things that may have made me different from the average OB/LW reader in this regard: