Zack_M_Davis comments on Holden's Objection 1: Friendliness is dangerous - Less Wrong

11 Post author: PhilGoetz 18 May 2012 12:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (428)

You are viewing a single comment's thread. Show more comments above.

Comment author: Zack_M_Davis 18 May 2012 06:25:42PM 2 points [-]

The only reason illiberal values persist is because those who hold them know (or are confident) that they're not personally going to be victims of them.

You might be right, but I'm less sure of this.

Someone with more historical or anthropological knowledge than I is welcome to correct me, but I'm given to understand that many of those whom we would consider victims of an oppressive social system, actually support the system. (E.g., while woman's suffrage seems obvious now, there were many female anti-suffragists at the time.) It's likely that such sentiments would be nullified by a "knew more, thought faster, &c." extrapolation, but I don't want to be too confident about the output of an algorithm that is as yet entirely hypothetical.

Furthermore, the veil of ignorance has its own problems: what does it mean for someone to have possibly been someone else? To illustrate the problem, consider an argument that might be made by (our standard counterexample) a hypothetical agent who wants only to maximize the number of paperclips in the universe:

The only reason non-paperclip-maximizing values persist is because those who hold them know (or are confident) that they're not personally going to be victims of them (because they already know that they happened to have been born as humans rather than paperclip-maximizers).

---which does not seem convincing. Of course, humans in oppressed groups and humans in privileged groups are inexpressibly more similar to each other than humans are to paperclip-maximizers, but I still think this thought experiment highlights a methodological issue that proponents of a veil of ignorance would do well to address.

Comment author: drnickbone 18 May 2012 06:40:23PM *  3 points [-]

Someone with more historical or anthropological knowledge than I is welcome to correct me, but I'm given to understand that many of those whom we would consider victims of an oppressive social system, actually support the system.

Isn't the main evidence that victims of oppressive social systems want to escape from them at every opportunity? There are reasons for refugees, and reasons that the flows are in consistent directions.

And if anti-suffragism had been truly popular, then having got the vote, women would have immediately voted to take it away again. Does this make sense?

Some other points:

  1. CEV is about human values, and human choices, rather than paper-clippers. I doubt we'd get a CEV across wildly-different utility functions in the first place.

  2. I'm happy to admit that CEV might not exist in the veil of ignorance case either, but it seems more likely to.

  3. I'm getting a few down-votes here. Is the general consensus here that this is too close to politics, and that is a taboo subject (as it is a mind-killer)? Or is the "veil of ignorance" idea not an important part of CEV?