You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

lukstafi comments on Non-orthogonality implies uncontrollable superintelligence - Less Wrong Discussion

14 Post author: Stuart_Armstrong 30 April 2012 01:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukstafi 01 May 2012 12:14:31PM 0 points [-]
  • I don't think that every consequentialist view of ethics reduces to equating morality with maximizing an arbitrary but fixed utility function which leaves no action as morally neutral.

  • Under bounded resources, I think there is (and I think remains as the horizon expands with the capability of the system) plenty of leeway in the "Pareto front" of actions judged at a given time not to be "likely worse in the long term" than any other action considered.

  • The trajectory of a system depends on its boundary conditions even if the dynamic is in some sense "convergent", so "convergence" does not exclude control over the particular trajectory.