lukstafi comments on Non-orthogonality implies uncontrollable superintelligence - Less Wrong

14 Post author: Stuart_Armstrong 30 April 2012 01:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukstafi 01 May 2012 12:14:31PM 0 points [-]
  • I don't think that every consequentialist view of ethics reduces to equating morality with maximizing an arbitrary but fixed utility function which leaves no action as morally neutral.

  • Under bounded resources, I think there is (and I think remains as the horizon expands with the capability of the system) plenty of leeway in the "Pareto front" of actions judged at a given time not to be "likely worse in the long term" than any other action considered.

  • The trajectory of a system depends on its boundary conditions even if the dynamic is in some sense "convergent", so "convergence" does not exclude control over the particular trajectory.