Viliam_Bur comments on Muehlhauser-Wang Dialogue - Less Wrong

24 Post author: lukeprog 22 April 2012 10:40PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (284)

You are viewing a single comment's thread. Show more comments above.

Comment author: Viliam_Bur 26 April 2012 04:36:33PM *  0 points [-]

You seem to be assuming that that state of ignorance is something we can't do anything about

No, no, no. We probably can do something about it. I just assume that it will be more complicated than "make an estimate that complexity C will take time T, and then run a simulation for time S<T"; especially if we have no clue at all what the word 'complexity' means, despite pretending that it is a value we can somehow measure on a linear scale.

First step, we must somehow understand what "self-improvement" means and how to measure it. Even this idea can be confused, so we need to get a better understanding. Only then it makes sense to plan the second step. Or maybe I'm even confused about this all.

The only part I feel sure about is that we should first understand what self-improvement is, and only then we can try to measure it, and only then we can attempt to use some self-improvement treshold as a safety mechanism in an AI simulator.

This is a bit different from other situations, where you can first measure something, and then it is enough time to collect data and develop some understanding. Here a situation where you have something to measure (when there is a self-improving process), it is already an existential risk. If you have to make a map of minefield, you don't start by walking on the field and stomping heavily, even if in other situation an analogical procedure would be very good.

Comment author: TheOtherDave 26 April 2012 05:12:58PM 0 points [-]

First step, we must somehow understand what "self-improvement" means and how to measure it. Even this idea can be confused, so we need to get a better understanding.

Yes, absolutely agreed. That's the place to start. I'm suggesting that doing this would be valuable, because if done properly it might ultimately lead to a point where our understanding is quantified enough that we can make reliable claims about how long we expect a given amount of self-improvement to take for a given algorithm given certain resources.

This is a bit different from other situations, where you can first measure something

Sure, situations where you can safely first measure something are very different from the situations we're discussing.