= 27d567bc2d48d9f40e9ccc20ea370b15)
Stuart_Armstrong comments on How to measure optimisation power - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (30)
"outcome achieved", "did you hit an outcome", "optimization processes produce surprises", "relative improbability of 'equally good or better' outcomes" - he's talking about the outcome produced (and then using the preference orderings to measure optimisation power given that that outcome was produced).
The time taken is not explicitly modeled, but is indirectly: exhaustive search only wins if the agent really has all the time in the world to implement its plans. An AI due to get smashed in a year if it doesn't produce anything will have an optimisation of zero if it uses exhaustive search.
"An optimisation of zero" ?!? <fx: shakes head, despairingly>
Are you objecting to the phrasing or to the point?
That was the terminology - though it isn't just the terminology that is busted here.
Frankly, I find it hard to believe that you seem to be taking this idea seriously.
I haven't decided whether the idea is good or bad yet - I haven't yet evaluated it properly.
But as far as I can tell, your objection to it is incorrect. A naive search program would have very low optimisation power by Eliezer's criteria - is there a flaw in my argument?
Essentially I agree that that particular objection is largely ineffectual. It is possible to build resource constraints into the environment if you like - though usually resource constraints are at least partly to do with the agent.
Resource constraints need to be specified somewhere. Otherwise exhaustive search (10 mins) gets one score and exhaustive search (10 years) gets another score - and the metric isn't well defined.
If you see the optimisation score as being attached to a particular system (agent+code+hardware+power available), then there isn't a problem. It's only if you want to talk about the optimisation power of an algorithm in a platonic sense, that the definition fails.
Upvoted because admitting to error is rare and admirable, even on Less Wrong :-)