= 27d567bc2d48d9f40e9ccc20ea370b15)
DanArmak comments on The Yudkowsky Ambition Scale - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (61)
2 has been done many times in human history (for some reasonably definition of what companies count as "previous Apples"). 9 has never been done. Why do you think 9 is no harder than 2, assuming it is possible?
9 has been done many times in human history too, for some reasonable definition of "create a better artificial optimizer."
Anyhow, to answer your question, I'm just guessing, based on calling "difficulty" something like marginal resources per rate of success. If you gave me 50 million dollars and said "make 2 happen," versus if you gave me 50 million dollars and said "make 9 happen," basically. Sure, someone is more likely to do 2 in the next few years than 9, ceteris paribus. But a lot more resources are on 2 (though there's a bit of a problem with this metric since 9 scales worse with resources than 2).
That's why 9 specifies "recursively self-improving", not "build a better optimizer", or even recursively improving optimizer. The computer counts for recursively improving, imho, it just needs some help, so it's not self-improving.
Presumably, if anyone ever solves 9, so did their mom.
Which is not in fact intended as a "your mom" joke, but I don't see any way around it being read that way.
If self-improving intelligence is somwehere on the hierarchy of "better optimizers," you just have to make better optimizers, and eventually you can make a self-improving optimizer. Easy peasy :P Note that this used the assumption the it's possible, and requires you to be charitable about interpreting "hierarchy of optimizers."