Eliezer_Yudkowsky comments on The Sword of Good - Less Wrong

85 Post author: Eliezer_Yudkowsky 03 September 2009 12:53AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (292)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 05 September 2009 07:10:38PM -1 points [-]

Cracking the protein folding problem, building nanotechnology, and reviving a cryonics patient at the highest possible fidelity. Redesigning the spaghetti code of the brain so as to permit it to live a flourishing and growing life rather than e.g. overloading with old memories at age 200.

I suppose you make a remarkable illustration of how people with no cosmic ambitions and brainwashed by the self-help industry, don't even have any goals in life that require direct brain editing, and aren't much willing to imagine them because it implies that their own brains are (gasp!) inadequate.

Comment author: pjeby 09 September 2009 03:24:25AM 2 points [-]

I suppose you make a remarkable illustration of how people with no cosmic ambitions and brainwashed by the self-help industry, don't even have any goals in life that require direct brain editing, and aren't much willing to imagine them because it implies that their own brains are (gasp!) inadequate.

Wow, somebody's cranky today. (I could equally note that you're an illustration of what happens when people try to build a technical solution to a human problem... while largely ignoring the human side of the problem.)

Solving cooler technical problems or having more brain horsepower sure would be nice. But as I already know from personal experience, just being smarter than other people doesn't help, if it just means you execute your biases and misconceptions with greater speed and an increased illusion of certainty.

Hence, I consider the sort of self-modification that removes biases, misconceptions, and motivated reasoning to be both vastly more important and incredibly more urgent than the sort that would let me think faster, while retaining the exact same blindspots.

But if you insist on hacking brain hardware directly or in emulation, please do start with debugging support: the ability to see in real-time what belief structures are being engaged in reaching a decision or conclusion, with nice tracing readouts of all their backing assumptions. That would be really, really useful, even if you never made any modifications outside the ones that would take place by merely observing the debugger output.

Comment author: Eliezer_Yudkowsky 09 September 2009 04:09:59AM 6 points [-]

you're an illustration of what happens when people try to build a technical solution to a human problem

If there were a motivator captioned "TECHNICAL SOLUTIONS TO HUMAN PROBLEMS", I would be honored to have my picture appear on it, so thank you very much.

Comment author: pjeby 09 September 2009 04:36:45AM 1 point [-]

If there were a motivator captioned "TECHNICAL SOLUTIONS TO HUMAN PROBLEMS", I would be honored to have my picture appear on it, so thank you very much.

You left out the "ignoring the human part of the problem" part.

The best technical solutions to human problems are the ones that leverage and use the natural behaviors of humans, rather than trying to replace those behaviors with a perfect technical process or system, or trying to force the humans to conform to expectations.

(I'd draw an analogy with Nelson's Xanadu vs. the web-as-we-know-it, but that could be mistaken for a pure Worse Is Better argument, and I certainly don't want any motivated superintelligences being built on a worse-is-better basis.)