= 27d567bc2d48d9f40e9ccc20ea370b15)
Wei_Dai comments on The Friendly AI Game - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (170)
The Philosophical Insight Generator - Using a model of a volunteer's mind, generate short (<200 characters, say) strings that the model rates as highly insightful after read each string by itself, and print out the top 100000 such strings (after applying some semantic distance criteria or using the model to filter out duplicate insights) after running for a certain number of ticks.
Have the volunteer read these insights along with the rest of the FAI team in random order, discuss, update the model, then repeat as needed.
This isn't a Friendly Artificial General Intelligence 1) because it is not friendly; it does not act to maximize an expected utility based on human values, 2) because it's not artificial; you've uploaded an approximate human brain and asked/forced it to evaluate stimuli, and 3) because, operationally, it does not possess any general intelligence; the Generator is not able to perform any tasks but write insightful strings.
Are you instead proposing an incremental review process of asking the AI to tell us its ideas?
You're right, my entry doesn't really fit the rules of this game. It's more of a tangential brainstorm about how an FAI team can make use of a large amount of computation, in a relatively safe way, to make progress on FAI.
Do you imagine this to be doable in such a way that the model of the volunteer's mind is not a morally relevant conscious person (or at least not one who is suffering)? I could be convinced either way.
Are you thinking that the model might suffer psychologically because it knows it will cease to exist after each run is finished? I guess you could minimize that danger by picking someone who thinks they won't mind being put into that situation, and do a test run to verify this. Let me know if you have another concern in mind.
Mmm, it's not so much that think the mind-model is especially likely to suffer; I just want to make sure that possibility is being considered. The test run sounds like a good idea. Or you could inspect a random sampling and somehow see how they're doing. Perhaps we need a tool along the lines of the nonperson predicate -- something like an is-this-person-observer-moment-suffering function.