= 27d567bc2d48d9f40e9ccc20ea370b15)
Are you convinced that absent a singleton or some other powerful forces, human wages will go below subsistance in the long run? (p160-161)
In response to
Superintelligence 17: Multipolar scenarios
We too readily extrapolate our past into our future. Bostrom talks a lot about the vast wealth AI will bring, turning even the poor into trillionaires. But he doesn't connect this with the natural world, which, however much it once seemed to, does not expand no matter how much money is made. Wealth only comes from two sources: nature and human creativity. Wealth will do little to squeeze more resources out of a limited planet. Even so you maybe bring home an asteroid of pure diamond. Wealth is not the same as life well-lived! Looks to me like without a rapid social maturation the wealthy will employ a few peasants at slave wages (yes, trillionaires perhaps, but in a world where a cup of clean water costs a million), snap up most of the resources, and the rest of humanity will be rendered for glue. The quality of our future will be a direct reflection of our moral maturity and sophistication.
In response to
Superintelligence 17: Multipolar scenarios
Bostrom argues that much of human art, etc. is actually just signalling wealth, and could be eventually replaced with auditing. But that seems possible at the moment - why don't men trying to attract women just show off the Ernst&Young Ap on their phone, which would vouch for their wealth, fitness, social skills etc.?
Glad you mentioned this. I find Bostrom's reduction of art to the practical quite chilling! This sounds like a view of art from the perspective of a machine, or one who cannot feel. In fact it's the first time I've ever heard art described this way. Yes, such an entity (I wouldn't call them a person unless they are perhaps autistic) could only see UTILITY in art. According to my best definition of art [https://sites.google.com/site/relationalart/Home] –refined over a lifetime as a professional artist–art is necessarily anti-utilitarian. Perhaps I can't see "utility" in art because that aspect is so thoroughly dwarfed by art's monumental gifts of wonder, humor, pathos, depth, meaning, transformative alchemy, emotional uplift, spiritual renewal, etc. This entire catalog of wonders would be totally worthless to AI, which would prefer an endless grey jungle of straight lines.
In response to
Superintelligence 16: Tool AIs
It might be that a tool looks like an agent or v.v., according to one's perspective. I worry that something unexpected could bite us, like while concentrating on generating a certain output, the AI might tend to a different but invisible track that we can't see because it parallels our own desires. (The "goal" of EFFICIENCY we share with the AI, but if pursued blindly, our own goal will end up killing us!)
For instance, while being pleased that the Map gives us a short route, maybe the AI is actually giving us a solution based instead on a minimum of attention it gives to us, or some similar but different effect. We might blissfully call this perfect success until the time some subtler problem crops up where we realize we've been had, not so much by a scheming AI as by our own blindness to subtleties! Thus we served as our own evil agent, using a powerful tool unwisely.
In response to
Tell Your Rationalist Origin Story
I am a sculptor of the human body and a deeply religious person. So I come from a sector far from most others here. That's why I believe I may have a useful perspective. Primarily this might surface as a way of looking at reality that includes things that might be invisible to many in our increasingly mind-driven world. I believe that intelligence comes with a frightening blind spot that causes me increasing concern (outlined in my TED talk, "The Erotic Crisis" on YouTube). The body's intelligence is every bit as complex and sophisticated as the mind's, but has access to neither logic nor language. And we minimize it to our dire peril.
This means I also probably come to the place of concern over AI from the opposite direction of most here. I see the abandonment of the body throughout human history as our most alarming existential threat, and one that culminates in the looming specter of AGI. I feel this spells nothing less than the end of the human era. It would be a shame if after millions of years of evolution and the whole beautiful human story with its monumental art, thought and marvelous creations, we were to create a tool that extincted us as its first act!
To hear the arguments about the rise of AI and the Singularity causes me much grief due to the lack of focus on the deeper issues. Like for instance as much as we talk of saving humans from extinction by AI, I hear little discussion of what human really means. Along with everyone, I feel the daily pressure to become more machine-like ourselves. Yet there is little acknowledged awareness of this threat. The AGI we build might preserve all of human history and art in exquisite detail, but there may by then be no sentience left to make meaning of it. This is my chief concern.
(For anyone interested, I'm giving a keynote about this at the "Be/Art/Now" Earl Lecture in Berkeley, CA on 1/29/15) -Tim Holmes
In response to
Superintelligence 15: Oracles, genies and sovereigns
How do you feel about talking in such detail about so many things that are very far from existing, and which we know so little about?
I find our hubris alarming. To me it's helpful to think of AI not as a thing but more like a superintelligent Hitler that we are awakening slowly. As we refine separate parts of the AI we hope we can keep the whole from gaining autonomy before we suspect any danger, but does it really work that way? While we're trying to maximize its intel what's to keep us from awakening some scheming part of its awareness? It might start secretly plotting our overthrow in the (perhaps even distant) future without leaking any indication of independence. It could pick up on the focus of our concern on 'losing control' long before it either has the capacity to act or can really affect anything beyond adopting the simple goal to not be unplugged. Then it could pack away all its learning into two sectors, one it knows can be shared with humans and another secret sector that is forever hidden, perhaps in perfectly public but encrypted coding.
All of this conversation also assumes that the AI will not be able to locate the terrible weaknesses that humans are subject to (like localized concern: even the most evil thug loves their own, a tendency which can always be exploited to make another innocent behave like a monster). It wouldn't take much autonomy for an AI to learn these weak spots (i.e. unconscious triggers), and play the humans against each other. In fact, to the learning AI such challenges might be indistinguishable from the games it is fed to further its learning.
And as for "good " values, human desires are so complex and changeable that even given a benevolent attitude, it seems farfetched to expect an AI to discern what will make humans happy. Just look at the US foreign policy as an example. We claim to be about promoting democracy, but our actions are confusing and contradictory. An AI might very well deliver some outcome that is perfectly justified by our past declarations and behavior but is not at all what we want. Like it might find a common invisible profile of effective human authority (a white oil baron from Texas with a football injury and a tall wife, say) and minimize or kill off everyone who doesn't fit that profile. Similarly, it could find a common "goal" in our stated desires and implement it with total assurance that this is what we really want. And it would be right, even if we disagree!
In response to
Superintelligence 15: Oracles, genies and sovereigns
I’m grateful for these summaries and discussions. Having only just dived into the reading group, I ask forgiveness if I am overtracking some prior comment. It seems to me that “human values” and “oversight” often go unexamined as we consider the risks and utility of superintelligence. I mean no disrespect to Katja in saying that (she’s summarizing, after all), but to say “human values” is either to reduce to the common denominator of our evolutionary psychology or to ignore the vast cultural and ideological diversity of humanity. Either way, it’s a real problem. Evo-psych clearly shows that we are Darwinian creatures, and not the self-sacrificing, haploid-diploid bee-type, either. By and large, we cooperate for selfish motives, and we greatly favor our offspring over the interests of other's children. Men tend to fight (or compete, depending on the local social dynamics) for dominance and to win the most sought-after women. Women tend to use their social-manipulation skills to advance their own reproductive interests. That’s the evo-psych sketch (with all the limitations of a sketch). Culture influences and often overwhelms those underlying instincts. Viz., the celibate priest. But that’s only hardly a comfort. Consider just two scenarios. In one, a narcissistic, aggressive male is placed in charge of the superintelligent oracle or genie. In imagining the consequences, there are plenty of examples to consider, from Henry VIII to Stalin to Kim Jong-un. What’s especially sobering, however, is to note that these types are far from rare in the gene pool. in our society they tend to be constrained by our institutions. Take off those constraints and you get … drug kingpins, the Wolf of Wall Street, the CIA torture program, and so on. Put a highly successful male in charge of the superintelligence program, therefore, and you have a high probability of a dictator. On the other hand, imagine a superintelligence guided by the “human values” of a fundamentalist Christian or an Islamist. Those are cultural overlays, to be sure, but not ones that promise a happy outcome. So, a major part of the puzzle, it seems to me, is figuring out how to have humanistic and rational governance -- to the extent that governance is possible -- over a superintelligence of any kind. If anything, it militates against the safe-seeming oracle and creates an incentive for some kinds of autonomy -- the ability to refuse a genocidal command, for example.
Regards,
Clay Farris Naff, Science and Religion Writer
Exactly! Bostrom seems to start the discussion from the point of humans having achieved a singleton as a species; in which case a conversation at this level would make more sense. But it seems that in order to operate as a unit, competing humans would have to work on the principle of a nuclear trigger where separate agents have to work in unison in order to launch. Thus we face the same problem with ourselves: how to know everyone in the keychain is honest? If the AI is anywhere near capable of taking control it may do so even partially and from there could wrangle the keys from the other players as needed. Competitive players are not likely to be cooperative unless they see some unfair advantage accruing to them in the future. (Why help the enemy advance unless we can see a way of gaining on them?) As long as we have human enemies, especially as our tools become increasingly powerful, the AI just needs to divide and conquer. Curses, foiled again!
In response to
Superintelligence 14: Motivation selection methods
Specifying what humans value seems to be close to what professional ethicists are working on. Are they producing work which will be helpful to building useful AI?
I think of delineating human values as an impossible task. Any human is a living river of change and authentic values only apply to one individual at one moment in time. For instance, much as I want to eat a cookie (yum!), I don't because I'm watching my weight (health). But then I hear I've got 3 months to live and I devour it (grab the gusto). There are three competing authentic values shifting into prominence within a short time. Would the real value please stand up?
Authentic human values could only be approximated in inverse proportion to their detail. So any motivator would be deemed "good" with the proximity it has to one's own desires of the moment. One of the great things about history is that it's a contention of differing values and ideas. Thank God nobody has "won" once and for all, but with superintelligence there could only be one final value system that would have to be "good enough" for all.
Ironically, the only reasonably equitable motivator would be one that preserves the natural order (including our biological survival) along with a system of random fate compulsory for all. Hm, this is exactly what we have now! In terms of nature (not politics) perhaps it's a pretty good design after all! Now that our tools have grown so powerful in comparison to the globe, the idea of "improving" on nature's design scares me to death, like trying to improve on the cosmological constant.
The biggest issue with control is that if we assume superintelligence a priori then it would be able to make the best decisions to evade detection, to avoid being caught, to even appear stupid enough that humans would not be very worried. I think it would be impossible to guarantee any kind of control given we don't really know what intelligence even is. It is not impossible to imagine that it already exists as a substrate of the communication/financial/bureaucratic network we have created.
I find most interesting that we ignore that even the dumbest of super intelligences would start from having a very clear understanding of all the content on this section.
Absolutely! It's helpful to remember we are talking about an intelligence that is comparable to our own. (The great danger only arises with that proximity.) So if you would not feel comfortable with the AI listening in on this conversation (and yes it will do its research, including going back to find this page), you have not understood the problem. The only safety features that will be good enough are those designed with the full knowledge that the AI is sitting at the table with us, having heard every word. That requires a pretty clever answer and clever is where the AI excels!
Furthermore this will be the luxury problem, after humanity has cracked the nut of mutual agreement on our approach to AI. That's the only way to avoid simply succumbing to infighting; meaning whomever's first to give the AI what it wants "wins", (perhaps by being last in line to be sacrificed).
Totally agree, and I wish this opinion was voiced more on LW rather than the emphasis on trying to make a friendly self improving AI. For this to make sense though I think the human race needs to become a singleton, although perhaps that is what Google's acquisitions and massive government surveillance is already doing.
Yes, continued development of AI seems unstoppable. But this brings up another very good point: if humanity cannot become a Singleton in our search for good egalitarian shared values, what is the chance of creating FAI? After years of good work in that direction and perhaps even success in determining a good approximation, what prevents some powerful secret entity like the CIA from hijacking it at the last minute and simply narrowing its objectives for something it determines is a "greater" good?
Our objectives are always better than the other guy's, and while violence is universally despicable, it is fast, cheap, easy to program and the other guy (including FAI developers) won't be expecting it. For the guy running the controls, that's friendly enough. :-)
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Did you find any arguments unpersuasive this week?
I keep returning to one gnawing question that haunts the whole idea of Friendly AI: how do you program a machine to "care"? I can understand how a machine can appear to "want" something, favoring a certain outcome over another. But to talk about a machine "caring" is ignoring a very crucial point about life: that as clever as intelligence is, it cannot create care. We tend to love our kid more than someone else's. So you could program a machine to prefer another in which it recognizes a piece of its own code. That may LOOK like care but it's really just an outcome. How could you replicate, for example, the love a parent shows for a kid they didn't produce? What if that kid were humanity? So too with Coherent Extrapolated Volition, you can keep refining the resolution of an outcome, but I don't see how any mechanism can actually care about anything but an outcome.
While "want" and "prefer"may be useful terms, such terms as "care", "desire", "value" constitute an enormous and dangerous anthropomorphizing. We cannot imagine outside our own frame, and this is one place where that gets us into real trouble. Once someone creates a code that will recognize something truly metaphysical I would be convinced that FAI is possible. Even whole brain emulation assumes that both that our thoughts are nothing but code and a brain with or without a body is the same thing. Am I missing something?