= 27d567bc2d48d9f40e9ccc20ea370b15)
You may be interested in Yvain's Blue-Minimizing Robot sequence, which addresses these concerns. To read it, go to http://lesswrong.com/user/Yvain/submitted/?count=25&after=t3_8kn, and read the posts from "The Blue-Minimizing Robot" to "Tendencies in reflective equilibrium".
Thanks! I've read some of the stuff by Yvain but not these posts.
In response to
Open thread, August 26 - September 1, 2013
Is Friendly AI or more specifically CEV predicated on Eliminative Materialism being false? To what extent is FAI predicated on folk psychological theories of mental content turning out to accurately reflect human neurobiology?
From the article:
Modern versions of eliminative materialism claim that our common-sense understanding of psychological states and processes is deeply mistaken and that some or all of our ordinary notions of mental states will have no home, at any level of analysis, in a sophisticated and accurate account of the mind. In other words, it is the view that certain common-sense mental states, such as beliefs and desires, do not exist.
Let me see if I can unpack this idea a bit more.
CEV is based on the idea that there is an algorithm that can look at the state of my brain, filter out various kinds of noise, and extrapolate what sort of desires and values I'd want to have if I lived in a kinder more benevolent society, wasn't subject to nearly as many serious cognitive biases, etc.
The problem I'm seeing is that the origin and meaning of terms like 'desire' and 'value' are in prescientific culture - folk psychology. they were created by people in absolute ignorance about how brains work, and it seems increasingly plausible that these concepts will be totally inadequate for any accurate scientific explanation of how brains produce human behaviour.
It seems to be common sense that desires and values and the like are indispensable theoretical posits simply because they are all we have. Our brains' extremely limited metacogntive abilities prevent us from modelling ourselves as brains, so our brains invent a kind of mythology to explain their behaviour, which is pure confabulation.
If these ideas are right, by asking CEV to consider folk psychological ideas like desires and values, we would be committing it to the existence of things that just aren't really present in our brain states in any objective sense.
In the worst case, running CEV might be somewhat analogous to asking the AI to use Aristotelian physics to build a better airplane.
What we perceive as the fragility and complexity of human based values might not map onto brain states at all - 'values' as we wish to conceive of them may not exist outside of narrative fiction and philosophy papers.
My recent thinking on these topics has been heavily influenced by the writings of Scott Bakker , Daniel Hutto and Peter Watts' Blindsight
I hope I'm wrong about this stuff, but I don't have the training to fully analyze and debunk these ideas by myself - if it's even possible. I hope LW and MIRI have some insights about these issues, because I am seriously troubled by the apparent implications for the future of humanity.
In response to
Open thread, August 26 - September 1, 2013
Is Friendly AI or more specifically CEV predicated on Eliminative Materialism being false? To what extent is FAI predicated on folk psychological theories of mental content turning out to accurately reflect human neurobiology?
From the article:
Modern versions of eliminative materialism claim that our common-sense understanding of psychological states and processes is deeply mistaken and that some or all of our ordinary notions of mental states will have no home, at any level of analysis, in a sophisticated and accurate account of the mind. In other words, it is the view that certain common-sense mental states, such as beliefs and desires, do not exist.
In response to
Rationality Quotes August 2013
Faced with the task of extracting useful future out of our personal pasts, we organisms try to get something for free (or at least at bargain price): to find the laws of the world -- and if there aren't any, to find approximate laws of the world -- anything at all that will give us an edge. From some perspectives it appears utterly remarkable that we organisms get any purchase on nature at all. Is there any deep reason why nature should tip its hand, or reveal its regularities to casual inspection? Any useful future-producer is apt to be something of a trick -- a makeshift system that happens to work, more often than not, a lucky hit on a regularity in the world that can be tracked. Any such lucky anticipators Mother Nature stumbles over are bound to be prized, of course, if they improve an organism's edge.
--Daniel Dennet Consciousness Explained
In response to
Rationality Quotes August 2013
Karl Popper used to begin his lecture course on the philosophy of science by asking the students simply to 'observe'. Then he would wait in silence for one of them to ask what they were supposed to observe. [...] So he would explain to them that scientific observation is impossible without pre-existing knowledge about what to look at, what to look for, how to look, and how to interpret what one sees. And he would explain that, therefore, theory has to come first. It has to be conjectured, not derived.
David Deutsch, The Beginning of Infinity
In response to
Who are some of the best writers in history?
Cervantes, the author of the first modern novel, widely regarded as one of the best books ever written
It's worth emphasizing just how modern and readable this book is, especially considering it is contemporaneous with Shakespeare. If you get a modern English translation, you will be delighted. Cervantes really invented modern literature.
In response to
More "Stupid" Questions
How do you pronounce 3^^^3?
Perhaps you would like to post this in the latest (July 2013) media thread? When I wonder what book to read next, I might look through the media threads, but not the open threads...
Okay, I did it. http://lesswrong.com/lw/hvc/july_2013_media_thread/9glb
In response to
July 2013 Media Thread
Fiction Books Thread
I don't read very much fiction, but recently I've read
The Eternal Flame by Greg Egan - book two of his Orthogonal series, where he imagines life in a universe with different spacetime symmetries, where the velocity of light is a function of its wavelength. In this instalment, alien scientists on a generation ship try to discover the secrets of matter, and of their own biology, which will allow them to return home. There is a lot of focus on the scientific method and the character of physical law, and the treatment of the (made up) physics is much, much more rigorous and principled than earlier physics-centric Egan books like Schilds Ladder, Diaspora, or the dreadful Distress
Neuropath by Scott Bakker - a disturbing psychological thriller that explores a radical reductionistic view of the mind and consciousness. If you still think that consciousness is a some sort of unique, special phenomenon, an inevitable byproduct of intelligence, than this book may be for you.
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Today I taught a bunch of 5th grade kids how to convert decimals into fractions and vice versa.