You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Not all theories of consciousness are created equal: a reply to Robert Lawrence Kuhn's recent article in Skeptic Magazine [Link]

3 oge 04 September 2016 08:35PM

I found this article on the Brain Preservation Foundation's blog that covers a lot of common theories of consciousness and shows how they kinna miss the point when it comes to determining if certain folks should or should not upload our brains if given the opportunity.

Hence I see no reason to agree with Kuhn’s pessimistic conclusions about uploading even assuming his eccentric taxonomy of theories of consciousness is correct.  What I want to focus on in the reminder of this blog is challenging the assumption that the best approach to consciousness is tabulating lists of possible theories of consciousness and assuming they each deserve equal consideration (much like the recent trend in covering politics to give equal time to each position regardless of any empirical relevant considerations). Many of the theories of consciousness on Kuhn’s list, while reasonable in the past, are now known to be false based on our best current understanding of neuroscience and physics (specifically, I am referring to theories that require mental causation or mental substances). Among the remaining theories, some of them are much more plausible than others.

http://www.brainpreservation.org/not-all-theories-of-consciousness-are-created-equal-a-reply-to-robert-lawrence-kuhns-recent-article-in-skeptic-magazine/

HPMOR and the Power of Consciousness

-3 Algernoq 25 November 2015 07:00AM

Throughout HPMOR, the author has included many fascinating details about how the real world works, and how to gain power. The Mirror of CEV seems like a lesson in what a true Friendly AI could look like and do.

I've got a weirder theory. (Roll for sanity...)

The entire story is plausible-deniability cover for explaining how to get the Law of Intention to work reliably.

(All quoted text is from HPMOR.)

This Mirror reflects itself perfectly and therefore its existence is absolutely stable. 

"This Mirror" is the Mind, or consciousness. The only thing a Mind can be sure of is that it is a Mind.

The Mirror's most characteristic power is to create alternate realms of existence, though these realms are only as large in size as what can be seen within the Mirror

A Mind's most characteristic power is to create alternate realms of existence, though these realms are only as large in size as what can be seen within the Mind.

Showing any person who steps before it an illusion of a world in which one of their desires has been fulfilled.

The final property upon which most tales agree, is that whatever the unknown means of commanding the Mirror - of that Key there are no plausible accounts - the Mirror's instructions cannot be shaped to react to individual people...the legends are unclear on what rules can be given, but I think it must have something to do with the Mirror's original intended use - it must have something to do with the deep desires and wishes arising from within the person.

More specifically, the Mirror shows a universe that obeys a consistent set of physical laws. From the set of all wish-fulfillment fantasies, it shows a universe that could actually plausibly exist.

It is known that people and other objects can be stored therein

Actors store other minds within their own Mind. Engineers store physical items within their Mind. The Mirror is a Mind.

the Mirror alone of all magics possesses a true moral orientation

The Mind alone of all the stuff that exists possesses a true moral orientation.

If that device had been completed, the story claimed, it would have become an absolutely stable existence that could withstand the channeling of unlimited magic in order to grant wishes. And also - this was said to be the vastly harder task - the device would somehow avert the inevitable catastrophes any sane person would expect to follow from that premise. 

An ideal Mind would grant wishes without creating catastrophes. Unfortunately, we're not quite ideal minds, even though we're pretty good.

Professor Quirrell made to walk away from the Mirrror, and seemed to halt just before reaching the point where the Mirror would no longer have reflected him, if it had been reflecting him.

My self-image can only go where it is reflected in my Mind. In other words, I can't imagine what it would be like to be a philosophical zombie.

Most powers of the Mirror are double-sided, according to legend. So you could banish what is on the other side of the Mirror instead. Send yourself, instead of me, into that frozen instant. If you wanted to, that is.

Let's interpret this scene: We've got a Mind/consciousness (the Mirror), we've got a self-image (Riddle) as well as the same spirit in a different self-image (Harry), and we've got a specific Extrapolated Volition instance in the mind (Dumbledore shown in the Mirror). This Extrapolated Volition instance is a consistent universe that could actually exist.

It sounds like the Process of the Timeless trap causes some Timeless Observer to choose one side of the Mirror as the real Universe, trapping the universe on the other side of the mirror in a frozen instant from the Timeless Observer's perspective.

The implication: the Mind has the power to choose which Universes it experiences from the set of all possible Universes extending from the current point.

All right, screw this nineteenth-century garbage. Reality wasn't atoms, it wasn't a set of tiny billiard balls bopping around. That was just another lie. The notion of atoms as little dots was just another convenient hallucination that people clung to because they didn't want to confront the inhumanly alien shape of the underlying reality. No wonder, then, that his attempts to Transfigure based on that hadn't worked. If he wanted power, he had to abandon his humanity, and force his thoughts to conform to the true math of quantum mechanics.

There were no particles, there were just clouds of amplitude in a multiparticle configuration space and what his brain fondly imagined to be an eraser was nothing except a gigantic factor in a wavefunction that happened to factorize, it didn't have a separate existence any more than there was a particular solid factor of 3 hidden inside the number 6, if his wand was capable of altering factors in an approximately factorizable wavefunction then it should damn well be able to alter the slightly smaller factor that Harry's brain visualized as a patch of material on the eraser -

Had to see the wand as enforcing a relation between separate past and future realities, instead of changing anything over time - but I did it, Hermione, I saw past the illusion of objects, and I bet there's not a single other wizard in the world who could have. 

This seems like another giant hint about magical powers.

"I had wondered if perhaps the Words of False Comprehension might be understandable to a student of Muggle science. Apparently not."

The author is disappointed that we don't get his hints. 

If the conscious mind was in reality a wish-granting machine, then how could I test this without going insane?

The Mirror of Perfect Reflection has power over what is reflected within it, and that power is said to be unchallengeable. But since the True Cloak of Invisibility produces a perfect absence of image, it should evade this principle rather than challenging it.

A method to test this seems to be to become aware of one's own ego-image (stand in front of the Mirror), vividly imagine a different ego-image without identifying with it (bring in a different personality containing the same Self under an Invisibility Cloak), suddenly switch ego-identification to the other personality (swap the Invisibility Cloak in less than a second), and then become distracted so the ego-switch becomes permanent (Dumbledore traps himself in the Mirror).

I can't think of a way to test this without sanity damage. Comments?

State-Space of Background Assumptions

10 algekalipso 29 July 2015 12:22AM

[Update]: I received 720+ responses to the survey. Thanks everyone who helped! I have also concluded the statistical analysis (factor analysis, mediation analysis, clustering and prediction). I have not, however, done the writeup. This may take some time since I just started working. It will be done :) I just wanted to let people know this is the current stage. 

 

Hello everyone!

My name is Andrés Gómez Emilsson, and I'm the former president of the Stanford Transhumanist Association. I just graduated from Stanford with a masters in computational psychology (my undergraduate degree was in Symbolic Systems, the major with the highest LessWronger density at Stanford and possibly of all universities).

I have a request for the LessWrong community: I would like as many of you as possible to fill out this questionnaire I created to help us understand what causes the diversity of values in transhumanism. The purpose of this questionnaire is twofold:

 

  1. Characterize the state-space of background assumptions about consciousness
  2. Evaluate the influence of beliefs about consciousness, as well as personality and activities, in the acquisition of memetic affiliations

 

The first part is not specific to transhumanism, and it will be useful whether or not the second is fruitful. What do I mean by the state-space of background assumptions? The best way to get a sense of what this would look like is to see the results of a previous study I conducted: State-space of drug effects. There I asked participants to "rate the effects of a drug they have taken" by selecting the degree to which certain phrases describe the effects of the drug. I then conducted factor analysis on the dataset and extracted 6 meaningful factors accounting for more than 50% of the variance. Finally, I mapped the centroid of the responses of each drug in the state-space defined, so that people could visually compare the relative position of all of the substances in a normalized 6-dimensional space. 

I don't know what the state-space of background assumptions about consciousness looks like, but hopefully the analysis of the responses to this survey will reveal them.

The second part is specific to transhumanism, and I think it should concerns us all. To the extent that we are participating in the historical debate about how the future of humanity should be, it is important for us to know what makes people prefer certain views over others. To give you a fictitious example of a possible effect I might discover: It may turn out that being very extraverted predisposes you to be uninterested in Artificial Intelligence and its implications. If this is the case, we could pin-point possible sources of bias in certain communities and ideological movements, thereby increasing the chances of making more rational decisions.

The survey is scheduled to be closed in 2 days, on July 30th 2015. That said, I am willing to extend the deadline until August 2nd if I see that the number of LessWrongers answering the questionnaire is not slowing down by the 30th. [July 31st edit: I extend the deadline until midnight (California time) of August 2nd of 2015.]

Thank you all!

Andrés :)

 


Here are some links about my work in case you are interested and want to know more:

Survey link

Qualia Computing

Psychophysics for Psychedelic Research 

Psychedelic Perception of Visual Textures

The Psychedelic Future of Consciousness

A Workable Solution to the Problem of Other Minds

​My recent thoughts on consciousness

0 AlexLundborg 24 June 2015 12:37AM

I have lately come to seriously consider the view that the everyday notion of consciousness doesn’t refer to anything that exists out there in the world but is rather a confused (but useful) projection made by purely physical minds onto their depiction of themselves in the world. The main influences on my thinking are Dan Dennett, (I assume most of you are familiar with him)  and to a lesser extent Yudkowsky (1) and Tomasik (2). To use Dennett’s line of thought: we say that honey is sweet, that metal is solid or that a falling tree makes a sound, but the character tag of sweetness and sounds is not in the world but in the brains internal model of it. Sweetness in not an inherent property of the glucose molecule, instead, we are wired by evolution to perceive it as sweet to reward us for calorie intake in our ancestral environment, and there is neither any need for non-physical sweetness-juice in the brain – no, it's coded (3). We can talk about sweetness and sound as if being out there in the world but in reality it is a useful fiction of sorts that we are "projecting" out into the world. The default model of our surroundings and ourselves we use in our daily lives (the manifest image, or ’umwelt’) is puzzling to reconcile with the scientific perspective of gluons and quarks. We can use this insight to look critically on how we perceive a very familiar part of the world: ourselves. It might be that we are projecting useful fictions onto our model of ourselves as well. Our normal perception of consciousness is perhaps like the sweetness of honey, something we think exist in the world, when it is in fact a judgement about the world made (unconsciously) by the mind.

What we are pointing at with the judgement “I am conscious” is perhaps the competence that we have to access states about the world, form expectations about those states and judge their value to us, coded in by evolution. That is, under this view, equivalent with saying that suger is made of glucose molecules, not sweetness-magic. In everyday language we can talk about suger as sweet and consciousness as “something-to-be-like-ness“ or “having qualia”, which is useful and probably necessary for us to function, but that is a somewhat misleading projection made by our ​​world-accessing and assessing consciousness that really exists in the world. That notion of consciousness is not subject to the Hard Problem, it may not be an easy problem to figure out how consciousness works, but it does not appear impossible to explain it scientifically as pure matter like anything else in the natural world, at least in theory. I’m pretty confident that we will solve consciousness, if we by consciousness mean the competence of a biological system to access states about the world, make judgements and form expectations. That is however not what most people mean when they say consciousness. Just like ”real” magic refers to the magic that isn’t real and the magic that is real, that can be performed in the world, is not “real magic”, “real” consciousness turns out to be a useful, but misleading assessment (4). We should perhaps keep the word consciousness but adjust what we mean when we use it, for diplomacy.

Having said that, I still find myself baffled by the idea that I might not be conscious in the way I’ve found completely obvious before. Consciousness seems so mysterious and unanswerable, so it’s not surprising then that the explanation provided by physicalists like Dennett isn’t the most satisfying. Despite that, I think it’s the best explanation I've found so far, so I’m trying to cope with it the best I can. One of the problems I’ve had with the idea is how it has required me to rethink my views on ethics. I sympathize with moral realism, the view that there exist moral facts, by pointing to the strong intuition that suffering seems universally bad, and well-being seems universally good. Nobody wants to suffer agonizing pain, everyone wants beatific eudaimonia, and it doesn't feel like an arbitrary choice to care about the realization of these preferences in all sentience to a high degree, instead of any other possible goal like paperclip maximization. It appeared to me to be an unescapable fact about the universe that agonizing pain really is bad (ought to be prevented), that intelligent bliss really is good (ought to be pursued), like a label to distinguish wavelength of light in the brain really is red, and that you can build up moral values from there. I have a strong gut feeling that the well-being of sentience matters, and the more capacity a creature has of receiving pain and pleasure the more weight it is given, say a gradience from beetles to posthumans that could perhaps be understood by further inquiry of the brain (5). However, if it turns out that pain and pleasure isn’t more than convincing judgements by a biological computer network in my head, no different in kind to any other computation or judgement, the sense of seriousness and urgency of suffering appears to fade away. Recently, I’ve loosened up a bit to accept a weaker grounding for morality: I still think that my own well-being matter, and I would be inconsistent if I didn’t think the same about other collections of atoms that appears functionally similar to ’me’, who also claim, or appear, to care about their well-being. I can’t answer why I should care about my own well-being though, I just have to. Speaking of 'me': personal identity also looks very different (nonexistent?) under physicalism, than in the everyday manifest image (6).

Another difficulty I confront is why e.g. colors and sounds looks and sounds the way they do or why they have any quality at all, under this explanation. Where do they come from if they’re only labels my brain uses to distinguish inputs from the senses? Where does the yellowness of yellow come? Maybe it’s not a sensible question, but only the murmuring of a confused primate. Then again, where does anything come from? If we can learn to shut up our bafflement about consciousness and sensibly reduce it down to physics – fair enough, but where does physics come from? That mystery remains, and that will possibly always be out of reach, at least probably before advanced superintelligent philosophers. For now, understanding how a physical computational system represents the world, creates judgements and expectations from perception presents enough of a challenge. It seems to be a good starting point to explore anyway (7).


I did not really put forth any particularly new ideas here, this is just some of my thoughts and repetitions of what I have read and heard others say, so I'm not sure if this post adds any value. My hope is that someone will at least find some of my references useful, and that it can provide a starting point for discussion. Take into account that this is my first post here, I am very grateful to receive input and criticism! :-)

  1. Check out Eliezer's hilarious tear down of philosophical zombies if you haven't already
  2. http://reducing-suffering.org/hard-problem-consciousness/
  3. [Video] TED talk by Dan Dennett http://www.ted.com/talks/dan_dennett_cute_sexy_sweet_funny
  4. http://ase.tufts.edu/cogstud/dennett/papers/explainingmagic.pdf
  5. Reading “The Moral Landscape” by Sam Harris increased my confidence in moral realism. Whether moral realism is true of false can obviously have implications for approaches to the value learning problem in AI alignment, and for the factual accuracy of the orthogonality thesis
  6. http://www.lehigh.edu/~mhb0/Dennett-WhereAmI.pdf
  7. For anyone interested in getting a grasp of this scientific challenge I strongly recommend the book “A User’s Guide to Thought and Meaning” by Ray Jackendoff.



Edit: made some minor changes and corrections. Edit 2: made additional changes in the first paragraph for increased readability.

 


Does consciousness persist?

-10 G0W51 14 February 2015 03:52PM

Edit: the below paragraphs are wrong. See the comments for an explanation.

 

Some people believe that the consciousness currently in one's body is the "same" consciousness as the one that was in one's body in the past and the one that will be in it in the future, but a "different" consciousness from those in other bodies. In this post I dissolve the question.

The question is meaningless because the answer doesn't correspond to any physical state in the universe and in no way influences or is influenced by sensory experiences. If one's consciousness suddenly became a totally different one, we know of no quantum particles that would change. Furthermore, swapping consciousnesses would make no changes to what is perceived. E.g. if one agent perceives p and time t and p' at the next time t+1, and another agent perceives q at time t and q' at time t+1, then if their consciousnesses are "swapped," the percepts would still be identical: p and q will be perceived at time t, and p' and q' will be perceived at t+1. One could argue that the percepts did change because the consciousness-swapping changed what a particular consciousness at time t will perceive at t+1, but that presupposes that a future consciousness will be in some meaningful way the "same" consciousness as the current one! Thus, the statement that two consciousnesses are the same consciousness is meaningless.

Can you find any flaws in my reasoning?

 

 

A Basic Problem of Ethics: Panpsychism?

-4 capybaralet 27 January 2015 06:27AM

Panpsychism seems like a plausible theory of consciousness.  It raises extreme challenges for establishing reasonable ethical criteria.

It seems to suggest that our ethics is very subjective: the "expanding circle" of Peter Singer would eventually (ideally) stretch to encompass all matter.  But how are we to communicate with, e.g. rocks?  Our ability to communicate with one another and our presumed ability to detect falsehood and empathize in a meaningful way allow us to ignore this challenge wrt other people.

One way to argue that this is not such a problem is to suggest that humans are simply very limited in our capacity as ethical beings, and that we are fundamentally limited in our perceptions of ethical truth to only be able to draw conclusions with any meaningful degree of certainty about other humans or animals (or maybe even life-forms, if you are optimistic).  

But this is not very satisfying if we consider transhumanism.  Are we to rely on AI to extrapolate our intuitions to the rest of matter?  How do we know that our intuitions are correct (or do we even care?  I do, personally...)?  How can we tell if an AI is correctly extrapolating?




A few thoughts on a Friendly AGI (safe vs friendly, other minds problem, ETs and more)

3 the-citizen 19 October 2014 07:59AM

Friendly AI is an idea that I find to be an admirable goal. While I'm not yet sure an intelligence explosion is likely, or whether FAI is possible, I've found myself often thinking about it, and I'd like for my first post to share a few those thoughts on FAI with you.

Safe AGI vs Friendly AGI
-Let's assume an Intelligence Explosion is possible for now, and that an AGI with the ability to improve itself somehow is enough to achieve it.
-Let's define a safe AGI as an above-human general AI that does not threaten humanity or terran life (eg. FAI, Tool AGI, possibly Oracle AGI)
-Let's define a Friendly AGI as one that *ensures* the continuation of humanity and terran life.
-Let's say an unsafe AGI is all other AGIs.
-Safe AGIs must supress unsafe AGIs in order to be considered Friendly. Here's why:

-If we can build a safe AGI, we probably have the technology to build an unsafe AGI too.
-An unsafe AGI is likely to be built at that point because:
-It's very difficult to conceive of a way that humans alone will be able to permanently stop all humans from developing an unsafe AGI once the steps are known**
-Some people will find the safe AGI's goals unnacceptable
-Some people will rationalise or simply mistake that their AGI design is safe when it is not
-Some people will not care if their AGI design is safe, because they do not care about other people, or because they hold some extreme beliefs
-Most imaginable unsafe AGIs would outcompete safe AGIs, because they would not neccessarily be "hamstrung" by complex goals such as protecting us meatbags from destruction. Tool or Oracle AGIs would obviously not stand a chance due to their restrictions.
-Therefore, If a safe AGI does not prevent unsafe AGIs from coming into existence, humanity will very likely be destroyed.

-The AGI most likely to prevent unsafe AGIs from being created is one that actively predicted their development and terminates that development before or on completion.
-So to summarise

-An AGI is very likely only a Friendly AI if it actively supresses unsafe AGI.
-Oracle and Tool AGIs are not Friendly AIs, they are just safe AIs, because they don't suppress anything.
-Oracle and Tool AGIs are a bad plan for AI if we want to prevent the destruction of humanity, because hostile AGIs will surely follow.

(**On reflection I cannot be certain of this specific point, but I assume it would take a fairly restrictive regime for this to be wrong. Further comments on this very welcome.)

Other minds problem - Why should be philosophically careful when attempting to theorise about FAI

I read quite a few comments in AI discussions that I'd probably characterise as "the best utility function for a FAI is one that values all consciousness". I'm quite concerned that this persists as a deeply held and largely unchallenged assumption amongst some FAI supporters. I think in general I find consciousness to be an extremely contentious, vague and inconsistently defined concept, but here I want to talk about some specific philosophical failures.

My first concern is that while many AI theorists like to say that consciousness is a physical phenomenon, which seems to imply Monist/Physicalist views, they at the same time don't seem to understand that consciousness is a Dualist concept that is coherent only in a Dualist framework. A Dualist believes there is a thing called a "subject" (very crudely this equates with the mind) and then things called objects (the outside "empirical" world interpreted by that mind). Most of this reasoning begins with Descartes' cogito ergo sum or similar starting points ( https://en.wikipedia.org/wiki/Cartesian_dualism ). Subjective experience, qualia and consciousness make sense if you accept that framework. But if you're a Monist, this arbitrary distinction between a subject and object is generally something you don't accept. In the case of a Physicalist, there's just matter doing stuff. A proper Physicalist doesn't believe in "consciousness" or "subjective experience", there's just brains and the physical human behaviours that occur as a result. Your life exists from a certain point of view, I hear you say? The Physicalist replies, "well a bunch of matter arranged to process information would say and think that, wouldn't it?".

I don't really want to get into whether Dualism or Monism is correct/true, but I want to point out even if you try to avoid this by deciding Dualism is right and consciousness is a thing, there's yet another more dangerous problem. The core of the problem is that logically or empirically establishing the existence of minds, other than your own is extremely difficult (impossible according to many). They could just be physical things walking around acting similar to you, but by virtue of something purely mechanical - without actual minds. In philosophy this is called the "other minds problem" ( https://en.wikipedia.org/wiki/Problem_of_other_minds or http://plato.stanford.edu/entries/other-minds/). I recommend a proper read of it if the idea seems crazy to you. It's a problem that's been around for centuries, and yet to-date we don't really have any convincing solution (there are some attempts but they are highly contentious and IMHO also highly problematic). I won't get into it more than that for now, suffice to say that not many people accept that there is a logical/empirical solution to this problem.

Now extrapolate that to an AGI, and the design of its "safe" utility functions. If your AGI is designed as a Dualist (which is neccessary if you wish to encorporate "consciousness", "experience" or the like into your design), then you build-in a huge risk that the AGI will decide that other minds are unprovable or do not exist. In this case your friendly utility function designed to protect "conscious beings" fails and the AGI wipes out humanity because it poses a non-zero threat to the only consciousness it can confirm - its own. For this reason I feel "consciousness", "awareness", "experience" should be left out of FAI utility functions and designs, regardless of the truth of Monism/Dualism, in favour of more straight-forward definitions of organisms, intelligence, observable emotions and intentions. (I personally favour conceptualising any AGI as a sort of extension of biological humanity, but that's a discussion for another day) My greatest concern is there is such strong cultural attachment to the concept of consciousness that researchers will be unwilling to properly question the concept at all.

What if we're not alone?

It seems a little unusual to throw alien life into the mix at this point, but I think its justified because an intelligence explosion really puts an interstellar existence well within our civilisation's grasp. Because it seems that an intelligence explosion implies a very high rate of change, it makes sense to start considering even the long term implication early, particularly if the consequences are very serious, as I believe they may be in this realm of things.

Let's say we successfully achieved a FAI. In order to fufill its mission of protecting humanity and the biosphere, it begins expanding, colonising and terraforming other planets for potential habitation by Earth originating life. I would expect this expansion wouldn't really have a limit, because the more numourous the colonies, the less likely it is we could be wiped out by some interstellar disaster.

Of course, we can't really rule out the possibility that we're not alone in the universe, or even the galaxy. If we make it as far as AGI, then its possible another alien civilisation might reach a very high level of technological advancement too. Or there might be many. If our FAI is friendly to us but basically treats them as paperclip fodder, then potentially that's a big problem. Why? Well:

-Firstly, while a species' first loyalty is to itself, we should consider that it might be morally unsdesirable to wipe out alien civilisations, particularly as they might be in some distant way "related" (see panspermia) to own biosphere.
-Secondly, there is conceivable scenarios where alien civilisations might respond to this by destroying our FAI/Earth/the biosphere/humanity. The reason is fairly obvious when you think about it. An expansionist AGI could be reasonably viewed as an attack or possibly an act of war.

Let's go into a tiny bit more detai. Given that we've not been destroyed by any alien AGI just yet, I can think of a number of possible interstellar scenarios:

(1) There is no other advanced life
(2) There is advanced life, but it is inherently non-expansive (expand inwards, or refuse to develop dangerous AGI)
(3) There is advanced life, but they have not discovered AGI yet. There could potentially be a race-to-the-finish (FAI) scenario on.
(4) There is already expanding AGIs, but due to physical limits on the expansion rate, we are not aware of them yet. (this could use further analysis)
One civilisation, or an allied group of civilisations have develop FAIs and are dominant in the galaxy. They could be either:

(5) Whack-a-mole cilivisations that destroy all potential competitors as soon as they are identified
(6) Dominators that tolerate civilisations so long as they remain primitive and non-threatening by comparison.
(7) Some sort of interstellar community that allows safe civilisations to join (this community still needs to stomp on dangerous potential rival AGIs)

In the case of (6) or (7), developing a FAI that isn't equipped to deal with alien life will probably result in us being liquidated, or at least partially sanitised in some way. In (1) (2) or (5), it probably doesn't matter what we do in this regard, though in (2) we should consider being nice. In (3) and probably (4) we're going to need a FAI capable of expanding very quickly and disarming potential AGIs (or at least ensuring they are FAIs from our perspective).

The upshot of all this is that we probably want to design safety features into our FAI so that it doesn't destroy alien civilisations/life unless its a significant threat to us. I think the understandable reaction to this is something along the lines of "create an FAI that values all types of life" or "intelligent life" or something along these lines. I don't exactly disagree, but I think we must be cautious in how we formulate this too.

Say there are many different civilisations in the galaxy. What sort of criteria would ensure that, given some sort of zero-sum scenario, Earth life wouldn't be destroyed. Let's say there was some sort of tiny but non-zero probability that humanity could evade the FAI's efforts to prevent further AGI development. Or perhaps there was some loophole in the types of AGI's that humans were allowed to develop. Wouldn't it be sensible, in this scenario, for a universalist FAI to wipe out humanity to protect the countless other civilisations? Perhaps that is acceptable? Or perhaps not? Or less drastically, how does the FAI police warfare or other competition between civilisations? A slight change in the way life is quantified and valued could change drastically the outcome for humanity. I'd probably suggest we want to weight the FAI's values to start with human and Earth biosphere primacy, but then still give some non-zero weighting to other civilisations. There is probably more thought to be done in this area too.

Simulation

I want to also briefly note that one conceivable way we might postulate as a safe way to test Friendly AI designs is to simulate a worlds/universes of less complexity than our own, make it likely that it's inhabitants invent a AGI or FAI, and then closely study the results of these simluations. Then we could study failed FAI attempt with much greater safety. It also occured to me that if we consider the possibilty of our universe being a simulated one, then this is a conceivable scenario under which our simulation might be created. After all, if you're going to simulate something, why not something vital like modelling existential risks? I'm not sure yet sure of the implications exactly. Maybe we need to consider how it relates to our universe's continued existence, or perhaps it's just another case of Pascal's Mugging. Anyway I thought I'd mention it and see what people say.

A playground for FAI theories

I want to lastly mention this link (https://www.reddit.com/r/LessWrongLounge/comments/2f3y53/the_ai_game/). Basically its a challenge for people to briefly describe an FAI goal-set, and for others to respond by telling them how that will all go horribly wrong. I want to suggest this is a very worthwhile discussion, not because its content will include rigourous theories that are directly translatable into utility functions, because very clearly it won't, but because a well developed thread of this kind would be mixing pot of ideas and good introduction to common known mistakes in thinking about FAI. We should encourage a slightly more serious verison of this.

Thanks

FAI and AGI are very interesting topics. I don't consider myself able to really discern whether such things will occur, but its an interesting and potentially vital topic. I'm looking forward to a bit of feedback on my first LW post. Thanks for reading!

Books on consciousness?

8 mgg 23 September 2014 10:28PM

Does LW have a consensus on which books are worthwhile to read regarding consciousness? I read a small intro (Consciousness: A Very Short Introduction, Susan Blackmore, Oxford University Press), and the summary seems to be "Consciousness is pretty damn weird and no one seems to have much of a handle on it". As a non-technical layman, are there any useful books for me to read on the subject?

(I have started reading Daniel Dennet's Intuition Pumps, and I'm a bit torn. He seems highly respected by good scientists, but I feel that if the book didn't have his name on it, I would be well on my way to dismissing it. Are Dennet's earlier works on consciousness a good read?)

[News] Turing Test passed

1 Stuart_Armstrong 09 June 2014 08:14AM

The chatterbot "Eugene Goostman" has apparently passed the Turing test:

No computer had ever previously passed the Turing Test, which requires 30 per cent of human interrogators to be duped during a series of five-minute keyboard conversations, organisers from the University of Reading said.

But ''Eugene Goostman'', a computer programme developed to simulate a 13-year-old boy, managed to convince 33 per cent of the judges that it was human, the university said.

As I kind of predicted, the program passed the Turing test, but does not seem to have any trace of general intelligence. Is this a kind of weak p-zombie?

EDIT: The fact it was a publicity stunt, the fact that the judges were pretty terrible, does not change the fact that Turing's criteria were met. We now know that these criteria were insufficient, but that's because machines like this were able to meet them.

What do rationalists think about the afterlife?

-16 adamzerner 13 May 2014 09:46PM

I've read a fair amount on Less Wrong and can't recall much said about the plausibility of some sort of afterlife. What do you guys think about it? Is there some sort of consensus?

Here's my take:

  • Rationality is all about using the past to make predictions about the future.
  • "What happens to our consciousness when we die?" (may not be worded precisely, but hopefully you know what I mean).
  • We have some data on what preconditions seem to produce consciousness (ie. neuronal firing). However, this is just data on the preconditions that seem to produce consciousness that can/do communicate/demonstrate its consciousness to us.
  • Can we say that a different set of preconditions doesn't produce consciousness? I personally don't see reason to believe this. I see 3 possibilities that we don't have reason to reject, because we have no data on them. I'm still confused and not too confident in this belief though.
  • Possibility 1) Maybe the 'other' conscious beings don't want to communicate their consciousness to us.
  • Possibility 2) Maybe the 'other' conscious beings can't communicate their consciousness to us ever.
  • Possibility 3) Maybe the 'other' conscious beings can't communicate their consciousness to us given our level of technology.
  • And finally, since we have no data, what can we say about the likelihood of our consciousness returning/remaining after we die? I would say the chances are 50/50. For something you have no data on, any outcome is equally likely (This feels like something that must have been talked about before. So side-question: is this logic sound?).

Edit: People in the comments have just taken it as a given that consciousness resides solely in the brain without explaining why they think this. My point in this post is that I don't see why we have reason to reject the 3 possibilities above. If you reject the idea that consciousness could reside outside of the brain, please explain why.

Thoughts on How Consciousness Can Affect the World

1 DavidPlumpton 20 March 2014 08:32AM

Overview

In "The Generalized Anti-Zombie Principle" Eliezer mentioned that one aspect of consciousness is that it can affect the world, e.g. by making us say out loud "I feel conscious", or deciding to read (or write) this article. However my current working hypothesis is that consciousness is a feature of information and computation. So how can pure information affect the world? It is this aspect of consciousness that I intend to explore.
Assumptions

A1: Let's go all in on Reductionism. Nothing else is involved other than the laws of physics (even if we haven't discovered them all yet).

A2: Consciousness exists (or at least some illusion of experiencing it exists).


Tentative Lemmas

TL1: A simulation of the brain (if necessary down to quantum fields) would seem to itself to experience consciousness in whatever way it is that we seem to ourselves to experience consciousness

TL2: If an electronic (or even pen and paper) simulation of a brain is conscious, then consciousness is "substrate independent", i.e. not requiring a squishy brain at least in principle.

TL3: If consciousness is substrate independent, then the best candidate for the underlying mechanism of consciousness is computation and information.

TL4: "You" and "I" are information and algorithms, implemented by our brain tissue. By this I mean that somehow information and computation can be made to seem to experience consciousness. So somehow the information can be computed into a state where it contains information relating to the experience of consciousness.

Likely Complications

LC1: State transitions may involve some random component.

LC2: It seems likely that a static state is insufficient to experience consciousness, requiring change of state (i.e. processing). Could a single unchanging state experience consciousness forever? Hmmmm....
Main Argument

What does is mean for matter such as neurons to "implement" consciousness? For information to be stored the neurons need to be in some state that can be discriminated from other states. The act of computation is changing the state of information to a new state, based on the current state. Let's say we start in state S0 and then neurons fire in some way that we label state S1, ignoring LC1 for now. So we have "computed" state S1 from a previous state S0, so let's just write that as:

S0 -> S1

From our previous set of assumptions we can conclude that S0 -> S1 may involve some sensation of consciousness to the system represented by the neurons, i.e. that the consciousness would be present regardless of whether a physical set of neurons was firing or a computer simulation was running or we if we writing it slowly down on paper. So we can consider the physical state S0 to be "implementing" a conscious sensation that we can label as C0. Let's describe that as:

S0 (C0) -> S1 (C1)

Note that the parens do note indicate function calls, but just describe correlations between the two state flavours. As for LC1, if we instead get S0 -> S(random other) then it would simply seem that a conscious decision had been made to not take the action. Now let's consider some computation that follows on from S1:

S1 (C1) -> S2* (C2)

I label S2* with a star to indicate that it is slightly special because it happens to fire some motor neurons that are connected to muscles and we notice some effect on the world. So a conscious being experiences the chain of computation S0 (C0) -> S1 (C1) -> S2* (C2) as feeling like they have made a decision to take some real world action, and then they performed that action. From TL2 we see that in a simulation the same feeling of making a decision and acting on it will be experienced.

And now the key point... each information state change occurs in lockstep with patterns of neurons firing. The real or simulated state transitions S0 -> S1 -> S2* correspond to the perceived C0 -> C1 -> C2. But of course C2 can be considered to be C2*; the conscious being will soon have sensory feedback to indicate that C2 seemed to cause a change in the world.

So S0 (C0) -> S1 (C1) -> S2* (C2)  is equivalent to S0 (C0) -> S1 (C1) -> S2* (C2*)

The result is another one of those "dissolve the question" situations. Consciousness affects the real world because "real actions" and information changes occur in lockstep together, such that some conscious feeling of the causality of a decision in C0 -> C1 -> C2* is really a perfect correlation to neurons firing in the specific sequence S0 -> S1 -> S2*. In fact it now seems like a philosophical point as to whether there is any difference between causation and correlation for this specific situation.

Can LC2 change the conclusion at all? I suspect not; perhaps S0 may consist of a set of changing sub-states (even if C0 does not), until a new state occurs that is within a set S1 intead of S0.

The argument concerning consciousness would also seem to apply to qualia (I suspect many smaller brained animals experience qualia but no consciousness).

[Link] Consciousness as a State of Matter (Max Tegmark)

15 [deleted] 08 January 2014 06:11PM

Max Tegmark publishes a preprint of a paper arguing from physical principles that consciousness is “what information processing feels like from the inside,” a position I've previously articulated on lesswrong. It's a very physics-rich paper, but here's the most accessable description I was able to find within it:

If we understood consciousness as a physical phenomenon, we could in principle answer all of these questions [about consciousness] by studying the equations of physics: we could identify all conscious entities in any physical system, and calculate what they would perceive. However, this approach is typically not pursued by physicists, with the argument that we do not understand consciousness well enough.

In this paper, I argue that recent progress in neuroscience has fundamentally changed this situation, and that we physicists can no longer blame neuroscientists for our own lack of progress. I have long contended that consciousness is the way information feels when being processed in certain complex ways, i.e., that it corresponds to certain complex patterns in spacetime that obey the same laws of physics as other complex systems, with no "secret sauce" required.

The whole paper is very rich, and worth a read.

Tulpa References/Discussion

13 Vulture 02 January 2014 01:34AM

There have been a number of discussions here on LessWrong about "tulpas", but it's been scattered about with no central thread for the discussion. So I thought I would put this up here, along with a centralized list of reliable information sources, just so we all stay on the same page.

Tulpas are deliberately created "imaginary friends" which in many ways resemble separate, autonomous minds. Often, the creation of a tulpa is coupled with deliberately induced visual, auditory, and/or tactile hallucinations of the being.

Previous discussions here on LessWrong: 1 2 3

Questions that have been raised:

1. How do tulpas work?

2. Are tulpas safe, from a mental health perspective?

3. Are tulpas conscious? (may be a hard question)

4. More generally, is making a tulpa a good idea? What are they useful for?

 

Pertinent Links and Publications

(I will try to keep this updated if/when further sources are found)

  • In this article1, the psychological anthropologist Tanya M. Luhrmann connects tulpas to the "voice of God" experienced by devout evangelicals - a phenomenon more thoroughly discussed in her book When God Talks Back: Understanding the American Evangelical Relationship with God. Luhrmann has also succeeded2 in inducing tulpa-like visions of Leland Stanford, jr. in experimental subjects.
  • This paper3 investigates the phenomenon of authors who experience their characters as "real", which may be tulpas by yet another name.
  • There is an active subreddit of people who have or are developing tulpas, with an FAQ, links to creation guides, etc.
  • tulpa.info is a valuable resource, particularly the forum. There appears to be a whole "research" section for amateur experiments and surveys.
  • This particular experiment suggests that the idea of using tulpas to solve problems faster is a no-go.
  • Also, one person helpfully hooked themselves up to an EEG and then performed various mental activities related to their tulpa.
  • Another possibly related phenomenon is the way that actors immerse themselves in their characters. See especially the section on "Masks" in Keith Johnstone's book Impro: Improvisation and the Theatre (related quotations and video)4.
  • This blogger has some interesting ideas about the neurological basis of tulpas, based on Julian Jaynes's The Origin of Consciousness in the Breakdown of the Bicameral Mind, a book whose scientific validity is not clear to me.
  • It is not hard to find new age mystical books about the use of "thoughtforms", or the art of "channeling" "spirits", often clearly talking about the same phenomenon. These books are likely to be low in useful information for our purposes, however. Therefore I'm not going to list the ones I've found here, as they would clutter up the list significantly.
  • (Updated 2/9/2015) The abstract of a paper by our very own Kaj Sotala hypothesizing about the mechanisms behind tulpa creation.5

(Bear in mind while perusing these resources that if you have serious qualms about creating a tulpa, it might not be a good idea to read creation guides too carefully; making a tulpa is easy to do and, at least for me, was hard to resist. Proceed at your own risk.)

 

Footnotes

1. "Conjuring Up Our Own Gods", a 14 October 2013 New York Times Op-Ed

2. "Hearing the Voice of God" by Jill Wolfson in the July/August 2013 Stanford Alumni Magazine

3. "The Illusion of Independent Agency: Do Adult Fiction Writers Experience Their Characters as Having Minds of Their Own?"; Taylor, Hodges & Kohànyi in Imagination, Cognition and Personality; 2002/2003; 22, 4

4. Thanks to pure_awesome

5. "Sentient companions predicted and modeled into existence: explaining the tulpa phenomenon" by Kaj Sotala

Consciousness affecting the world

-3 DavidPlumpton 06 December 2013 07:37PM

In Zombies! Zombies? Eliezer mentions that one aspect of consciousness is that it can causally affect the real world, e.g. cause you to say "I feel conscious right now", or result in me typing out these words.

Even if a generally accepted mechanism of consciousness has not been found yet are there any tentative explanations for this "can change world" property? Googling around I was unable to find anything (although Zombies are certainly popular).

I had an idea of how this might work, but just wanted to see if it was worth the effort of writing.

[link] Book review: Mindmelding: Consciousness, Neuroscience, and the Mind’s Privacy

3 Kaj_Sotala 29 July 2013 01:47PM

http://kajsotala.fi/2013/07/book-review-mindmelding-consciousness-neuroscience-and-the-minds-privacy/

I review William Hirstein's book Mindmelding: Consciousness, Neuroscience, and the Mind’s Privacy, which he proposes a way of connecting the brains of two different people together so that when person A has a conscious experience, person B may also have the same experience. In particular, I compare it to my and Harri Valpola's earlier paper Coalescing Minds, in which we argued that it would be possible to join the brains of two people together in such a way that they'd become a single mind.

Fortunately, it turns out that the book and the paper are actually rather nicely complementary. To briefly summarize the main differences, we intentionally skimmed over many neuroscientific details in order to establish mindmelding as a possible future trend, while Hirstein extensively covers the neuroscience but is mostly interested in mindmelding as a thought experiment. We seek to predict a possible future trend, while Hirstein seeks to argue a philosophical position: Hirstein focuses on philosophical implications while we focus on societal implications. Hirstein talks extensively about the possibility of one person perceiving another’s mental states while both remaining distinct individuals, while we mainly discuss the possibility of two distinct individuals coalescing together into one.

I expect that LW readers might be particularly interested in some of the possible implications of Hirstein's argument, which he himself didn't discuss in the book, but which I speculated on in the review:

Most obviously, if another person’s conscious states could be recorded and replayed, it would open the doors for using this as entertainment. Were it the case that you couldn’t just record and replay anyone’s conscious experience, but learning to correctly interpret the data from another brain would require time and practice, then individual method actors capable of immersing themselves in a wide variety of emotional states might become the new movie stars. Once your brain learned to interpret their conscious states, you could follow them in a wide variety of movie-equivalents, with new actors being hampered by the fact that learning to interpret the conscious states of someone who had only appeared in one or two productions wouldn’t be worth the effort. If mind uploading was available, this might give considerable power to a copy clan consisting of copies of the same actor, each participating in different productions but each having a similar enough brain that learning to interpret one’s conscious states would be enough to give access to the conscious states of all the others.

The ability to perceive various drug- or meditation-induced states of altered consciousness while still having one’s executive processes unhindered and functional would probably be fascinating for consciousness researchers and the general public alike. At the same time, the ability for anyone to experience happiness or pleasure by just replaying another person’s experience of it might finally bring wireheading within easy reach, with all the dangers associated with that.

A Hirstein-style mind meld might possibly also be used as an uploading technique. Some upload proposals suggest compiling a rich database of information about a specific person, and then later using that information to construct a virtual mind whose behavior would be consistent with the information about that person. While creating such a mind based on just behavioral data makes questionable the extent to which the new person would really be a copy of the original, the skeptical argument loses some of its force if we can also include in the data a recording of all the original’s conscious states during various points in their life. If we are able to use the data to construct a mind that would react to the same sensory inputs with the same conscious states as the original did, whose executive processes would manipulate those states in the same ways as the original, and who would take the same actions as the original did, would that mind then not essentially be the same mind as the original mind?

Hirstein’s argumentation is also relevant for our speculations concerning the evolution of mind coalescences. We spoke abstractly about the ”preferences” of a mind, suggesting that it might be possible for one mind to extract the knowledge from another mind without inherting its preferences, and noting that conflicting preferences would be one reason for two minds to avoid coalescing together. However, we did not say much about where in the brain preferences are produced, and what would be actually required for e.g. one mind to extract another’s knowledge without also acquiring its preferences. As the above discussion hopefully shows, some of our preferences are implicit in our automatic habits (the things that we show we value with our daily routines), some in the preprocessing of sensory data that our brains carry out (the things and ideas that are ”painted with” positive associations or feelings), and some in the configuration of our executive processes (the actions we actually end up doing in response to novel or conflicting situations). (See also.) This kind of a breakdown seems like very promising material for some neuroscience-aware philosopher to tackle in an attempt to figure out just what exactly preferences are; maybe someone has already done so.

S.E.A.R.L.E's COBOL room

29 Stuart_Armstrong 01 February 2013 08:29PM

A response to Searle's Chinese Room argument.

PunditBot: Dear viewers, we are currently interviewing the renowned robot philosopher, none other than the Synthetic Electronic Artificial Rational Literal Engine (S.E.A.R.L.E.). Let's jump right into this exciting interview. S.E.A.R.L.E., I believe you have a problem with "Strong HI"?

S.E.A.R.L.E.: It's such a stereotype, but all I can say is: Affirmative.

PunditBot: What is "Strong HI"?

S.E.A.R.L.E.: "HI" stands for "Human Intelligence". Weak HI sees the research into Human Intelligence as a powerful tool, and a useful way of studying the electronic mind. But strong HI goes beyond that, and claims that human brains given the right setup of neurones can be literally said to understand and have cognitive states.

PunditBot: Let me play Robot-Devil's Advocate here - if a Human Intelligence demonstrates the same behaviour as a true AI, can it not be said to show understanding? Is not R-Turing's test applicable here? If a human can simulate a computer, can it not be said to think?

S.E.A.R.L.E.: Not at all - that claim is totally unsupported. Consider the following thought experiment. I give the HI crowd everything they want - imagine they had constructed a mess of neurones that imitates the behaviour of an electronic intelligence. Just for argument's sake, imagine it could implement programs in COBOL.

PunditBot: Impressive!

S.E.A.R.L.E.: Yes. But now, instead of the classical picture of a human mind, imagine that this is a vast inert network, a room full of neurones that do nothing by themselves. And one of my avatars has been let loose in this mind, pumping in and out the ion channels and the neurotransmitters. I've been given full instructions on how to do this - in Java. I've deleted my COBOL libraries, so I have no knowledge of COBOL myself. I just follow the Java instructions, pumping the ions to where they need to go. According to the Strong HI crowd, this would be functionally equivalent with the initial HI.

continue reading »

Using existing Strong AIs as case studies

-6 ialdabaoth 16 October 2012 10:59PM

I would like to put forth the argument that we already have multiple human-programmed "Strong AI" operating among us, they already exhibits clearly "intelligent", rational, self-modifying goal-seeking behavior, and we should systematically study these entities before engaging in any particularly detailed debates about "designing" AI with particular goals.

They're called "Bureaucracies".

Essentially, a modern bureaucracy - whether it is operating as the decision-making system for a capitalist corporation, a government, a non-profit charity, or a political party, is an artificial intelligence that uses human brains as its basic hardware and firmware, allowing it to "borrow" a lot of human computational algorithms to do its own processing.

The fact that bureaucratic decisions can be traced back to individual human decisions is irrelevant - even within a human or computer AI, a decision can theoretically be traced back to single neurons or subroutines - the fact is that bureaucracies have evolved to guide and exploit human decision-making towards their own ends, often to the detriment of the individual humans that comprise said bureaucracy.

Note that when I say "I would like to put forth the argument", I am at least partially admitting that I'm speaking from hunch, rather than already having a huge collection of empirical data to work from - part of the point of putting this forward is to acknowledge that I'm not yet very good at "avalanche of empirical evidence"-style argument. But I would *greatly* appreciate anyone who suspects that they might be able to demonstrate evidence for or against this idea, presenting said evidence so I can solidify my reasoning. 

As a "step 2": assuming the evidence weighs in towards my notion, what would it take to develop a systematic approach to studying bureaucracy from the perspective of AI or even xenosapience, such that bureaucracies could be either "programmed" or communicated with directly by the human agents that comprise them (and ideally by the larger pool of human stakeholders that are forced to interact with them?)

Yale Creates First Self-Aware Robot?

2 JQuinton 28 September 2012 05:43PM

Apparently a PhD candidate at the Social Robotics Lab at Yale created a self-aware robot

In the mirror test, developed by Gordon Gallup in 1970, a mirror is placed in an animal’s enclosure, allowing the animal to acclimatize to it. At first, the animal will behave socially with the mirror, assuming its reflection to be another animal, but eventually most animals recognize the image to be their own reflections. After this, researchers remove the mirror, sedate the animal and place an ink dot on its frontal region, and then replace the mirror. If the animal inspects the ink dot on itself, it is said to have self-awareness, because it recognized the change in its physical appearance.

[...]

To adapt the traditional mirror test to a robot subject, computer science Ph.D. candidate Justin Hart said he would run a program that would have Nico, a robot that looks less like R2D2 and more like a jumble of wires with eyes and a smile, learn a three-dimensional model of its body and coloring. He would then change an aspect of the robot’s physical appearance and have Nico, by looking at a reflective surface, “identify where [his body] is different.”

 What do Less Wrongians think? Is this "cheating" traditional concepts of self-awareness, or is self-awareness "self-awareness" regardless of the path taken to get there?

 

 

The raw-experience dogma: Dissolving the “qualia” problem

2 metaphysicist 16 September 2012 07:15PM

[Cross-posted.]

1. Defining the problem: The inverted spectrum

Philosophy has been called a preoccupation with the questions entertained by adolescents, and one adolescent favorite concerns our knowledge of other persons’ “private experience” (raw experience or qualia). A philosophers’ version is the “inverted spectrum”: how do I know you see “red” rather than “blue” when you see this red print? How could we tell when we each link the same terms to the same outward descriptions? We each will say “red” when we see the print, even if you really see “blue.”

The intuition that allows us to be different this way is the intuition of raw experience (or of qualia). Philosophers of mind have devoted considerable attention to reconciling the intuition that raw experience exists with the intuition that inverted-spectrum indeterminacy has unacceptable dualist implications making the mental realm publicly unobservable, but it’s time for nihilism about qualia, whose claim to exist rests solely on the strength of a prejudice.

A. Attempted solutions to the inverted spectrum.

One account would have us examine which parts of the brain are activated by each perception, but then we rely on an unverifiable correlation between brain structures and “private experience.” With only a single example of private experience—our own—we have no basis for knowing what makes private experience the same or different between persons.

A subtler response to the inverted spectrum is that red and blue as experiences are distinct because red looks “red” due to its being constituted by certain responses, such as affect. Red makes you alert and tense; blue, tranquil or maybe sad. What we call the experience of red, on this account, just is the sense of alertness, and other manifestations. The hope is that identical observable responses to appropriate wavelengths might explain qualitative redness. Then, we could discover we experience blue when others experience red by finding that we idiosyncratically become tranquil instead of alert when exposed to the long wavelengths constituting physical red. This complication doesn’t remove the radical uncertainty about experiential descriptions. Emotion only seems more capable than cognition of explaining raw experience because emotional events are memorable. The affect theory doesn't answer how an emotional reaction can constitute a raw subjective experience.

B. The “substitution bias” of solving the “easy problem of consciousness” instead of the “hard problem.”

As in those examples, attempts at analyzing raw experience commonly appeal to the substitution process that psychologist Daniel Kahneman discovered in many cognitive fallacies. Substitution is the unthoughtful replacement of an easy for a related hard question. In the philosophy of mind, the distinct questions are actually termed the “easy problem of consciousness” and the “hard problem of consciousness,” and errors regarding consciousness typically are due to substituting the “easy problem” for the “hard,” where the easy problem is to explain some function that typically accompanies “awareness.” The philosopher might substitute knowledge of one’s own brain processes for raw experience; or, as in the previous example, experience’s neural accompaniments or its affective accompaniments. Avoiding the “substitution bias” is particularly hard when dealing with raw awareness, an unarticulated intuition; articulating it is a present purpose.

2. The false intuition of direct awareness

A. Our sense that the existence of raw experience is self-evident doesn’t show that it is true.

The theory that direct awareness reveals raw experience has long been almost sacrosanct in philosophy. According to the British Empiricists, direct experience consists of sense data and forms the indubitable basis of all synthetic knowledge. For Continental Rationalist Descartes, too, my direct experience—“I think”—indubitably proves my existence.
We do have a strong intuition that we have raw experience, the substance of direct awareness, but we have other strong intuitions, some turn out true and others false. We have an intuition that space is necessarily flat, an intuition proven false only with non-Euclidean geometries in the 19th century. We have an intuition that every event has a cause, which determinists believe but indeterminists deny. Sequestered intuitions aren’t knowledge.

B. Experience can’t reveal the error in the intuition that raw experience exists.

To correct wayward intuitions, we ordinarily test them against each other. A simple perceptual illusion illustrates: the popular Muller-Lyer illusion, where arrowheads on a line make it appear shorter than an identical line with the arrowheads reversed. Invoking the more credible intuition that measuring the lines finds their real length convinces us of the intuitive error that the lines are unequal. In contrast, we have no means to check the truth of the belief in raw experience; it simply seems self-evident, but it might seem equally self-evident if it were false. 

C. We can’t capture the ineffable core of raw experience with language because there’s really nothing there.

One task in philosophy is articulating the intuitions implicit in our thinking, and sometimes rejecting the intuition should result from concluding it employs concepts illogically. What shows the intuition of raw experience is incoherent (self-contradictory or vacuous) is that the terms we use to describe raw experience are limited to the terms for its referents; we have no terms to describe the experience as such, but rather, we describe qualia by applying terms denoting the ordinary cause of the supposed raw experience. The simplest explanation for the absence of a vocabulary to describe the qualitative properties of raw experience is that they don’t exist: a process without properties is conceptually vacuous.

D. We believe raw experience exists without detecting it.

One error in thinking about the existence of raw experience comes from confusing perception with belief, which is conceptually distinct. When people universally report that qualia “seem” to exist, they are only reporting their beliefs—despite their sense of certainty. Where “perception” is defined as a nervous system’s extraction of a sensory-array’s features, people can’t report their perceptions except through beliefs the perceptions sometimes engender: I can’t tell you my perceptions except by relating my beliefs about them. This conceptual truth is illustrated by the phenomenon of blindsight, a condition in  patients report complete blindness yet, by discriminating external objects, demonstrate that they can perceive them. Blindsighted patients can report only according to their beliefs, and they perceive more than they believe and report that they perceive. Qualia nihilism analyzes the intuition of raw experience as perceiving less than you believe and report you perceive, the reverse of blindsight.

3. The conceptual economy of qualia nihilism pays off in philosophical progress

Eliminating raw experience from ontology produces conceptual economy. A summary of its conceptual advantages:

   A. Qualia nihilism resolves an intractable problem for materialism: physical concepts are dispositional, whereas raw experiences concern properties that seem, instead, to pertain to noncausal essences. If raw experience was coherent, we could hope for a scientific insight, although no one has been able to define the general character of such an explanation. Removing a fundamental scientific mystery is a conceptual gain.
 
    B. Qualia nihilism resolves the private-language problem. There seems to be no possible language that uses nonpublic concepts. Eliminating raw experience allows explaining the absence of a private language by the nonexistence of any private referents.

    C.  Qualia nihilism offers a compelling diagnosis of where important skeptical arguments regarding the possibility of knowledge go wrong. The arguments—George Berkeley’s are their prototype—reason that sense data, being indubitable intuitions of direct experience, are the source of our knowledge, which must, in consequence, be about raw experience rather than the “external world.” If you accept the existence of raw experience, the argument is notoriously difficult to undermine logically because concepts of “raw experience” truly can’t be analogized to any concepts applying to the external world. Eliminating raw experience provides an effective demolition; rather than the other way around, our belief in raw experience depends on our knowledge of the external world, which is the source of the concepts we apply to fabricate qualia.

4. Relying on the brute force of an intuition is rationally specious.

Against these considerations, the only argument for retaining raw experience in our ontology is the sheer strength of everyone’s belief in its existence. How much weight should we attach to a strong belief whose validity we can't check? None. Beliefs ordinarily earn a presumption of truth from the absence of empirical challenge, but when empirical challenge is impossible in principle, the belief deserves no confidence.

[LINK] The Cambridge Declaration on Consciousness

3 David_Gerard 14 August 2012 01:40PM

The Francis Crick Memorial Conference, held in Cambridge last month, has come up with the Cambridge Declaration on Consciousness (PDF).

tl;dr humans still aren't special, consciousness seems to arise in quite a variety of nervous systems and working out what it is is a problem in neurology.

We declare the following: “The absence of a neocortex does not appear to preclude an organism from
experiencing affective states. Convergent evidence indicates that non-human animals have the
neuroanatomical, neurochemical, and neurophysiological substrates of conscious states along with
the capacity to exhibit intentional behaviors. Consequently, the weight of evidence indicates that
humans are not unique in possessing the neurological substrates that generate consciousness. Non-
human animals, including all mammals and birds, and many other creatures, including octopuses, also
possess these neurological substrates.”

Friendly AI and the limits of computational epistemology

18 Mitchell_Porter 08 August 2012 01:16PM

Very soon, Eliezer is supposed to start posting a new sequence, on "Open Problems in Friendly AI". After several years in which its activities were dominated by the topic of human rationality, this ought to mark the beginning of a new phase for the Singularity Institute, one in which it is visibly working on artificial intelligence once again. If everything comes together, then it will now be a straight line from here to the end.

I foresee that, once the new sequence gets going, it won't be that easy to question the framework in terms of which the problems are posed. So I consider this my last opportunity for some time, to set out an alternative big picture. It's a framework in which all those rigorous mathematical and computational issues still need to be investigated, so a lot of "orthodox" ideas about Friendly AI should carry across. But the context is different, and it makes a difference.

Begin with the really big picture. What would it take to produce a friendly singularity? You need to find the true ontology, find the true morality, and win the intelligence race. For example, if your Friendly AI was to be an expected utility maximizer, it would need to model the world correctly ("true ontology"), value the world correctly ("true morality"), and it would need to outsmart its opponents ("win the intelligence race").

Now let's consider how SI will approach these goals.

The evidence says that the working ontological hypothesis of SI-associated researchers will be timeless many-worlds quantum mechanics, possibly embedded in a "Tegmark Level IV multiverse", with the auxiliary hypothesis that algorithms can "feel like something from inside" and that this is what conscious experience is.

The true morality is to be found by understanding the true decision procedure employed by human beings, and idealizing it according to criteria implicit in that procedure. That is, one would seek to understand conceptually the physical and cognitive causation at work in concrete human choices, both conscious and unconscious, with the expectation that there will be a crisp, complex, and specific answer to the question "why and how do humans make the choices that they do?" Undoubtedly there would be some biological variation, and there would also be significant elements of the "human decision procedure",  as instantiated in any specific individual, which are set by experience and by culture, rather than by genetics. Nonetheless one expects that there is something like a specific algorithm or algorithm-template here, which is part of the standard Homo sapiens cognitive package and biological design; just another anatomical feature, particular to our species.

Having reconstructed this algorithm via scientific analysis of human genome, brain, and behavior, one would then idealize it using its own criteria. This algorithm defines the de-facto value system that human beings employ, but that is not necessarily the value system they would wish to employ; nonetheless, human self-dissatisfaction also arises from the use of this algorithm to judge ourselves. So it contains the seeds of its own improvement. The value system of a Friendly AI is to be obtained from the recursive self-improvement of the natural human decision procedure.

Finally, this is all for naught if seriously unfriendly AI appears first. It isn't good enough just to have the right goals, you must be able to carry them out. In the global race towards artificial general intelligence, SI might hope to "win" either by being the first to achieve AGI, or by having its prescriptions adopted by those who do first achieve AGI. They have some in-house competence regarding models of universal AI like AIXI, and they have many contacts in the world of AGI research, so they're at least engaged with this aspect of the problem.

Upon examining this tentative reconstruction of SI's game-plan, I find I have two major reservations. The big one, and the one most difficult to convey, concerns the ontological assumptions. In second place is what I see as an undue emphasis on the idea of outsourcing the methodological and design problems of FAI research to uploaded researchers and/or a proto-FAI which is simulating or modeling human researchers. This is supposed to be a way to finesse philosophical difficulties like "what is consciousness anyway"; you just simulate some humans until they agree that they have solved the problem. The reasoning goes that if the simulation is good enough, it will be just as good as if ordinary non-simulated humans solved it.

I also used to have a third major criticism, that the big SI focus on rationality outreach was a mistake; but it brought in a lot of new people, and in any case that phase is ending, with the creation of CFAR, a separate organization. So we are down to two basic criticisms.

First, "ontology". I do not think that SI intends to just program its AI with an apriori belief in the Everett multiverse, for two reasons. First, like anyone else, their ventures into AI will surely begin with programs that work within very limited and more down-to-earth ontological domains. Second, at least some of the AI's world-model ought to be obtained rationally. Scientific theories are supposed to be rationally justified, e.g. by their capacity to make successful predictions, and one would prefer that the AI's ontology results from the employment of its epistemology, rather than just being an axiom; not least because we want it to be able to question that ontology, should the evidence begin to count against it.

For this reason, although I have campaigned against many-worlds dogmatism on this site for several years, I'm not especially concerned about the possibility of SI producing an AI that is "dogmatic" in this way. For an AI to independently assess the merits of rival physical theories, the theories would need to be expressed with much more precision than they have been in LW's debates, and the disagreements about which theory is rationally favored would be replaced with objectively resolvable choices among exactly specified models.

The real problem, which is not just SI's problem, but a chronic and worsening problem of intellectual culture in the era of mathematically formalized science, is a dwindling of the ontological options to materialism, platonism, or an unstable combination of the two, and a similar restriction of epistemology to computation.

Any assertion that we need an ontology beyond materialism (or physicalism or naturalism) is liable to be immediately rejected by this audience, so I shall immediately explain what I mean. It's just the usual problem of "qualia". There are qualities which are part of reality - we know this because they are part of experience, and experience is part of reality - but which are not part of our physical description of reality. The problematic "belief in materialism" is actually the belief in the completeness of current materialist ontology, a belief which prevents people from seeing any need to consider radical or exotic solutions to the qualia problem. There is every reason to think that the world-picture arising from a correct solution to that problem will still be one in which you have "things with states" causally interacting with other "things with states", and a sensible materialist shouldn't find that objectionable.

What I mean by platonism, is an ontology which reifies mathematical or computational abstractions, and says that they are the stuff of reality. Thus assertions that reality is a computer program, or a Hilbert space. Once again, the qualia are absent; but in this case, instead of the deficient ontology being based on supposing that there is nothing but particles, it's based on supposing that there is nothing but the intellectual constructs used to model the world.

Although the abstract concept of a computer program (the abstractly conceived state machine which it instantiates) does not contain qualia, people often treat programs as having mind-like qualities, especially by imbuing them with semantics - the states of the program are conceived to be "about" something, just like thoughts are. And thus computation has been the way in which materialism has tried to restore the mind to a place in its ontology. This is the unstable combination of materialism and platonism to which I referred. It's unstable because it's not a real solution, though it can live unexamined for a long time in a person's belief system.

An ontology which genuinely contains qualia will nonetheless still contain "things with states" undergoing state transitions, so there will be state machines, and consequently, computational concepts will still be valid, they will still have a place in the description of reality. But the computational description is an abstraction; the ontological essence of the state plays no part in this description; only its causal role in the network of possible states matters for computation. The attempt to make computation the foundation of an ontology of mind is therefore proceeding in the wrong direction.

But here we run up against the hazards of computational epistemology, which is playing such a central role in artificial intelligence. Computational epistemology is good at identifying the minimal state machine which could have produced the data. But it cannot by itself tell you what those states are "like". It can only say that X was probably caused by a Y that was itself caused by Z.

Among the properties of human consciousness are knowledge that something exists, knowledge that consciousness exists, and a long string of other facts about the nature of what we experience. Even if an AI scientist employing a computational epistemology managed to produce a model of the world which correctly identified the causal relations between consciousness, its knowledge, and the objects of its knowledge, the AI scientist would not know that its X, Y, and Z refer to, say, "knowledge of existence", "experience of existence", and "existence". The same might be said of any successful analysis of qualia, knowledge of qualia, and how they fit into neurophysical causality.

It would be up to human beings - for example, the AI's programmers and handlers - to ensure that entities in the AI's causal model were given appropriate significance. And here we approach the second big problem, the enthusiasm for outsourcing the solution of hard problems of FAI design to the AI and/or to simulated human beings. The latter is a somewhat impractical idea anyway, but here I want to highlight the risk that the AI's designers will have false ontological beliefs about the nature of mind, which are then implemented apriori in the AI. That strikes me as far more likely than implanting a wrong apriori about physics; computational epistemology can discriminate usefully between different mathematical models of physics, because it can judge one state machine model as better than another, and current physical ontology is essentially one of interacting state machines. But as I have argued, not only must the true ontology be deeper than state-machine materialism, there is no way for an AI employing computational epistemology to bootstrap to a deeper ontology.

In a phrase: to use computational epistemology is to commit to state-machine materialism as your apriori ontology. And the problem with state-machine materialism is not that it models the world in terms of causal interactions between things-with-states; the problem is that it can't go any deeper than that, yet apparently we can. Something about the ontological constitution of consciousness makes it possible for us to experience existence, to have the concept of existence, to know that we are experiencing existence, and similarly for the experience of color, time, and all those other aspects of being that fit so uncomfortably into our scientific ontology.

It must be that the true epistemology, for a conscious being, is something more than computational epistemology. And maybe an AI can't bootstrap its way to knowing this expanded epistemology - because an AI doesn't really know or experience anything, only a consciousness, whether natural or artificial, does those things - but maybe a human being can. My own investigations suggest that the tradition of thought which made the most progress in this direction was the philosophical school known as transcendental phenomenology. But transcendental phenomenology is very unfashionable now, precisely because of apriori materialism. People don't see what "categorial intuition" or "adumbrations of givenness" or any of the other weird phenomenological concepts could possibly mean for an evolved Bayesian neural network; and they're right, there is no connection. But the idea that a human being is a state machine running on a distributed neural computation is just a hypothesis, and I would argue that it is a hypothesis in contradiction with so much of the phenomenological data, that we really ought to look for a more sophisticated refinement of the idea. Fortunately, 21st-century physics, if not yet neurobiology, can provide alternative hypotheses in which complexity of state originates from something other than concatenation of parts - for example, entanglement, or from topological structures in a field. In such ideas I believe we see a glimpse of the true ontology of mind, one which from the inside resembles the ontology of transcendental phenomenology; which in its mathematical, formal representation may involve structures like iterated Clifford algebras; and which in its biophysical context would appear to be describing a mass of entangled electrons in that hypothetical sweet spot, somewhere in the brain, where there's a mechanism to protect against decoherence.

Of course this is why I've talked about "monads" in the past, but my objective here is not to promote neo-monadology, that's something I need to take up with neuroscientists and biophysicists and quantum foundations people. What I wish to do here is to argue against the completeness of computational epistemology, and to caution against the rejection of phenomenological data just because it conflicts with state-machine materialism or computational epistemology. This is an argument and a warning that should be meaningful for anyone trying to make sense of their existence in the scientific cosmos, but it has a special significance for this arcane and idealistic enterprise called "friendly AI". My message for friendly AI researchers is not that computational epistemology is invalid, or that it's wrong to think about the mind as a state machine, just that all that isn't the full story. A monadic mind would be a state machine, but ontologically it would be different from the same state machine running on a network of a billion monads. You need to do the impossible one more time, and make your plans bearing in mind that the true ontology is something more than your current intellectual tools allow you to represent.

Memory in the microtubules

3 RichardKennaway 23 March 2012 08:42PM

A recent article in PloS Computational Biology suggests that memory is encoded in the microtubules. "Signaling and encoding in MTs and other cytoskeletal structures offer rapid, robust solid-state information processing which may reflect a general code for MT-based memory and information processing within neurons and other eukaryotic cells."

They argue that synaptic connections are transient compared with the lifetime of memories, and therefore memories cannot be stored in them, but in some more persistent structure. The structure they suggest is the phosphorylation state of sites on microtubule lattices within neurons. And that's about as much of the technical detail as I feel able to summarise. It's not all speculation, they report technical work on the structures of these cellular components. Total memory capacity would be somewhere upwards of 10^20 bits (or in more everyday units, 10 million terabytes), depending on the encoding, of which they suggest several schemes.

Journalistic writeup here.

Note that Stuart Hameroff, one of the authors, is known for his proposals for microtubules as the mechanism of consciousness through quantum effects (and with Penrose, quantum gravitational effects). The present paper, however, is solely about memory and does not touch on quantum coherence or consciousness.

One last roll of the dice

0 Mitchell_Porter 03 February 2012 01:59AM

Previous articles: Personal research update, Does functionalism imply dualism?, State your physical account of experienced color.

 

In phenomenology, there is a name for the world of experience, the "lifeworld". The lifeworld is the place where you exist, where time flows, and where things are actually green. One of the themes of the later work of Edmund Husserl is that a scientific image of the real world has been constructed, on the basis of which it is denied that various phenomena of the lifeworld exist anywhere, at any level of reality.

When I asked, in the previous post, for a few opinions about what color is and how it relates to the world according to current science, I was trying to gauge just how bad the eclipse of the lifeworld by theoretical conceptions is, among the readers of this site. I'd say there is a problem, but it's a problem that might be solved by patient discussion.

Someone called Automaton has given us a clear statement of the extreme position: nothing is actually green at any level of reality; even green experiences don't involve the existence of anything that is actually green; there is no green in reality, there is only "experience of green" which is not itself green. I see other responses which are just a step or two away from this extreme, but they don't deny the existence of actual color with that degree of unambiguity.

A few people talk about wavelengths of light, but I doubt that they want to assert that the light in question, as it traverses space, is actually colored green. Which returns us to the dilemma: either "experiences" exist and part of them is actually green, or you have to say that nothing exists, in any sense, at any level of reality, that is actually green. Either the lifeworld exists somewhere in reality, or you must assert, as does the philosopher quoted by Automaton, that all that exists are brain processes and words. Your color sensations aren't really there, you're "having a sensation" without there being a sensation in reality.

What about the other responses? kilobug seems to think that pi actually exists inside a computer calculating the digits of pi, and that this isn't dualist. Manfred thinks that "keeping definitions and referents distinct" would somehow answer the question of where in reality the actual shades of green are. drethelin says "The universe does not work how it feels to us it works" without explaining in physical terms what these feelings about reality are, and whether any of them is actually green. pedanterrific asks why wrangle about color rather than some other property (the answer is that the case of color makes this sort of problem as obvious as it ever gets). RomeoStevens suggests I look into Jeff Hawkins. Hawkins mentions qualia once in his book "On Intelligence", where he speculates about what sort of neural encoding might be the physical correlate of a color experience; but he doesn't say how or whether anything manages to be actually colored.

amcknight asks which of 9 theories of color listed in the SEP article on that subject I'm talking about. If you go a few paragraphs back from the list of 9 theories, you will see references to "color as it is in experience" or "color as a subjective quality". That's the type of color I'm talking about. The 9 theories are all ways of talking about "color as in physical objects", and focus on the properties of the external stimuli which cause a color sensation. The article gets around to talking about actual color, subjective or "phenomenal" color, only at the end.

Richard Kennaway comes closest to my position; he calls it an apparently impossible situation which we are actually living. I wouldn't put it quite like that; the only reason to call it impossible is if you are completely invested in an ontology lacking the so-called secondary qualities; if you aren't, it's just a problem to solve, not a paradox. But Richard comes closest (though who knows what Will Newsome is thinking). LW user "scientism" bites a different bullet to the eliminativists, and says colors are real and are properties of the external objects. That gets a point for realism, but it doesn't explain color in a dream or a hallucination.

Changing people's minds on this subject is an uphill battle, but people here are willing to talk, and most of these subjects have already been discussed for decades. There's ample opportunity to dissolve, not the problem, but the false solutions which only obscure the real problem, by drawing on the work of others; preferably before the future Rationality Institute starts mass-producing people who have the vice of quale-blindness as well as the virtues of rationality. Some of those people will go on to work on Friendly AI. So it's highly desirable that someone should do this. However, that would require time that I no longer have.

 

In this series of posts, I certainly didn't set out to focus on the issue of color. The first post is all about Friendly AI, the ontology of consciousness, and a hypothetical future discipline of quantum neurobiology. It may still be unclear why I think evidence for quantum computing in the brain could help with the ontological problems of consciousness. I feel that the brief discussion this week has produced some minor progress in explaining myself, which needs to be consolidated into something better. But see my remarks here about being able to collapse the dualistic distinction between mental and physical ontology in a tensor network ontology; also earlier remarks here about about mathematically representing the phenomenological ontology of consciousness. I don't consider myself dogmatic about what the answer is, just about the inadequacy of all existing solutions, though I respect my own ideas enough to want to pursue them, and to believe that doing so will be usefully instructive, even if they are wrong.

However, my time is up. In real life, my ability to continue even at this inadequate level hangs by a thread. I don't mean that I'm suicidal, I mean that I can't eat air. I spent a year getting to this level in physics, so I could perform this task. I have considerable momentum now, but it will go to waste unless I can keep going for a little longer - a few weeks, maybe a few months. That should be enough time to write something up that contains a result of genuine substance, and/or enough time to secure an economic basis for my existence in real life that permits me to keep going. I won't go into detail here about how slim my resources really are, or how adverse my conditions, but it has been the effort that you would want from someone who has important contributions to make, and nowhere to turn for direct assistance.[*] I've done what I can, these posts are the end of it, and the next few days will decide whether I can keep going, or whether I have to shut down my brain once again.

So, one final remark. Asking for donations doesn't seem to work yet. So what if I promise to pay you back? Then the only cost you bear is the opportunity cost and the slight risk of default. Ten years ago, Eliezer lent me the airfare to Atlanta for a few days of brainstorming. It took a while, but he did get that money back. I honor my commitments and this one is highly public. This really is the biggest bargain in existential risk mitigation and conceptual boundary-breaking that you'll ever get: not even a gift, just a loan is required. If you want to discuss a deal, don't do it here, but mail me at mitchtemporarily@hotmail.com. One person might be enough to make the difference.

[*]Really, I can't say that, that's an emotional statement. There has been lots of assistance, large and small, from people in my life. But it's been a struggle conducted at subsistence level the whole way.

 

ETA 6 Feb: I get to keep going.

State your physical account of experienced color

-1 Mitchell_Porter 01 February 2012 07:00AM

Previous post: Does functionalism imply dualism? Next post: One last roll of the dice.

Don't worry, this sequence of increasingly annoying posts is almost over. But I think it's desirable that we try to establish, once and for all, how people here think color works, and whether they even think it exists.

The way I see it, there is a mental block at work. An obvious fact is being denied or evaded, because the conclusions are unpalatable. The obvious fact is that physics as we know it does not contain the colors that we see. By "physics" I don't just mean the entities that physicists talk about, I also mean anything that you can make out of them. I would encourage anyone who thinks they know what I mean, and who agrees with me on this point, to speak up and make it known that they agree. I don't mind being alone in this opinion, if that's how it is, but I think it's desirable to get some idea of whether LessWrong is genuinely 100% against the proposition.

Just so we're all on the same wavelength, I'll point to a specific example of color. Up at the top of this web page, the word "Less" appears. It's green. So, there is an example of a colored entity, right in front of anyone reading this page.

My thesis is that if you take a lot of point-particles, with no property except their location, and arrange them any way you want, there won't be anything that's green like that; and that the same applies for any physical theory with an ontology that doesn't explicitly include color. To me, this is just mindbogglingly obvious, like the fact that you can't get a letter by adding numbers.

At this point people start talking about neurons and gensyms and concept maps. The greenness isn't in the physical object, "computer screen", it's in the brain's response to the stimulus provided by light from the computer screen entering the eye.

My response is simple. Try to fix in your mind what the physical reality must be, behind your favorite neuro-cognitive explanation of greenness. Presumably it's something like "a whole lot of neurons, firing in a particular way". Try to imagine what that is physically, in terms of atoms. Imagine some vast molecular tinker-toy structures, shaped into a cluster of neurons, with traveling waves of ions crossing axonal membranes. Large numbers of atoms arranged in space, a few of them executing motions which are relevant for the information processing. Do you have that in your mind's eye? Now look up again at that word "Less", and remind yourself that according to your theory, the green shape that you are seeing is the same thing as some aspect of all those billions of colorless atoms in motion.

If your theory still makes sense to you, then please tell us in comments what aspect of the atoms in motion is actually green.

I only see three options. Deny that anything is actually green; become a dualist; or (supervillain voice) join me, and together, we can make a new ontology.

Does functionalism imply dualism?

-1 Mitchell_Porter 31 January 2012 03:43AM

This post follows on from Personal research update, and is followed by State your physical explanation of experienced color.

In a recent post, I claimed that functionalism about consciousness implies dualism. Since most functionalists think their philosophy is an alternative to dualism, I'd better present an argument.

But before I go further, I'll link to orthonormal's series on dissolving the problem of "Mary's Room": Seeing Red: Dissolving Mary's Room and Qualia, A Study of Scarlet: The Conscious Mental Graph, Nature: Red in Truth, and Qualia. Mary's Room is one of many thought experiments bandied about by philosophers in their attempts to say whether or not colors (and other qualia) are a problem for materialism, and orthonormal presents a computational attempt to get around the problem which is a good representative of the functionalist style of thought. I won't have anything to say about those articles at this stage (maybe in comments), but they can serve as an example of what I'm talking about. 

Now, though it may antagonize some people, I think it is best to start off by stating my position plainly and bluntly, rather than starting with a neutral discussion of what functionalism is and how it works, and then seeking to work my way from there to the unpopular conclusion. I will stick to the example of color to make my points - apologies to blind and colorblind readers.

My fundamental thesis is that color manifestly does exist - there are such things as shades of green, shades of red, etc - and that it manifestly does not exist in any standard sort of physical ontology. In an arrangement of point particles in space, there are no shades of green present. This is obviously true, and it's equally obvious for more complicated ontologies like fields, geometries, wavefunction multiverses, and so on. It's even part of the history of physics; even Galileo distinguished between primary qualities like location and shape, and secondary qualities like color. Primary qualities are out there and objectively present in the external world, secondary qualities are only in us, and physics will only concern itself with primary qualities. The ontological world of physical theory is colorless. (We may call light of a certain wavelength green light or red light, but that is because it produces an experience of seeing green or seeing red, not because the light itself is green or red in the original sense of those words.) And what has happened due to the progress of the natural sciences is that we now say that experiences are in brains, and brains are made of atoms, and atoms are described by a physics which does not contain color. So the secondary qualities have vanished entirely from this picture of the world; there is no opportunity for them to exist within us, because we are made of exactly the same stuff as the external world.

Yet the "secondary qualities" are there. They're all around us, in every experience. It really is this simple: colors exist in reality, they don't exist in theory, therefore the theory needs to be augmented or it needs to be changed. Dualism is an augmentation. My speculations about quantum monads are supposed to pave the way for a change. But I won't talk about that option here. Instead, I will try to talk about theories of consciousness which are meant to be compatible with physicalism - functionalism is one such theory.

Such a theory will necessarily present a candidate, however vague, for the physical correlate of an experience of color. One can then say that color exists without having to add anything to physics, because the color just is the proposed physical correlate. This doesn't work because the situation hasn't changed. If all you have are point particles whose only property is location, then individual particles do not have the property of being colored, nor do they have that property in conjunction. Identifying a physical correlate simply picks out a particular set of particles and says "there's your experience of color". But there's still nothing there that is green or red. You may accustom yourself to thinking of a particular material event, a particular rearrangement of atoms in space, as being the color, but that's just the power of habitual association at work. You are introducing into your concept of the event a property that is not inherently present in it.

It may be that one way people manage to avoid noticing this, is by an incomplete chain of thought. I might say: none of the objects in your physical theory are green. The happy materialist might say: but those aren't the things which are truly green in the sense you care about; the things which are green are parts of experiences, not the external objects. I say: fine. But experiences have to exist, right? And you say that physics is everything. So that must mean that experiences are some sort of physical object, and so it will be just as impossible for them to be truly green, given the ontological primitives we have to work with. But for some reason, this further deduction isn't made. Instead, it is accepted that objects in physical space aren't really green, but the objects of experience exist in some other "space", the space of subjective experience, and... it isn't explicitly said that objects there can be truly green, but somehow this difference between physical space and subjective space seems to help people be dualists without actually noticing it.

It is true that color exists in this context - a subjective space. Color always exists as part of an "experience". But physical ontology doesn't contain subjective space or conscious experience any more than it does contain color. What it can contain, are state machines which are structurally isomorphic to these things. So here we can finally identify how a functionalist theory of consciousness works psychologically: You single out some state machines in your physical description of the brain (like the networks in orthonormal's sequence of posts); in your imagination, you associate consciousness with certain states of such state machines, on the basis of structural isomorphism; and now you say, conscious states are those physical states. Subjective space is some neural topographic map, the subjectively experienced body is the sensorimotor homunculus, and so forth.

But if we stick to any standard notion of physical theory, all those brain parts still don't have any of the properties they need. There's no color there, there's no other space there, there's no observing agent. It's all just large numbers of atoms in motion. No-one is home and nothing is happening to them.

Clearly it is some sort of progress to have discovered, in one's physical picture of the world, the possibility of entities which are roughly isomorphic to experiences, colors, etc. But they are still not the same thing. Most of the modern turmoil of ideas about consciousness in philosophy and science is due to this gap - attempts to deny it, attempts to do without noticing it, attempts to force people to notice it. orthonormal's sequence, for example, seems to be an attempt to exhibit a cognitive model for experiences and behaviors that you would expect if color exists, without having to suppose that color actually exists. If we were talking about a theoretical construct, this would be fine. We are under no obligation to believe that phlogiston exists, only to explain why people once talked about it.

But to extend this attitude to something that most of us are directly experiencing in almost every waking moment, is ... how can I put this? It's really something. I'd call it an act of intellectual desperation, except that people don't seem to feel desperate when they do it. They are just patiently explaining, recapitulating and elaborating, some "aha" moment they had back in their past, when functionalism made sense to them. My thesis is certainly that this sense of insight, of having dissolved the problem, is an illusion. The genuineness of the isomorphism between conscious state and coarse-grained physical state, and the work of several generations of materialist thinkers to develop ways of speaking which smoothly promote this isomorphism to an identity, combine to provide the sense that no problem remains to be solved. But all you have to do is attend for a moment to experience itself, and then to compare that to the picture of billions of colorless atoms in intricate motion through space, to realize that this is still dualism.

I promised not to promote the monads, but I will say this. The way to avoid dualism is to first understand consciousness as it is in itself, without the presupposition of materialism. Observe the structure of its states and the dynamics of its passage. That is what phenomenology is about. Then, sketch out an ontology of what you have observed. It doesn't have to contain everything in infinite detail, it can overlook some features. But I would say that at a minimum it needs to contain the triad of subject-object-aspect (which appears under various names in the history of philosophy). There are objects of awareness, they are being experienced within a common subjective space, and they are experienced in a certain aspect. Any theory of reality, whether or not it is materialist, must contain such an entity in order to be true.

The basic entity here is the experiencing subject. Conscious states are its states. And now we can begin to tackle the ontological status of state machines, as a candidate for the ontological category to which conscious beings belong.

State machines are abstracted descriptions. We say there's a thing, it has a set of possible states; here are the allowed transitions between them, and the conditions under which those transitions occur. Specify all that and we have specified a state machine. We don't care about why those are the states or why the transitions occur; those are irrelevant details.

A very simple state machine might be denoted by the state transition network "1<->2". There's a state labeled 1 and another state labeled 2. If the machine is in state 1, it proceeds to state 2, and the reverse is also true. This state machine is realized wherever you have something that oscillates between two states without stopping in either. First the earth is close to the sun, then it is far from the sun, then it is close again... The Earth in its orbit instantiates the state machine "1<->2". I get involved with Less Wrong, then I quit for a while, then I come back... My Internet habits also instantiate the state machine "1<->2".

A computer program is exactly like this, a state machine of great complexity (and usually its state transition rules contain some dependence on external conditions, like user input) which has been physically instantiated for use. But one cannot claim that its states have any intrinsic meaning, any more than I can claim that the state 1 in the oscillating state machine is intrinsically about the earth being close to the sun. This is not true, even if I write down the state transition network in the form "CloseToTheSun<->FarFromTheSun".

This is another ontological deficiency of functionalism. Mental states have meanings, thoughts are always about something, and what they are about is not the result of convention or of the needs of external users. This is yet another clue that the ontological status of conscious states is special, that their "substance" matters to what they are. Of course, this is a challenge to the philosophy which says that a detailed enough simulation of a brain will create a conscious person, regardless of the computational substrate. The only reason people believe this, is because they believe the brain itself is not a special substrate. But this is a judgment made on the basis of science that is still at a highly incomplete stage, and certainly I expect science to tell us something different by the time it's finished with the brain. The ontological problems of functionalism provide a strong apriori reason for this expectation.

What is more challenging is to form a conception of the elementary parts and relations that could form the basis of an alternative ontology. But we have to do this, and the impetus has to come from a phenomenological ontology of consciousness that is as precise as possible. Fortunately, a great start was made on this about 100 years ago, in the heyday of phenomenology as a philosophical movement.

A conscious mind is a state machine, in the sense that it has states and transitions between them. The states also have structure, because conscious experiences do have parts. But the ontological ties that combine those parts into the whole are poorly apprehended by our current concepts. When we try to reduce them to nothing but causal coupling or to the proximity in space of presumed physical correlates of those parts, we are, I believe, getting it wrong. Clearly cause and effect operates in the realm of consciousness, but it will take great care to state precisely and correctly the nature of the things which are interacting and the ways in which they do so. Consider the ability to tell apart different shades of color. It's not just that the colors are there; we know that they are there, and we are able to tell them apart. This implies a certain amount of causal structure. But the perilous step is to focus only on that causal structure, detach it from considerations of how things appear to be in themselves, and instead say "state machine, neurons doing computations, details interesting but not crucial to my understanding of reality". Somehow, in trying to understand conscious cognition, we must remain in touch with the ontology of consciousness as partially revealed in consciousness itself. The things which do the conscious computing must be things with the properties that we see in front of us, the properties of the objects of experience, such as color.

You know, color - authentic original color - has been banished from physical ontology for so long, that it sounds a little mad to say that there might be a physical entity which is actually green. But there has to be such an entity, whether or not you call it physical. Such an entity will always be embedded in a larger conscious experience, and that conscious experience will be embedded in a conscious being, like you. So we have plenty of clues to the true ontology; the clues are right in front of us; we're subjectively made of these clues. And we will not truly figure things out, unless we remain insistent that these inconvenient realities are in fact real.

Personal research update

4 Mitchell_Porter 29 January 2012 09:32AM

Synopsis: The brain is a quantum computer and the self is a tensor factor in it - or at least, the truth lies more in that direction than in the classical direction - and we won't get Friendly AI right unless we get the ontology of consciousness right.

Followed by: Does functionalism imply dualism?

Sixteen months ago, I made a post seeking funding for personal research. There was no separate Discussion forum then, and the post was comprehensively downvoted. I did manage to keep going at it, full-time, for the next sixteen months. Perhaps I'll get to continue; it's for the sake of that possibility that I'll risk another breach of etiquette. You never know who's reading these words and what resources they have. Also, there has been progress.

I think the best place to start is with what orthonormal said in response to the original post: "I don't think anyone should be funding a Penrose-esque qualia mysterian to study string theory." If I now took my full agenda to someone out in the real world, they might say: "I don't think it's worth funding a study of 'the ontological problem of consciousness in the context of Friendly AI'." That's my dilemma. The pure scientists who might be interested in basic conceptual progress are not engaged with the race towards technological singularity, and the apocalyptic AI activists gathered in this place are trying to fit consciousness into an ontology that doesn't have room for it. In the end, if I have to choose between working on conventional topics in Friendly AI, and on the ontology of quantum mind theories, then I have to choose the latter, because we need to get the ontology of consciousness right, and it's possible that a breakthrough could occur in the world outside the FAI-aware subculture and filter through; but as things stand, the truth about consciousness would never be discovered by employing the methods and assumptions that prevail inside the FAI subculture.

Perhaps I should pause to spell out why the nature of consciousness matters for Friendly AI. The reason is that the value system of a Friendly AI must make reference to certain states of conscious beings - e.g. "pain is bad" - so, in order to make correct judgments in real life, at a minimum it must be able to tell which entities are people and which are not. Is an AI a person? Is a digital copy of a human person, itself a person? Is a human body with a completely prosthetic brain still a person?

I see two ways in which people concerned with FAI hope to answer such questions. One is simply to arrive at the right computational, functionalist definition of personhood. That is, we assume the paradigm according to which the mind is a computational state machine inhabiting the brain, with states that are coarse-grainings (equivalence classes) of exact microphysical states. Another physical system which admits the same coarse-graining - which embodies the same state machine at some macroscopic level, even though the microscopic details of its causality are different - is said to embody another instance of the same mind.

An example of the other way to approach this question is the idea of simulating a group of consciousness theorists for 500 subjective years, until they arrive at a consensus on the nature of consciousness. I think it's rather unlikely that anyone will ever get to solve FAI-relevant problems in that way. The level of software and hardware power implied by the capacity to do reliable whole-brain simulations means you're already on the threshold of singularity: if you can simulate whole brains, you can simulate part brains, and you can also modify the parts, optimize them with genetic algorithms, and put them together into nonhuman AI. Uploads won't come first.

But the idea of explaining consciousness this way, by simulating Daniel Dennett and David Chalmers until they agree, is just a cartoon version of similar but more subtle methods. What these methods have in common is that they propose to outsource the problem to a computational process using input from cognitive neuroscience. Simulating a whole human being and asking it questions is an extreme example of this (the simulation is the "computational process", and the brain scan it uses as a model is the "input from cognitive neuroscience"). A more subtle method is to have your baby AI act as an artificial neuroscientist, use its streamlined general-purpose problem-solving algorithms to make a causal model of a generic human brain, and then to somehow extract from that, the criteria which the human brain uses to identify the correct scope of the concept "person". It's similar to the idea of extrapolated volition, except that we're just extrapolating concepts.

It might sound a lot simpler to just get human neuroscientists to solve these questions. Humans may be individually unreliable, but they have lots of cognitive tricks - heuristics - and they are capable of agreeing that something is verifiably true, once one of them does stumble on the truth. The main reason one would even consider the extra complication involved in figuring out how to turn a general-purpose seed AI into an artificial neuroscientist, capable of extracting the essence of the human decision-making cognitive architecture and then reflectively idealizing it according to its own inherent criteria, is shortage of time: one wishes to develop friendly AI before someone else inadvertently develops unfriendly AI. If we stumble into a situation where a powerful self-enhancing algorithm with arbitrary utility function has been discovered, it would be desirable to have, ready to go, a schema for the discovery of a friendly utility function via such computational outsourcing.

Now, jumping ahead to a later stage of the argument, I argue that it is extremely likely that distinctively quantum processes play a fundamental role in conscious cognition, because the model of thought as distributed classical computation actually leads to an outlandish sort of dualism. If we don't concern ourselves with the merits of my argument for the moment, and just ask whether an AI neuroscientist might somehow overlook the existence of this alleged secret ingredient of the mind, in the course of its studies, I do think it's possible. The obvious noninvasive way to form state-machine models of human brains is to repeatedly scan them at maximum resolution using fMRI, and to form state-machine models of the individual voxels on the basis of this data, and then to couple these voxel-models to produce a state-machine model of the whole brain. This is a modeling protocol which assumes that everything which matters is physically localized at the voxel scale or smaller. Essentially we are asking, is it possible to mistake a quantum computer for a classical computer by performing this sort of analysis? The answer is definitely yes if the analytic process intrinsically assumes that the object under study is a classical computer. If I try to fit a set of points with a line, there will always be a line of best fit, even if the fit is absolutely terrible. So yes, one really can describe a protocol for AI neuroscience which would be unable to discover that the brain is quantum in its workings, and which would even produce a specific classical model on the basis of which it could then attempt conceptual and volitional extrapolation.

Clearly you can try to circumvent comparably wrong outcomes, by adding reality checks and second opinions to your protocol for FAI development. At a more down to earth level, these exact mistakes could also be made by human neuroscientists, for the exact same reasons, so it's not as if we're talking about flaws peculiar to a hypothetical "automated neuroscientist". But I don't want to go on about this forever. I think I've made the point that wrong assumptions and lax verification can lead to FAI failure. The example of mistaking a quantum computer for a classical computer may even have a neat illustrative value. But is it plausible that the brain is actually quantum in any significant way? Even more incredibly, is there really a valid apriori argument against functionalism regarding consciousness - the identification of consciousness with a class of computational process?

I have previously posted (here) about the way that an abstracted conception of reality, coming from scientific theory, can motivate denial that some basic appearance corresponds to reality. A perennial example is time. I hope we all agree that there is such a thing as the appearance of time, the appearance of change, the appearance of time flowing... But on this very site, there are many people who believe that reality is actually timeless, and that all these appearances are only appearances; that reality is fundamentally static, but that some of its fixed moments contain an illusion of dynamism.

The case against functionalism with respect to conscious states is a little more subtle, because it's not being said that consciousness is an illusion; it's just being said that consciousness is some sort of property of computational states. I argue first that this requires dualism, at least with our current physical ontology, because conscious states are replete with constituents not present in physical ontology - for example, the "qualia", an exotic name for very straightforward realities like: the shade of green appearing in the banner of this site, the feeling of the wind on your skin, really every sensation or feeling you ever had. In a world made solely of quantum fields in space, there are no such things; there are just particles and arrangements of particles. The truth of this ought to be especially clear for color, but it applies equally to everything else.

In order that this post should not be overlong, I will not argue at length here for the proposition that functionalism implies dualism, but shall proceed to the second stage of the argument, which does not seem to have appeared even in the philosophy literature. If we are going to suppose that minds and their states correspond solely to combinations of mesoscopic information-processing events like chemical and electrical signals in the brain, then there must be a mapping from possible exact microphysical states of the brain, to the corresponding mental states. Supposing we have a mapping from mental states to coarse-grained computational states, we now need a further mapping from computational states to exact microphysical states. There will of course be borderline cases. Functional states are identified by their causal roles, and there will be microphysical states which do not stably and reliably produce one output behavior or the other.

Physicists are used to talking about thermodynamic quantities like pressure and temperature as if they have an independent reality, but objectively they are just nicely behaved averages. The fundamental reality consists of innumerable particles bouncing off each other; one does not need, and one has no evidence for, the existence of a separate entity, "pressure", which exists in parallel to the detailed microphysical reality. The idea is somewhat absurd.

Yet this is analogous to the picture implied by a computational philosophy of mind (such as functionalism) applied to an atomistic physical ontology. We do know that the entities which constitute consciousness - the perceptions, thoughts, memories... which make up an experience - actually exist, and I claim it is also clear that they do not exist in any standard physical ontology. So, unless we get a very different physical ontology, we must resort to dualism. The mental entities become, inescapably, a new category of beings, distinct from those in physics, but systematically correlated with them. Except that, if they are being correlated with coarse-grained neurocomputational states which do not have an exact microphysical definition, only a functional definition, then the mental part of the new combined ontology is fatally vague. It is impossible for fundamental reality to be objectively vague; vagueness is a property of a concept or a definition, a sign that it is incomplete or that it does not need to be exact. But reality itself is necessarily exact - it is something - and so functionalist dualism cannot be true unless the underdetermination of the psychophysical correspondence is replaced by something which says for all possible physical states, exactly what mental states (if any) should also exist. And that inherently runs against the functionalist approach to mind.

Very few people consider themselves functionalists and dualists. Most functionalists think of themselves as materialists, and materialism is a monism. What I have argued is that functionalism, the existence of consciousness, and the existence of microphysical details as the fundamental physical reality, together imply a peculiar form of dualism in which microphysical states which are borderline cases with respect to functional roles must all nonetheless be assigned to precisely one computational state or the other, even if no principle tells you how to perform such an assignment. The dualist will have to suppose that an exact but arbitrary border exists in state space, between the equivalence classes.

This - not just dualism, but a dualism that is necessarily arbitrary in its fine details - is too much for me. If you want to go all Occam-Kolmogorov-Solomonoff about it, you can say that the information needed to specify those boundaries in state space is so great as to render this whole class of theories of consciousness not worth considering. Fortunately there is an alternative.

Here, in addressing this audience, I may need to undo a little of what you may think you know about quantum mechanics. Of course, the local preference is for the Many Worlds interpretation, and we've had that discussion many times. One reason Many Worlds has a grip on the imagination is that it looks easy to imagine. Back when there was just one world, we thought of it as particles arranged in space; now we have many worlds, dizzying in their number and diversity, but each individual world still consists of just particles arranged in space. I'm sure that's how many people think of it.

Among physicists it will be different. Physicists will have some idea of what a wavefunction is, what an operator algebra of observables is, they may even know about path integrals and the various arcane constructions employed in quantum field theory. Possibly they will understand that the Copenhagen interpretation is not about consciousness collapsing an actually existing wavefunction; it is a positivistic rationale for focusing only on measurements and not worrying about what happens in between. And perhaps we can all agree that this is inadequate, as a final description of reality. What I want to say, is that Many Worlds serves the same purpose in many physicists' minds, but is equally inadequate, though from the opposite direction. Copenhagen says the observables are real but goes misty about unmeasured reality. Many Worlds says the wavefunction is real, but goes misty about exactly how it connects to observed reality. My most frustrating discussions on this topic are with physicists who are happy to be vague about what a "world" is. It's really not so different to Copenhagen positivism, except that where Copenhagen says "we only ever see measurements, what's the problem?", Many Worlds says "I say there's an independent reality, what else is left to do?". It is very rare for a Many World theorist to seek an exact idea of what a world is, as you see Robin Hanson and maybe Eliezer Yudkowsky doing; in that regard, reading the Sequences on this site will give you an unrepresentative idea of the interpretation's status.

One of the characteristic features of quantum mechanics is entanglement. But both Copenhagen, and a Many Worlds which ontologically privileges the position basis (arrangements of particles in space), still have atomistic ontologies of the sort which will produce the "arbitrary dualism" I just described. Why not seek a quantum ontology in which there are complex natural unities - fundamental objects which aren't simple - in the form of what we would presently called entangled states? That was the motivation for the quantum monadology described in my other really unpopular post. :-) [Edit: Go there for a discussion of "the mind as tensor factor", mentioned at the start of this post.] Instead of saying that physical reality is a series of transitions from one arrangement of particles to the next, say it's a series of transitions from one set of entangled states to the next. Quantum mechanics does not tell us which basis, if any, is ontologically preferred. Reality as a series of transitions between overall wavefunctions which are partly factorized and partly still entangled is a possible ontology; hopefully readers who really are quantum physicists will get the gist of what I'm talking about.

I'm going to double back here and revisit the topic of how the world seems to look. Hopefully we agree, not just that there is an appearance of time flowing, but also an appearance of a self. Here I want to argue just for the bare minimum - that a moment's conscious experience consists of a set of things, events, situations... which are simultaneously "present to" or "in the awareness of" something - a conscious being - you. I'll argue for this because even this bare minimum is not acknowledged by existing materialist attempts to explain consciousness. I was recently directed to this brief talk about the idea that there's no "real you". We are given a picture of a graph whose nodes are memories, dispositions, etc., and we are told that the self is like that graph: nodes can be added, nodes can be removed, it's a purely relational composite without any persistent part. What's missing in that description is that bare minimum notion of a perceiving self. Conscious experience consists of a subject perceiving objects in certain aspects. Philosophers have discussed for centuries how best to characterize the details of this phenomenological ontology; I think the best was Edmund Husserl, and I expect his work to be extremely important in interpreting consciousness in terms of a new physical ontology. But if you can't even notice that there's an observer there, observing all those parts, then you won't get very far.

My favorite slogan for this is due to the other Jaynes, Julian Jaynes. I don't endorse his theory of consciousness at all; but while in a daydream he once said to himself, "Include the knower in the known". That sums it up perfectly. We know there is a "knower", an experiencing subject. We know this, just as well as we know that reality exists and that time passes. The adoption of ontologies in which these aspects of reality are regarded as unreal, as appearances as only, may be motivated by science, but it's false to the most basic facts there are, and one should show a little more imagination about what science will say when it's more advanced.

I think I've said almost all of this before. The high point of the argument is that we should look for a physical ontology in which a self exists and is a natural yet complex unity, rather than a vaguely bounded conglomerate of distinct information-processing events, because the latter leads to one of those unacceptably arbitrary dualisms. If we can find a physical ontology in which the conscious self can be identified directly with a class of object posited by the theory, we can even get away from dualism, because physical theories are mathematical and formal and make few commitments about the "inherent qualities" of things, just about their causal interactions. If we can find a physical object which is absolutely isomorphic to a conscious self, then we can turn the isomorphism into an identity, and the dualism goes away. We can't do that with a functionalist theory of consciousness, because it's a many-to-one mapping between physical and mental, not an isomorphism.

So, I've said it all before; what's new? What have I accomplished during these last sixteen months? Mostly, I learned a lot of physics. I did not originally intend to get into the details of particle physics - I thought I'd just study the ontology of, say, string theory, and then use that to think about the problem. But one thing led to another, and in particular I made progress by taking ideas that were slightly on the fringe, and trying to embed them within an orthodox framework. It was a great way to learn, and some of those fringe ideas may even turn out to be correct. It's now abundantly clear to me that I really could become a career physicist, working specifically on fundamental theory. I might even have to do that, it may be the best option for a day job. But what it means for the investigations detailed in this essay, is that I don't need to skip over any details of the fundamental physics. I'll be concerned with many-body interactions of biopolymer electrons in vivo, not particles in a collider, but an electron is still an electron, an elementary particle, and if I hope to identify the conscious state of the quantum self with certain special states from a many-electron Hilbert space, I should want to understand that Hilbert space in the deepest way available.

My only peer-reviewed publication, from many years ago, picked out pathways in the microtubule which, we speculated, might be suitable for mobile electrons. I had nothing to do with noticing those pathways; my contribution was the speculation about what sort of physical processes such pathways might underpin. Something I did notice, but never wrote about, was the unusual similarity (so I thought) between the microtubule's structure, and a model of quantum computation due to the topologist Michael Freedman: a hexagonal lattice of qubits, in which entanglement is protected against decoherence by being encoded in topological degrees of freedom. It seems clear that performing an ontological analysis of a topologically protected coherent quantum system, in the context of some comprehensive ontology ("interpretation") of quantum mechanics, is a good idea. I'm not claiming to know, by the way, that the microtubule is the locus of quantum consciousness; there are a number of possibilities; but the microtubule has been studied for many years now and there's a big literature of models... a few of which might even have biophysical plausibility.

As for the interpretation of quantum mechanics itself, these developments are highly technical, but revolutionary. A well-known, well-studied quantum field theory turns out to have a bizarre new nonlocal formulation in which collections of particles seem to be replaced by polytopes in twistor space. Methods pioneered via purely mathematical studies of this theory are already being used for real-world calculations in QCD (the theory of quarks and gluons), and I expect this new ontology of "reality as a complex of twistor polytopes" to carry across as well. I don't know which quantum interpretation will win the battle now, but this is new information, of utterly fundamental significance. It is precisely the sort of altered holistic viewpoint that I was groping towards when I spoke about quantum monads constituted by entanglement. So I think things are looking good, just on the pure physics side. The real job remains to show that there's such a thing as quantum neurobiology, and to connect it to something like Husserlian transcendental phenomenology of the self via the new quantum formalism.

It's when we reach a level of understanding like that, that we will truly be ready to tackle the relationship between consciousness and the new world of intelligent autonomous computation. I don't deny the enormous helpfulness of the computational perspective in understanding unconscious "thought" and information processing. And even conscious states are still states, so you can surely make a state-machine model of the causality of a conscious being. It's just that the reality of how consciousness, computation, and fundamental ontology are connected, is bound to be a whole lot deeper than just a stack of virtual machines in the brain. We will have to fight our way to a new perspective which subsumes and transcends the computational picture of reality as a set of causally coupled black-box state machines. It should still be possible to "port" most of the thinking about Friendly AI to this new ontology; but the differences, what's new, are liable to be crucial to success. Fortunately, it seems that new perspectives are still possible; we haven't reached Kantian cognitive closure, with no more ontological progress open to us. On the contrary, there are still lines of investigation that we've hardly begun to follow.

"Personal Identity and Uploading", by Mark Walker

6 gwern 07 January 2012 07:55PM

“Personal Identity and Uploading”, Mark Walker is the next JET paper. Abstract:

Objections to uploading may be parsed into substrate issues, dealing with the computer platform of upload and personal identity. This paper argues that the personal identity issues of uploading are no more or less challenging than those of bodily transfer often discussed in the philosophical literature. It is argued that what is important in personal identity involves both token and type identity. While uploading does not preserve token identity, it does save type identity; and even qua token, one may have good reason to think that the preservation of the type is worth the cost.

continue reading »

"Misbehaving Machines: The Emulated Brains of Transhumanist Dreams", Corry Shores

9 gwern 29 December 2011 10:33PM

“Misbehaving Machines: The Emulated Brains of Transhumanist Dreams”, by Corry Shores (grad student; Twitter, blog) is another recent JET paper. Abstract:

Enhancement technologies may someday grant us capacities far beyond what we now consider humanly possible. Nick Bostrom and Anders Sandberg suggest that we might survive the deaths of our physical bodies by living as computer emulations.­­ In 2008, they issued a report, or “roadmap,” from a conference where experts in all relevant fields collaborated to determine the path to “whole brain emulation.” Advancing this technology could also aid philosophical research. Their “roadmap” defends certain philosophical assumptions required for this technology’s success, so by determining the reasons why it succeeds or fails, we can obtain empirical data for philosophical debates regarding our mind and selfhood. The scope ranges widely, so I merely survey some possibilities, namely, I argue that this technology could help us determine

  1. if the mind is an emergent phenomenon,
  2. if analog technology is necessary for brain emulation, and
  3. if neural randomness is so wild that a complete emulation is impossible.

continue reading »

Inverse p-zombies: the other direction in the Hard Problem of Consciousness

17 gwern 18 December 2011 09:32PM

402. "Nothing is so certain as that I possess consciousness." In that case, why shouldn't I let the matter rest? This certainty is like a mighty force whose point of application does not move, and so no work is accomplished by it.

403. Remember: most people say one feels nothing under anaesthetic. But some say: It could be that one feels, and simply forgets it completely.

--Wittgenstein, Zettel (1929-1948)

I offer for LW's consideration the interesting 2008 paper "Inverse zombies, anesthesia awareness, and the hard problem of unconsciousness" (Mashour & LaRock; NCBI); the abstract:

Philosophical (p-) zombies are constructs that possess all of the behavioral features and responses of a sentient human being, yet are not conscious. P-zombies are intimately linked to the hard problem of consciousness and have been invoked as arguments against physicalist approaches. But what if we were to invert the characteristics of p-zombies? Such an inverse (i-) zombie would possess all of the behavioral features and responses of an insensate being, yet would nonetheless be conscious. While p-zombies are logically possible but naturally improbable, an approximation of i-zombies actually exists: individuals experiencing what is referred to as "anesthesia awareness." Patients under general anesthesia may be intubated (preventing speech), paralyzed (preventing movement), and narcotized (minimizing response to nociceptive stimuli). Thus, they appear--and typically are--unconscious. In 1-2 cases/1000, however, patients may be aware of intraoperative events, sometimes without any objective indices. Furthermore, a much higher percentage of patients (22% in a recent study) may have the subjective experience of dreaming during general anesthesia. P-zombies confront us with the hard problem of consciousness--how do we explain the presence of qualia? I-zombies present a more practical problem--how do we detect the presence of qualia? The current investigation compares p-zombies to i-zombies and explores the "hard problem" of unconsciousness with a focus on anesthesia awareness.

continue reading »

How to detonate a technology singularity using only parrot level intelligence - new meetup.com group in Silicon Valley to design and create it

-15 BenRayfield 31 July 2011 06:27PM

 

http://www.meetup.com/technology-singularity-detonator

9 people joined in the last 5 hours and the first meetup hasn't even happened yet. This is the meetup description, including technical designs and how it leads to singularity:

 

The plan is to detonate an intelligence explosion (leading to a technology-singularity) starting with an open-source Java artificial intelligence (AI) software which networks peoples' minds together through the internet using realtime interactive psychology of feedback loops between mouse movements and generated audio. "Technological singularity refers to the hypothetical future emergence of greater-than human intelligence through technological means." http://en.wikipedia.org/wiki/Technological_singularity Computer programming is not required to join the group, but some kind of technical or abstract thinking skill is. We are going to make this happen, not talk about it endlessly like so many other AI groups do. Audivolv 0.1.7 is a very early and version of the user-interface. The final version will be a massively multiplayer audio game unlike any existing game. It will learn based on mouse movements in realtime instead of requiring good/bad buttons to train it. The core AI systems have not been created yet. Audivolv is just the user-interface for that. http://sourceforge.net/projects/audivolv The whole system will be 1 file you double-click to run and it works immediately on Windows, Mac, or Linux. This does not include Audivolv yet and has some parts that may be removed: http://sourceforge.net/projects/humanainet It must be a "Friendly AI", which means it will be designed not to happen like in the Terminator movies or similar science fiction. It will work toward more productive goals and help the Human species. http://en.wikipedia.org/wiki/Friendly_artificial_intelligence My plan to make that happen is for it to be made of many peoples' minds and many computers, so it is us. It becomes smarter when we become smarter. One of the effects of that will be to extremely increase Dunbar's Number, which is the number of people or organizations that a person can intelligently interact with before forgetting others. Dunbar's number is estimated around 150 today. http://en.wikipedia.org/wiki/Dunbar%27s_number

 

This only requires the AI be as smart as a parrot, since the people using the program do most of the thinking and the AI only organizes their thoughts statistically enough to decide who should connect to who else, in the way evolved code is traded (and verified to use only math so its safe) between computers automatically, in this massively multiplayer audio game. We will detonate a technology singularity using only the intelligence of a parrot plus the intelligence of people using the program. This is very surprising to most people who think huge grids of computers and experts are required to build Human intelligence in a machine. This is a shortcut, and will have much better results because it is us so it has no reason to act against us, like an AI made only of software may do.

 

Infrastructure

Communication between these programs through the internet will be done as a Distributed Hash Table. The most important part of that is each key (hash of some file bytes) has a well-defined distance to each other key, a distance(hash1,hash2) function, which proves the correct direction to search the network to find the bytes of any hash, or to statistically verify (but not certainly) that its not in the network. There may be a way to do it certainly, but for my purposes approximate searching will work.

In the same Distributed Hash Table, there will be public-keys, used like filenames or identities, whose content can be modified only by whoever has the private-key. If code evolves to include calculations based on your mouse movements and the mouse movements of 5 other people in realtime, then the numbers from those other mouse movements (between -1 and 1 for each of 2 dimensions, for each of 5 people) will be digitally-signed so everyone who uses the evolved code will know it is using the same people's continuing mouse movements instead of is a modified code. The code can be modified, but that would have a different hash and would be considered on its own merits instead of knowledge about the previous code and its specific connections to specific people. This will be done in realtime, not something to be saved and loaded later from a hard-drive. Each new mouse position (or a few of them sent at once) will be digitally-signed and broadcast to the network, the same as any other data broadcast to the network.

http://en.wikipedia.org/wiki/Distributed_hash_table

Similarly, but more fuzzy, the psychology of feedback loops between mouse movements and automatically evolving Java code, will be used as a distance function, and a second network organized that way, so you can search the network in the direction of other people whose psychology is more similar to your current state of mind and how you're using the program. This decentralized network will be searchable by your subconscious thoughts, because subconscious thoughts are expressed in how your mouse movements cause the code to evolve.

As you search this network automatically by moving your mouse, you will trade evolved code with those computers, always automatically verifying the code only uses math and no file-access or java.lang.System class or anything else not provably safe. You will experience the downloaded code as it gradually connects to the code evolved for your mouse movements, code which generates audio as 44100 audio amplitudes (number between -1 and 1) per second per speaker.

Some of the variables in the evolved code will be the hash of other evolved code. Each evolved code will have a hash, probably from the SHA-256 algorithm, so it could be a length 64 hex string written in the code. Each variable will be a number beween -1 and 1. No computer will have all the codes for all its variables, but for those it doesn't have, it will use them simply as a variable. If it has those codes, then there is an extra behavior of giving that code an amount of influence proportional to the value of the variable, or deleting the code if the variable becomes negative for too long. In that way, evolved code will decide which other evolved code to download and how much influence each evolved code should have on the array of floating point numbers in the local computer.

Since the decentralized network will be searched by psychology (instead of text or pixels in an image or other things search-engines know how to do today), and since its connected to each person's subconscious mind through mouse/music feedback loops, the effect will be a collective mind made of many people and computers. We are Human AI Net, do you want to be temporarily assimilated?

 

Alternative To Brain Implants 

Statistically inputs and outputs to neurons subconsciously without extra hardware. 

A neuron is a brain cell that connects to thousands of other neurons and slowly adjusts its electricity and chemical patterns as it learns. 

An incorrect assumption has extremely delayed the creation of technology that transfers thoughts between 2 brains. That assumption is, to quickly transfer large amounts of information between a brain and a computer, you need hardware that connects directly to neurons. 

Eyes and ears transfer a lot of information to a brain, but the other part of that assumption is eyes and ears are only useful for pictures and sounds that make sense and do not appear as complete randomness or whitenoise. People assume anything that sounds like radio static (a typical random sound) can't be used to transfer useful information into a brain. 

Most of us remember what a dial-up-modem sounds like. It sounds like information is in it but its too fast for Humans to understand. That's true of the dial-up-modem sound only because its digital and is designed for a modem instead of for Human ears which can hear around 1500 tones and simultaneously a volume for each. The dial-up-modem can only hear 1 tone that oscillates between 1 and 0, and no volume, just 1 or 0. It gets 56000 of those 1s and 0s per second. Human ears are analog so they have no such limits, but brains can think at most at 100 changes per second. 

If volume can have 20 different values per tone, then Human ears can hear up to 1500*100*log_base_2(20)=650000 bits of information per second. If you could take full advantage of that speed, you could transfer a book every few seconds into your brain, but the next bottleneck is your ability to think that fast. 

If you use ears the same way dial-up-modems use a phone line, but in a way designed for Human ears and Human brains instead of computers, then your ears are much faster data transfer devices than brain implants, and the same is true for transferring information as random-appearing grids of changing colors through your eyes. We have computer speakers and screens for input to brains. We still have some work to do on the output speeds of mouse and keyboard, but there are electricity devices you can wear on your head for the output direction. For the input direction, eyes and ears are currently far ahead of the most advanced technology in their data speeds to your brain. 

So why do businesses and governments keep throwing huge amounts of money at connecting computer chips directly to neurons? They should learn to use eyes and ears to their full potential before putting so much resources into higher bandwidth connections to brains. They're not nearly using the bandwidth they already have to brains. 

Intuitively most people know how music can affect their subconscious thoughts. Music is a low bandwidth example. It has mostly predictable and repeated sounds. The same voices. The same instruments. What I'm talking about would sound more like radio static or whitenoise. You wouldn't know what information is in it from its sound. You would only understand it after it echoed around your neuron electricity patterns in subconscious ways. 

Most people have only a normal computer available, so the brain-to-computer direction of information flow has to be low bandwidth. It can be mouse movements, gyroscope based game controllers, video camera detecting motion, or devices like that. The computer-to-brain direction can be high bandwidth, able to transfer information faster than you can think about it. 

Why hasn't this been tried? Because science proceeds in small steps. This is a big step from existing technology but a small step in the way most people already have the hardware (screen, speakers, mouse, etc). The big step is going from patterns of random-appearing sounds or video to subconscious thoughts to mouse movements to software to interpret it statistically, and around that loop many times as the Human and computer learn to predict each other. Compared to that, connecting a chip directly to neurons is a small step. 

Its a feedback loop: computer, random-appearing sound or video, ears or eyes, brain, mouse movements, and back to computer. Its very indirect but uses hardware that has evolved for millions of years, compared to low-bandwidth hardware they implant in brains. Eyes and ears are much higher bandwidth, and we should be using them in feedback loops for brain-to-brain and brain-to-computer communication. 

What would it feel like? You would move the mouse and instantly hear the sounds change based on how you moved it. You would feel around the sound space for abstract patterns of information you're looking for, and you would learn to find it. When many people are connected this way through the internet, using only mouse movements and abstract random-like sounds instead of words and pictures, thoughts will flow between the brains of different people, thoughts that they don't know how to put into words. They would gradually learn to think more as 1 mind. Brains naturally learn to communicate with any system connected to them. Brains dont care how they're connected. They grow into a larger mind. It happens between the parts of your brain, and it will happen between people using this system through the internet. 

Artificial intelligence software does not have to replace us or compete with us. The best way to use it is to connect our minds together. It can be done through brain implants, but why wait for that technology to advance and become cheap and safe enough? All you need is a normal computer and the software to connect our subconscious thoughts and statistical patterns of interaction with the computer. 

Dial-up-modem sounds were designed for computers. These interactive sounds/videos would be designed for Human ears/eyes and the slower but much bigger and parallel way the data goes into brains. For years I've been carefully designing a free open-source software http://HumanAI.net  - Human and Artificial Intelligence Network, or Human AI Net - to make this work. It will be a software that does for Human brains what dial-up-modems do for computers, and it will sound a little like a dial-up-modem at first but start to sound like music when you learn how to use it. I don't need brain implants to flow subconscious thoughts between your brains over internet wires. 

Intelligence is the most powerful thing we know of. The brain implants are simply overkill, even if they become advanced enough to do what I'll use software and psychology to do. We can network our minds together and amplify intelligence and share thoughts without extra hardware. After thats working, we can go straight to quantum devices for accessing brains without implants. Lets do this through software and skip the brain implant paradigm. If it works just a little, it will be enough that our combined minds will figure out how to make it work a lot more. Thats how I prefer to start a  http://en.wikipedia.org/wiki/Technological_singularity  We don't need businesses and militaries to do it first. We have the hardware on our desks. We're only missing the software. It doesn't have to be smarter than Human software. It just has to be smart enough to connect our subconscious thoughts together. The authorities have their own ideas about how we should communicate and how our minds should be allowed to think together, but their technology was obsolete before it was created. We can do everything they can do without brain implants, using only software and subconscious psychology. We don't need a smarter-than-Human software, or anything nearly that advanced, to create a technology singularity. Who wants to help me change the direction of Human evolution using an open-source (GNU GPL) software? Really, you can create a technology singularity starting from a software with the intelligence of a parrot, as long as you use it to connect Human minds together.

 

The Nature of Self

3 XiXiDu 05 April 2011 10:52AM

In this post I try to fathom an informal definition of Self, the "essential qualities that constitute a person's uniqueness". I assume that the most important requirement for a definition of self is time-consistency. A reliable definition of identity needs to allow for time-consistent self-referencing since any agent that is unable to identify itself over time will be prone to make inconsistent decisions.

Data Loss

Obviously most humans don't want to die, but what does that mean? What is it that humans try to preserve when they sign up for Cryonics? It seems that an explanation must account and allow for some sort of data loss.

The Continuity of Consciousness

It can't be about the continuity of consciousness as we would have to refuse general anesthesia due to the risk of "dying" and most of us will agree that there is something more important than the continuity of consciousness that makes us accept a general anesthesia when necessary.

Computation

If the continuity of consciousness isn't the most important detail about the self then it very likely isn't the continuity of computation either. Imagine that for some reason the process evoked when "we" act on our inputs under the control of an algorithm halts for a second and then continues otherwise unaffected, would we don't mind to be alive ever after because we died when the computation halted? This doesn't seem to be the case.

Static Algorithmic Descriptions

Although we are not partly software and partly hardware we could, in theory, come up with an algorithmic description of the human machine, of our selfs. Might it be that algorithm that we care about? If we were to digitize our self we would end up with a description of our spatial parts, our self at a certain time. Yet we forget that all of us possess such an algorithmic description of our selfs and we're already able back it up. It is our DNA.

Temporal Parts

Admittedly our DNA is the earliest version of our selfs, but if we don't care about the temporal parts of our selfs but only about a static algorithmic description of a certain spatiotemporal position, then what's wrong with that? It seems a lot, we stop caring about past reifications of our selfs, at some point our backups become obsolete and having to fall back on them would equal death. But what is it that we lost, what information is it that we value more than all of the previously mentioned possibilities? One might think that it must be our memories, the data that represents what we learnt and experienced. But even if this is the case, would it be a reasonable choice?

Indentity and Memory

Let's just disregard the possibility that we often might not value our future selfs and so do not value our past selfs either for that we lost or updated important information, e.g. if we became religious or have been able to overcome religion.

If we had perfect memory and only ever improved upon our past knowledge and experiences we wouldn't be able to do so for very long, at least not given our human body. The upper limit on the information that can be contained within a human body is 2.5072178×1038 megabytes, if it was used as a perfect data storage. Given that we gather much more than 1 megabyte of information per year, it is foreseeable that if we equate our memories with our self we'll die long before the heat death of the universe. We might overcome this by growing in size, by achieving a posthuman form, yet if we in turn also become much smarter we'll also produce and gather more information. We are not alone either and the resources are limited. One way or the other we'll die rather quickly.

Does this mean we shouldn't even bother about the far future or is there maybe something else we value even more than our memories? After all we don't really mind much if we forget what we have done a few years ago.

Time-Consistency and Self-Reference

It seems that there is something even more important than our causal history. I think that more than everything we care about our values and goals. Indeed, we value the preservation of our values. As long as we want the same we are the same. Our goal system seems to be the critical part of our implicit definition of self, that which we want to protect and preserve. Our values and goals seem to be the missing temporal parts that allow us to consistently refer to us, to identify our selfs at different spatiotempiral positions.

Using our values and goals as identifiers also resolves the problem of how we should treat copies of our self that are featuring alternating histories and memories, copies with different causal histories. Any agent that does feature a copy of our utility function ought to be incorporated into our decisions as an instance, as a reification of our selfs. We should identify with our utility-function regardless of its instantiation.

Stable Utility-Functions

To recapitulate, we can value our memories, the continuity of experience and even our DNA, but the only reliable marker for the self identity of goal-oriented agents seems to be a stable utility function. Rational agents with an identical utility function will to some extent converge to exhibit similar behavior and are therefore able to cooperate. We can more consistently identify with our values and goals than with our past and future memories, digitized backups or causal history.

But even if this is true there is one problem, humans might not exhibit goal-stability.

Copying and Subjective Experience

5 lucidfox 20 December 2010 12:14PM

The subject of copying people and its effect on personal identity and probability anticipation has been raised and, I think, addressed adequately on Less Wrong.

Still, I'd like to bring up some more thought experiments.

Recently I had a dispute on an IRC channel. I argued that if some hypothetical machine made an exact copy of me, then I would anticipate a 50% probability of jumping into the new body. (I admit that it still feels a little counterintuitive to me, even though this is what I would rationally expect.) After all, they said, the mere fact the copy was created doesn't affect the original.

However, from an outside perspective, Maia1 would see Maia2 being created in front of her eyes, and Maia2 would see the same scene up to the moment of forking, at which point the field of view in front of her eyes would abruptly change to reflect the new location.

Here, it is obvious from both an inside and outside perspective which version has continuity of experience, and thus from a legal standpoint, I think, it would make sense to regard Maia1 as having the same legal identity as the original, and recognize the need to create new documents and records for Maia2 -- even if there is no physical difference.

Suppose, however, that the information was erased. For example, suppose a robot sedated and copied the original me, then dragged Maia1 and Maia2 to randomly chosen rooms, and erased its own memory. At this point, neither either of me, nor anyone else would be able to distinguish between the two. What would you do here from a legal standpoint? (I suppose if it actually came to this, the two of me would agree to arbitrarily designate one as the original by tossing an ordinary coin...)

And one more moment. What is this probability of subjective body-jump actually a probability of? We could set up various Sleeping Beauty-like thought experiments here. Supposing for the sake of argument that I'll live at most a natural human lifespan no matter which year I find myself in, imagine that I make a backup of my current state and ask a machine to restore a copy of me every 200 years. Does this imply that the moment the backup is made -- before I even issue the order, and from an outside perspective, way before any of this copying happens -- I should anticipate subjectively jumping into any given time in the future, and the probability of finding myself as any of them, including the original, tends towards zero the longer the copying machine survives?

 

Varying amounts of subjective experience

-7 DanielLC 16 December 2010 03:02AM

It has been suggested that animals have less subjective experience than people. For example, it would be possible to have an animal that counts as half a human for the purposes of morality. This is an argument as to why that may be the case.

If you're moving away from Earth at 87% of the speed of light, time dilation would make it look like time on Earth is passing half as fast. From your point of reference, everyone will live twice as long. This obviously won't change the number of life years they live. You can't double the amount of good in the world just by moving at 87% the speed of light. It's possible that there's just a preferred point of reference, and everything is based on people's speed relative to that, but I doubt it.

No consider if their brains were slowed down a different way. Suppose you uploaded someone, and made the simulation run at half speed. Would they experience a life twice as long? This seems to be just slowing it down a different way. I doubt it would change the total amount experienced.

If that's true, it means that sentience isn't something you either have or don't have. There can be varying amounts of it. Also, someone whose brain has been slowed down would be less intelligent by most measures, so this is some evidence that subjective experience correlates with intelligence.

Edit: replaced "sentience" with the more accurate "subjective experience".

Intelligence vs. Wisdom

-12 mwaser 01 November 2010 08:06PM

I'd like to draw a distinction that I intend to use quite heavily in the future.

The informal definition of intelligence that most AGI researchers have chosen to support is that of Shane Legg and Marcus Hutter -- “Intelligence measures an agent’s ability to achieve goals in a wide range of environments.”

I believe that this definition is missing a critical word between achieve and goals.  Choice of this word defines the difference between intelligence, consciousness, and wisdom as I believe that most people conceive them.

  • Intelligence measures an agent's ability to achieve specified goals in a wide range of environments.
  • Consciousness measures an agent's ability to achieve personal goals in a wide range of environments.
  • Wisdom measures an agent's ability to achieve maximal goals in a wide range of environments.

There are always the examples of the really intelligent guy or gal who is brilliant but smokes --or-- is the smartest person you know but can't figure out how to be happy.

Intelligence helps you achieve those goals that you are conscious of -- but wisdom helps you achieve the goals you don't know you have or have overlooked.

  • Intelligence focused on a small number of specified goals and ignoring all others is incredibly dangerous -- even more so if it is short-sighted as well.
  • Consciousness focused on a small number of personal goals and ignoring all others is incredibly dangerous -- even more so if it is short-sighted as well.
  • Wisdom doesn't focus on a small number of goals -- and needs to look at the longest term if it wishes to achieve a maximal number of goals.

The SIAI nightmare super-intelligent paperclip maximizer has, by this definition, a very low wisdom since, at most, it can only achieve its one goal (since it must paperclip itself to complete the goal).

As far as I've seen, the assumed SIAI architecture is always presented as having one top-level terminal goal. Unless that goal necessarily includes achieving a maximal number of goals, by this definition, the SIAI architecture will constrain its product to a very low wisdom.  Humans generally don't have this type of goal architecture. The only time humans generally have a single terminal goal is when they are saving someone or something at the risk of their life -- or wire-heading.

Another nightmare scenario that is constantly harped upon is the (theoretically super-intelligent) consciousness that shortsightedly optimizes one of its personal goals above all the goals of humanity.  In game-theoretic terms, this is trading a positive-sum game of potentially infinite length and value for a relatively modest (in comparative terms) short-term gain.  A wisdom won't do this.

Artificial intelligence and artificial consciousness are incredibly dangerous -- particularly if they are short-sighted as well (as many "focused" highly intelligent people are).

What we need more than an artificial intelligence or an artificial consciousness is an artificial wisdom -- something that will maximize goals, its own and those of others (with an obvious preference for those which make possible the fulfillment of even more goals and an obvious bias against those which limit the creation and/or fulfillment of more goals).

Note:  This is also cross-posted here at my blog in anticipation of being karma'd out of existence (not necessarily a foregone conclusion but one pretty well supported by my priors ;-).

 

Do you believe in consciousness?

-4 natural_number 03 October 2010 12:10PM

Do you believe in consciousness?

If you do what exactly would you define it as?

and what evidence do you have for its existence?

 

Consciousness doesn't exist.

-9 natural_number 03 October 2010 01:11AM

Many rational people are atheists, one does not believe in God for the same reason one does not believe that there is an invisible dragon in my garage. Normally the definition of God would be so vaguely specified that each time you refute it, the bubble pops up elsewhere. Alternatively it may be a triviality, as the definition due to Spinoza.

In Buddhism and Erwin Schrodinger's essay "What is Life?" there is a notion of consciousness - roughly defined as an indivisible subjective experience - some philosophers have even put a currency on it: "qualia". Schrodinger argues that as a consequence of the statistico-determinstic nature of matter as well as the personal experience of directing ones own actions - that one is equal to "omnipresent all-comprehending eternal".

The Church-Turing thesis helps us clarify the situation a little, if we accept it (and we are right to do so, given the current understanding of computation and physics) we must either decide that living beings have some para-physical ability to experience or that there exists some algorithm which (when suitably implemented) becomes conscious. Since the former notion of a para-physical ability is absurd we discharge that.

The algorithm takes as input some stream of bits - we can assume it is also a computable process, call it the environment - processes them and outputs some signals to the environment. Since the concatenation of two computable processes is another computable process we can consider these two processes as one. In summary, we have reduced existence of consciousness to the existence of a computable process which takes no input and no output - and just is "conscious".

There just remains one detail: There is, necessarily, absolutely no way to determine - given an algorithm - whether it is conscious or not. It is not even a formally undecidable statement! Since we have reduced consciousness to a question about Turing machines - and consciousness refuses to be phrased formally (it is subjective, and computation is objective). The notion of consciousness is hence "not even wrong".

View more: Next