I don't get paid on the basis of Omega's prediction given my action. I get paid on the basis of my action given Omega's prediction. I at least need to know the base-rate probability with which I actually one-box (or two-box), although with only two minutes, I would probably need to know the base rate at which Omega predicts that I will one-box. Actually, just getting the probability for each of P(Ix|Ox) and P(Ix|O~x) would be great.
I also don't have a mechanism to determine if 1033 is prime that is readily available to me without getting hit by a trolley (with what probability do I get hit by the trolley, incidentally?), nor... (read more)
Accounting for qualia and starting from qualia are two entirely different things. Saying "X must have qualia" is unhelpful if we cannot determine whether or not a given thing has qualia.
Qualia can perhaps best be described, briefly, as "subjective experience." So what do we mean by 'subjective' and 'experience'?
If by 'subjective' we mean 'unique to the individual position' and by 'experience' we mean 'alters its internal state on the basis of some perception' then qualia aren't that mysterious: a video camera can be described as having qualia if that's what we are talking about. Of course, many philosophers won't be happy with that sort of breakdown. But it isn't clear that they... (read more)
- You've chosen one of the easier aspects of consciousness: self-awareness rather than, eg. qualia.
I cover this a bit when I talk about awareness, but I find qualia to often be used in such a way as to obscure what consciousness is rather than explicate it. (If I tell you that consciousness requires qualia, but can't tell you how to distinguish things which have qualia from things which do not, along with good reason to believe that this way of distinguishing is legitimate, then rocks could have qualia.)
- The "necessarily biological" could be aposteriori nomic necessity, not apriori conceptual necessity, which is the only kind you knock down in your comment.
If the defenders of... (read more)
I find it helps to break down the category of 'consciousness.' What is it that one is saying when one says that "Consciousness is essentially biological"? Here it's important to be careful: there are philosophers who gerrymander categories. We can start by pointing to human beings, as we take human beings to be conscious, but obviously we aren't pointing at every human attribute. (For instance, having 23 base pairs of chromosomes isn't a characteristic we are pointing at.) We have to be careful that when we point at an attribute, that we are actually trying to solve the problem and not just obscure it: if I tell you that Consciousness is only... (read 405 more words →)
On those criteria, I would say Plato. Because Plato came up with a whole mess of ideas that were... well, compelling but obviously mistaken. Much of Western Philosophy can be put in terms of people wrestling with Plato and trying to show just why he is wrong. (Much of the rest is wrestling with Aristotle and trying to show why HE is wrong... but then, one can put Aristotle into the camp of "people trying to show why Plato is wrong.")
There's a certain sort of person who is most easily aroused from inertia when someone else says something so blatantly, utterly false that they want to pull their hair out. Plato helped motivate these people a lot.
The New Organon, particularly Aphorisms 31-46, show not only an early attempt to diagnose human biases (what Bacon referred to as "The Idols of the Mind") but also some of the reasons why he rejected Aristotelian thought, common at the time, in favor of experimental practice.
Maybe there are better ways to expand than through spacetime, better ways to make yourself into this sort of maximizing agent, and we are just completely unaware of them because we are comparatively dull next to the sort of AGI that has a brain the size of a planet? Some way to beat out entropy, perhaps. That'd make it inconsistent to see any sort of sky with UFAI or FAI in it.
I can somewhat imagine what these sorts of ways would be, but I have no idea if those things are likely or even feasible, since I am not a world-devouring AGI and can only do wild mass speculation at what's beyond... (read more)
I get that feeling whenever I hit a milestone in something: if I run a couple miles further than I had previously, if I understand something that was opaque before, if I am able to do something that I couldn't before, I get this "woo hoo!" feeling that I associate with levelling up.
Even if they are sapient, it might not have the same psychological effect.
The effect of killing a large, snarling, distinctly-not-human-thing on one's mental faculties and the effect of killing a human being are going to be very different, even if one recognizes that thing to be sapient.
If they are, Harry would assign moral weight to the act after the fact: but the natural sympathy that is described as eroding in the above quote doesn't seem as likely to be affected given a human being's psychology.
All the posts on FAI theory as of late have given me cause to think. There's something in the conversations about it that has always bugged me, but it is something that I haven't found the words for before now.
It is something like this:
Say that you manage to construct an algorithm for FAI...
Say that you can show that it isn't going to be a dangerous mistake...
And say you do all of this, and popularize it, before AGI is created (or at least, before an AGI goes *FOOM*)...
...
How in the name of Sagan are you actually going to ENFORCE the idea that all AGIs are FAIs?
I mean, if it required some rare material... (read 180 more words →)
One could judge the strength of these with a few empirical tests: such as for (2), comparing industries where it is clear that the skills learned in college (or in a particular major) are particularly relevant vs. industries where it is not as clear, and comparing the number of college grads w/ the relevant skill-signals vs. college grads w/o the relevant skill-signals vs. non-college grads; and for (3), looking to industries where signals of pre-existing ability in that industry do not conform to being in college and comparing their rate of hiring grads vs. non-grads. (This would presumably be jobs in sectors where some sort of loosely defined intellectual ability is not... (read more)