All of HalFinney's Comments + Replies

Years ago, before coming up with even crazier ideas, Wei Dai invented a concept that I named UDASSA. One way to think of the idea is that the universe actually consists of an infinite number of Universal Turing Machines running all possible programs. Some of these programs "simulate" or even "create" virtual universes with conscious entities in them. We are those entities.

Generally, different programs can produce the same output; and even programs that produce different output can have identical subsets of their output that may include ... (read more)

You make a lot of interesting points, but how do you apply them to the question at hand: what should you have for dinner, and why?

This is a fascinating topic, and I hope it attracts more commentary. As Bentarm says, it is important and relevant to each of us, yet the topic is fraught with uncertainty, and it is expensive to try to reduce the uncertainty.

I do not believe Taubes. No one book can outweigh the millions of pages of scientific research which have led to the current consensus in the field. Taubes is polemical, argumentative, biased, and one-sided in his presentation. He makes no pretense of offering an objective weighing of the evidence for and against various nutritional h... (read more)

Here is a remarkable variation on that puzzle. A tiny change makes it work out completely differently.

Same setup as before, two private dice rolls. This time the question is, what is the probability that the sum is either 7 or 8? Again they will simultaneously exchange probability estimates until their shared estimate is common knowledge.

I will leave it as a puzzle for now in case someone wants to work it out, but it appears to me that in this case, they will eventually agree on an accurate probability of 0 or 1. And they may go through several rounds of a... (read more)

I thought of a simple example that illustrates the point. Suppose two people each roll a die privately. Then they are asked, what is the probability that the sum of the dice is 9?

Now if one sees a 1 or 2, he knows the probability is zero. But let's suppose both see 3-6. Then there is exactly one value for the other die that will sum to 9, so the probability is 1/6. Both players exchange this first estimate. Now curiously although they agree, it is not common knowledge that this value of 1/6 is their shared estimate. After hearing 1/6, they know that the ot... (read more)

Let me give an argument in favor of #4, doing what the others do, in the thermometer problem. Now we seem to have them behaving badly. I think in practice many people would in fact look at other thermometers too in making their guesses. So why aren't they doing it? Two possibilities: they're stupid; or they have a good reason to do it. An example good reason: some thermometers don't read properly from a side angle, so although you think you can see and read all of them, you might be wrong. (This could be solved by #3, writing down the average of the cards,... (read more)

Actually if Omega literally materialized out of thin air before me, I would be amazed and consider him a very powerful and perhaps supernatural entity, so would probably pay him just to stay on his good side. Depending on how literally we take the "Omega appears" part of this thought experiment, it may not be as absurd as it seems.

Even if Omega just steps out of a taxi or whatever, some people in some circumstances would pay him. The Jim Carrey movie "Yes Man" is supposedly based on a true story of someone who decided to say yes to everything, and had very good results. Omega would only appear to such people.

When I signed up for cryonics, I opted for whole body preservation, largely because of this concern. But I would imagine that even without the body, you could re-learn how to move and coordinate your actions, although it might take some time. And possibly a SAI could figure out what your body must have been like just from your brain, not sure.

Now recently I have contracted a disease which will kill most of my motor neurons. So the body will be of less value and I may change to just the head.

The way motor neurons work is there is an upper motor neuron (UMN)... (read more)

Like others, I see some ambiguity here. Let me assume that the substrate includes not just the neurons, but the glial and other support cells and structures; and that there needs to be blood or equivalent to supply fuel, energy and other stuff. Then the question is whether this brain as a physical entity can function as the substrate, by itself, for high level mental functions.

I would give this 95%.

That is low for me, a year ago I would probably have said 98 or 99%. But I have been learning more about the nervous system these past few months. The brain's w... (read more)

Another sample problem domain is crossword puzzles:

Don't stop at the first good answer - You can't write in the first word that seems to fit, you need to see if it is going to let you build the other words.

Explore multiple approaches simultaneously - Same idea, you often can think of a few different possible words that could work in a particular area of the puzzle, and you need to keep them all in mind as you work to solve the other words.

Trust your intuitions, but don't waste too much time arguing for them - This one doesn't apply much because usually peo... (read more)

A perhaps similar example, sometimes I have solved geometry problems (on tests) by using analytical geometry. Transform the problem into algebra by letting point 1 be (x1,y1), point 2 be (x2,y2), etc, get equations for the lines between the points, calculate their points of intersection, and so on. Sometimes this gives the answer with just mechanical application of algebra, no real insight or pattern recognition needed.

I wouldn't be so quick to discard the idea of the AI persuading us that things are pretty nice the way they are. There are probably strong limits to the persuadability of human beings, so it wouldn't be a disaster. And there is a long tradition of advice regarding the (claimed) wisdom of learning to enjoy life as you find it.

I agree about the majoritarianism problem. We should pay people to adopt and advocate independent views, to their own detriment. Less ethically we could encourage people to think for themselves, so we can free-ride on the costs they experience.

Suppose it turned out that the part of the brain devoted to experiencing (or processing) the color red actually was red, and similarly for the other colors. Would this explain anything?

Wouldn't we then wonder why the part of the brain devoted to smelling flowers did not smell like flowers, and the part for smelling sewage didn't stink?

Would we wonder why the part of the brain for hearing high pitches didn't sound like a high pitch? Why the part which feels a punch in the nose doesn't actually reach out and punch us in the nose when we lean close?

I can't help feeling that this line of questioning is bizarre and unproductive.

An example regarding the brain would be successful resuscitation of people who have drowned in icy water. At one time they would have been given up for dead, but now it is known that for some reason the brain often survives for a long time without air, even as much as an hour.

I don't think your question is well represented by the phrase "where is computation".

Let me ask whether you would agree that a computer executing a program can be said to be a computer executing a program. Your argument would suggest not, because you could attribute various other computations to various parts of the computer's hardware.

For example, consider a program that repeatedly increments the value in a register. Now we could alternatively focus on just the lowest bit of the register and see a program that repeatedly complements that bit. Wh... (read more)

Thomas Nagel's classic essay What is it like to be a bat? raises the question of a bat's qualia:

Our own experience provides the basic material for our imagination, whose range is therefore limited. It will not help to try to imagine that one has webbing on one's arms, which enables one to fly around at dusk and dawn catching insects in one's mouth; that one has very poor vision, and perceives the surrounding world by a system of reflected high-frequency sound signals; and that one spends the day hanging upside down by one's feet in an attic. In so far as

A bit OT, but it makes me wonder whether the scientific discoveries of the 21st century are likely to appear similarly insane to a scientist of today? Or would some be so bold as to claim that we have crossed a threshold of knowledge and/or immunity to science shock, and there are no surprises lurking out there bad enough to make us suspect insanity?

One question on your objections: how would you characterize the state of two human rationalist wannabes who have failed to reach agreement? Would you say that their disagreement is common knowledge, or instead are they uncertain if they have a disagreement?

ISTM that people usually find themselves rather certain that they are in disagreement and that this is common knowledge. Aumann's theorem seems to forbid this even if we assume that the calculations are intractable.

The rational way to characterize the situation, if in fact intractability is a practical o... (read more)

Try a concrete example: Two dice are thrown, and each agent learns one die's value. In addition, each learns whether the other die is in the range 1-3 vs 4-6. Now what can we say about the sum of the dice?

Suppose player 1 sees a 2 and learns that player 2's die is in 1-3. Then he knows that player 2 knows that player 1's die is in 1-3. It is common knowledge that the sum is in 2-6.

You could graph it by drawing a 6x6 grid and circling the information partition of player 1 in one color, and player 2 in another color. You will find that the meet is a partiti... (read more)

How about Scott Aaronson:

http://www.scottaaronson.com/papers/agree-econ.pdf

He shows that you do not have to exchange very much information to come to agreement. Now maybe this does not address the question of the potential intractability of the deductions to reach agreement (the wannabe papers may do this) but I think it shows that it is not necessary to exchange all relevant information.

The bottom line for me is the flavor of the Aumann theorem: that there must be a reason why the other person is being so stubborn as not to be convinced by your own tenaci... (read more)

I agree about the issue of unresolved arguments. Was agreement reached and that''s why the debate stopped? No way to tell.

Particularly the epic AI-foom debate between Robin and Eliezer on OB, over whether AI or brain simulations were more likely to dominate the next century, was never clearly resolved with updated probability estimates from the two participants. In fact probability estimates were rare in general. Perhaps a step forward would be for disputants to publicize their probability estimates and update them as the conversation proceeds.

BTW sorry to see that linkrot continues to be a problem in the future.

Yes, I think that's a good explanation. One question it raises is ambiguity in thinking of QM via "many worlds". What constitutes a "world"? If we put a system into a coherent superposition, does that mean there are two worlds? Then if we transform it back into a pure state, has a world gone away? What about the fact that whether it is pure or in a superposition depends arbitrarily on the chosen basis? A pure-state vertically polarized photon is in a superposition of states using the diagonal basis. How many worlds are there, two or one... (read more)

Here are the four papers relating to influence from the future and the LHC:

http://arxiv.org/find/physics/1/au:+Ninomiya_M/0/1/0/all/0/1

The basic idea is that these physicists have a theory that the Higgs particle would be highly unusual, such that its presence in a branch of the multiverse would greatly decrease the measure of that branch. Now I don't claim to understand their math, but it seems that this might produce a different result than the usual anthropic-type arguments regarding earth-destroying experiments.

The authors refer to an "influence f... (read more)

Wei, I understand the paper probably less well than you do, but I wanted to comment that p~, which you call r, is not what Robin calls a pre-prior. He uses the term pre-prior for what he calls q. p~ is simply a prior over an expanded state space created by taking into consideration all possible prior assignments. Now equation 2, the rationality condition, says that q must equal p~ (at least for some calculations), so maybe it all comes out to the same thing.

Equation 1 defines p~ in terms of the conventional prior p. Suppressing the index i since we have o... (read more)

The voice banking software I'm using is from the Speech Research Lab at the University of Delaware. They say they are in the process of commercializing it; hopefully it will still be free to the disabled. Probably not looking for donations though.

Another interesting communications assistance project is Dasher. They have a Java applet demo as well as programs for PC and smart phones. It does predictive input designed to maximize effective bandwidth. A little confusing at first but supposedly after some practice you can type fast with only minimal use of the... (read more)

I want to thank everyone for their good wishes and, um, hugs :)

As it stands, my condition is quite good. In fact at the time of my diagnosis two months ago, I was skeptical that it was correct. The ALS expert seemed rather smug that he had diagnosed me so early, saying that I was the least affected of any of his patients. Not only were my symptoms mild, I had had little or no progression in the three months at that time since I had first noticed anything wrong.

However, since then there has been noticeable progression. My initial symptoms were in my speech,... (read more)

It was actually extremely reassuring as the reality of the diagnosis sunk in. I was surprised, because I've always considered cryonics a long shot. But it turns out that in this kind of situation, it helps tremendously to have reasons for hope, and cryonics provides another avenue for a possibly favorable outcome. That is a good point that my circumstances may allow for a well controlled suspension which could improve my odds somewhat.

You're right though that with this diagnosis, life insurance is no longer an option. In retrospect I would be better off if... (read more)

I am indeed signed up, having been an Alcor client for 20 years.

Ironically I chose full-body suspension as opposed to so-called neurosuspension (head only) on the theory that the spinal cord and peripheral nervous system might include information useful for reconstruction and recovery. Now it turns out that half of this data will be largely destroyed by the disease. Makes me wonder if I should convert to neuro.

Indeed even the popular (mis)conception of head-only revival wouldn't be that bad for me, not unlike the state I will have lived in for a while. In ... (read more)

"[the mind] could be a physical system that cannot be recreated by a computer"

Let me quote an argument in favor of this, despite the apparently near universal consensus here that it is wrong.

There is a school of thought that says, OK, let's suppose the mind is a computation, but it is an unsolved problem in philosophy how to determine whether a given physical system implements a given computation. In fact there is even an argument that a clock implements every computation, and it has yet to be conclusively refuted.

If the connection between physic... (read more)

Two comments. First, your point about counterfactuals is very valid. Hofstadter wrote an essay about how we tend to automatically only consider certain counterfactuals, when an infinite variety are theoretically possible. There are many ways that the world might be changed so that Joe one-boxes. A crack in the earth might open and swallow one box, allowing Joe to take only the other. Someone might have offered Joe a billion dollars to take one box. Joe might aim to take two but suffer a neurological spasm which caused him to grasp only one box and then lea... (read more)

Two comments. First, your point about counterfactuals is very valid. Hofstadter wrote an essay about how we tend to automatically only consider certain counterfactuals, when an infinite variety are theoretically possible. There are many ways that the world might be changed so that Joe one-boxes. A crack in the earth might open and swallow one box, allowing Joe to take only the other. Someone might have offered Joe a billion dollars to take one box. Joe might aim to take two but suffer a neurological spasm which caused him to grasp only one box and then lea... (read more)

Two comments. First, your point about counterfactuals is very valid. Hofstadter wrote an essay about how we tend to automatically only consider certain counterfactuals, when an infinite variety are theoretically possible. There are many ways that the world might be changed so that Joe one-boxes. A crack in the earth might open and swallow one box, allowing Joe to take only the other. Someone might have offered Joe a billion dollars to take one box. Joe might aim to take two but suffer a neurological spasm which caused him to grasp only one box and then lea... (read more)

Two comments. First, your point about counterfactuals is very valid. Hofstadter wrote an essay about how we tend to automatically only consider certain counterfactuals, when an infinite variety are theoretically possible. There are many ways that the world might be changed so that Joe one-boxes. A crack in the earth might open and swallow one box, allowing Joe to take only the other. Someone might have offered Joe a billion dollars to take one box. Joe might aim to take two but suffer a neurological spasm which caused him to grasp only one box and then lea... (read more)

We talk a lot here about creating Artificial Intelligence. What I think Tiiba is asking about is how we might create Artificial Consciousness, or Artificial Sentience. Could there be a being which is conscious and which can suffer and have other experiences, but which is not intelligent? Contrariwise, could there be a being which is intelligent and a great problem solver, able to act as a Bayesian agent very effectively and achieve goals, but which is not conscious, not sentient, has no qualia, cannot be said to suffer? Are these two properties, intelligen... (read more)

Reading the comments here, there seem to be two issues entangled. One is which organisms are capable of suffering (which is probably roughly the same set that is capable of experiencing qualia; we might call this the set of sentient beings). The other is which entities we would care about and perhaps try to help.

I don't think the second question is really relevant here. It is not the issue Tiiba is trying to raise. If you're a selfish bastard, or a saintly altruist, fine. That doesn't matter. What matters is what constitutes a sentient being which can expe... (read more)

I thought maybe we were hearing about the LOTR story through something like the chronophone - the translation into English also translated the story into something analogous for us.

I remember reading once about an experiment that was said to make rats superstitious.

These rats were used in learning experiments. They would be put into a special cage and they'd have to do something to get a treat. Maybe they'd have to push a lever, or go to a certain spot. But they were pretty good at learning whatever they had to do. They were smart rats. They knew the score, they knew what the cage was for.

So they did a new experiment, where they put them into the training cage as usual. But instead of what they did bringing the treat, they always got... (read more)

Actually, why doesn't the Hero's world have a Counter-Force? Shouldn't every world have something like it? How many times have our world escaped from the brink of nuclear annihilation, for example?

Right, like the way the LHC keeps breaking before they can turn it on and have it destroy the universe. Sooner or later we'll figure out what's happening.

I agree with the logic of this analysis, but I have a problem with one of the implicit premises: that "we" should care about political issues at all, and that "we" make governmental decisions. I think this is wrong, and its wrongness explains the seemingly puzzling phenomenon of jumping from tree to forest.

There was no need for anyone beyond the jury to have an opinion on the Duke lacrosse case. We weren't making any decisions there. I certainly wasn't, anyway. So of course when people do express an interest, it is for entertainment and... (read more)

A typical comment from an anti-Cheerios advocate. Is this what LW is coming to? Cheerios lovers unite!

Anyway it was probably not clear but I was a little tongue in cheek with my Cheerios rant. I think what I wrote is correct but mostly I was having fun pretending that there could be a big political battle over even the narrow issue of the Cheerios study and what it means.

I'm afraid I have to take issue with your Cheerios story in the linked comment. You say of the 4% cholesterol lowering claim, "This is false. It is based on a 'study' sponsored by General Mills where subjects took more than half their daily calories from Cheerios (apparently they ate nothing but Cheerios for two of their three daily meals)." You link to http://www.askdeb.com/blog/health/will-cheerios-really-help-lower-your-cholesterol/ but that says nothing about how much Cheerios subjects ate.

I found this article that describes the 1998 Cheerios... (read more)

Let me try restating the scenario more explicitly, see if I understand that part.

Omega comes to you and says, "There is an urn with a red or blue ball in it. I decided that if the ball were blue, I would come to you and ask you to give me 1000 utilons. Of course, you don't have to agree. I also decided that if the ball were red, I would come to you and give you 1000 utilons - but only if I predicted that if I asked you to give me the utilons in the blue-ball case, you would have agreed. If I predicted that you would not have agreed to pay in the blue-... (read more)

Thanks for the answer, but I am afraid I am more confused than before. In the part of the post which begins, "So, new problem...", the coin is gone, and instead Omega will decide what to do based on whether an urn contains a red or blue marble, about which you have certain information. There is no coin. Can you restate your explanation in terms of the urn and marble?

I don't see where Omega the mugger plays a central role in this question. Aren't you just asking how one would guess whether a marble in an urn is red or blue, given the sources of information you describe in the last paragraph? (Your own long-term study, a suddenly-discovered predictions market.)

Isn't the answer the usual: do the best you can with all the information you have available?

Maybe a better heuristic is to consider whether your degree of assurance in your position is more or less than your average degree of assurance over all topics on which you might encounter disagreements. Hopefully there would be less of a bias on this question of whether you are more confident than usual. Then, if everyone adopted the policy of believing themselves if they are unusually confident, and believing the other person if they are less confident than usual, average accuracy would increase.

I'd agree that "in general, you should believe yourself" is a simpler rule than "in general, you should believe yourself, except when you come across someone else who has a different belief". And simplicity is a plus. There are good reasons to prefer simple rules.

The question is whether this simplicity outweighs the theoretical arguments that greater accuracy can be attained by using the more complex rule. Perhaps someone who sufficiently values simplicity can reasonably argue for adopting the first rule.

ETA: Maybe I am wrong about the ... (read more)

I meant, do you have a sense of what percentage of top-level posts have comments which show the problem?

I'd like to see a more popular discussion of Aumann's disagreement theorem (and its follow-ons), and what I believe is called Kripkean possible-world semantics, an alternative formulation of Bayes theorem, used in Aumann's original proof. The proof is very short, just a couple of sentences, but explaining the possible-world formalism is a big job.

Tilba, Wei's earlier post pointed to this article:

http://weidai.com/black-holes.txt

You might also need to know that computation can be done in principle almost without expending energy, and the colder you do the computation, the less energy is wasted. Hence being cold is a good thing, and black holes are very cold.