because producing new evidence is not possible anymore.

Okay...

So, say it turns out that, well, Eve is irrational. Somehow.

Now what? Do we go "neener-neener" at her? What's the point? What's the use that you could get out of labeling this behavior irrational?

Suppose Adam dies and is cryo-frozen. During Eve's life, there will be no resuscitation of Adam. Sometime afterward, however, Omega will arrive, deem the problem interesting and simulate Adam via really really really advanced technology.

Turns out he didn't do it.

Is she now rational because, well, turns out she was right after all? Well, no, because getting the right answer for the wrong reasons is not the rational way to go about things (in general, it might help in specific cases if you need to get the answer right but don't care how).

....

Actually, let me just skip over a few paragraphs I was going to write and skip to the end.

You cannot have 100% confidence interval. Because then your belief is set in stone and it cannot change. You can have a googleplex nines if you want, but not 100% confidence.

Fallacy of argument from probability (if it can happen then it must happen) aside; How is it rational to discard a belief you are holding on shaky evidence if you think with near absolute certainty that no more evidence will arrive, ever? What will you do when there is more evidence? (Hint: Meeting Adam's mother at the funeral and hearing childhood stories about what a nice kid he was is more evidence for his character, albeit very weak evidence - and so are studies that show that certain demographics of the timeperiod that Adam lived in had certain characteristics) You gotta update! (I don't think that fallacy I mentioned applies; if it does, we can fix it with big numbers; if you are to hold this belief everywhere, then... the probabilities go up as it turns from "in this situation" to "in at least one of all these situations")

So to toss a belief aside because you think there will be no more evidence is the wrong action to me. You can park a belief. That is to take no action. Maintain status quo. No change in input is no change in output. But you do NOT clear the belief.

Let me put up a strawman - I'll leave it up to others to see if there's something harder underneath - if you hold this action - "I think there will be no more evidence, and I am not very confident either way, so I will discard the output" to be the rightful action to take, how do you prevent yourself from getting boiled like a frog in a pan (yes, that's a false story - still, I intend the metaphorical meaning: how do you stop yourself from discarding every bit of evidence that comes your way, because you "know" there to be no more evidence?)

In my opinion, to do as you say weakens or even destroys the gradual "update" mechanism. This leads to less effective beliefs, and thus is irrational.

Were we to now look at the 3 questions, I'd answer..

Again, Eve is irrational because she says it cannot be falsified. If we let Eve say "I still think he didn't do it because of his character, and I will keep believing this until I see evidence to the contrary - and if such evidence doesn't exist, I will keep believing this forever" - then yes, Eve is rational.

The second question, yes via this specific example. Here it can, thus it can.

Yes, it can be extended to belief in God. Provided we restrict "God" to a REALLY TINY thing. As in, gee, a couple thousand years ago, something truly fantastic happened - it was God! I saw it with my own eyes! You can keep believing there was, at that point in time, an entity causing this fantastic thing. Until you get other evidence, which may never happen. What you CANNOT do is say, "hey, maybe this 'God' that caused this one fantastic thing is also responsible for creating the universe and making my neighbor win the lottery and my aunt get cancer and ..." That's unloading a huge complexity on an earlier belief without paying appropriate penalties.

You don't only need evidence that the fantastical events were caused, you also need evidence they were caused by the same thing if you wish to attribute them to that same thing.

Saying "X is emergent" is conveying some information, if there is someone in the room that does not already know this fact. Here is an example:

Quarks are emergent.

This is not an explanation though. It is more like a anti-explanation. I just claimed that there is an underlying explanation to quarks, and then stopped. I told you to make space for an explanation, in you mental world model, and then I left you with that space empty. If you believed my statement, and if you don't already know how quarks emerges and from what, I just made an explanation shaped hole in your mind. This is not nice of me.

But at least you now know that there is an explanation to be found. When you thought quarks was fundamental, you did not even know to look, because fundamental things can not be explained, only described.

While you could be right, you're claiming the students have a clear and accurate model of their own beliefs. What does that mean? Could they explain the nature of technical explanation?

There's a certain popular series of books which portrays intelligence as a matter of parroting facts, without trying to connect any two of them - not even the disappearance of every child in the world and an event in world politics shortly thereafter. Now, you could try to explain this by saying the authors (plural) are deliberately selling their customers garbage to maximize return-on-investment. And you could try to claim that their fans are buying the books just to signal tribal membership. And that explanation may have some power - but I draw the line at saying that they all understand clearly what the books lack in terms of credibility.

I totally, I agree that it is often better to study narrow and deep.

But this word policing is not net helpful, all things coincided.

Yudkovsky does not like when people call the invention of LSD a Singularity. Ok, I can see why. But I don't like Yudkovsky use of the word singularity, because that is absolutely not what the word means in physics or math. I used to be quite upset over the fact that AI people had generalized the word "singularity" to mean "exponential or super exponential growth". On the other hand, what ever. It is really not that big of a deal. I will have to say "mathematical singularity" some times, to specify what I mean, when ever it is not clear from the context. I can live with that compromise.

Different fields use the same word to mean different things. This some times leads to misunderstanding, which is bad. But the alternative would be for every field to make up their own strings of syllables for every technical word, which is just too unpractical.

Also, I happen to know, that when astrophysicists talk about the evolution of stars, they are not borrowing the word "evolution" from the biological use. They are using "evolution" in the more original meaning, which is "how something change over time", from the word "evolve". The evolution of a star is the process of how the star change over time, from creation to end. No one in the field thinks that they should borrow ideas from biology on the ground that biologists use the same word. Nether can I imagine anyone in evolutionary biology deciding to draw conclusions from theories of the evolution of starts, just because of the common word "evolution".

I can totally imagine someone how knows close to nothing about both stars and biology, being confused by this word "evolution" being used in different settings. Confusing the uneducated public is generally bad. More specifically it is uncooperative, since in most field, you yourself is part of the uneducated public. But there is also a trade-off. How much effort should we put on this? Avoiding otherwise useful use of words is a high cost.

The Singularity, Quantum Cromodynamics, Neural networks, Tree (in graph theory), Imaginary numbers, Magic (as placeholder for the un-explained), Energy, etc.

The use of metaphors and other types of borrowed words in technical language is widespread, because it is so damned practical. Sometimes we use metaphors the same way as good poet, to lend the preciseness from one concept to another. But sometimes one just needs a label and reusing an old word is less effort than coming up with, and remember, an actual new sound.

Back to the trade-off. How much would it help if different topics never borrowed language of each other? Would the general public be significantly less confused? For this tactic to work, everyone, not just scientists, has to stop borrowing words of each other. And we have to the restrict usage of hundreds (maybe thousands) of words that are already in use.

But maybe there is a third way? Instead of teaching everyone not to borrow words, we could teach everyone that words can have different meanings in different context. This is also a huge project, but considerably smaller for several reasons.

1. It is an easier lesson to learn. At least for me, and generalizing from one example.
2. It is more aligned with how natural language actually work.
3. It is a lesson that can be tough one person a the time. We don't have to change all at once, for it to work.

My model of Yudkowsky (which is created solely from reading many of his LessWrong posts) now complains that my suggestion will not work, because of how the brain work. Using the same words causes our brain to use the same mental bucket, or something like that.

But I know that my suggestion works, at least it works for me. My brain have different mental settings for different topics and situations, where words can have different meaning in different settings. It does not mean that I have conflicting mental models of the world, just that I keep conflicting definitions of words. It is very much like switching to a different language. The word "bard" means shed in English, but it means child in my native language, Swedish, and this is not a problem. I would never even have connected the English::barn and Swedish::barn, if it was not pointed out to me in a totally unrelated discussion.

Unfortunately I don't know how my brain ended up like this, so I can't show you the way. I can only testify that the destination exists. But if I where to guess, I would say that, I just gradually built up different sets of technical vocabulary, which sometimes had overlapping sounds. Maybe being bi-lingual helps? Not overly thinking in words probably helps too.

Sometimes when a conversation is sliding from one topic to an other, maybe a physics conversation take a turn in to pure math, I will notice that my brain have switched language setting, because the sentence I remember just saying, does not make sense to me anymore.

Large corporations are not really very like AIs at all. An Artificial Intelligence is an intelligence with a single utility function, whereas a company is a group of intelligences with many complex utility functions. I remain unconvinced that aggregating intelligences and applying the same terms is valid - it is, roughly speaking, like trying to apply chromodynamics to atoms and molecules. Maximising shareholder value is also not a simple problem to solve (if it were, the stock market would be a lot simpler!), especially since "shareholder value" is a very vague concept. In reality, large corporations almost never seek to maximise shareholder value (that is, in theory one might, but I can't actually imagine such a firm). The relevant terms to look up are "satisficing" and "principal-agent problem".

This rather spoils the idea of firms being intelligent - the term does not appear applicable (which is, I think, Eliezer's point).

Is the claim that this is a school thing or a life thing?

This is a life thing. One programming example might be people running code they've copy-pasted off of StackOverflow to see if it solves their problem--they don't understand what it will do, but they have a vague hope that it will be the magic incantation that will do what they want it to do.

But even there they may have a sense that programming has some objectivity to it. Probably a better example is dysfunctional organizational dynamics, where guessing what the boss wants you to say serves you better than trying to estimate what best accomplishes organizational goals.

"If I write 'because of heat conduction' on a test, I have a chance of getting points." is an anticipation controller.

Right, but read this section again:

This is not a hypothesis about the metal plate. This is not even a proper belief. It is an attempt to guess the teacher's password.

Guessing the teacher's password is obviously a hypothesis--but it's a hypothesis about the teacher, not the plate.

I notice that I am confused by this post.

Is the claim that this is a school thing or a life thing? I can see how this behavior might happen if a student is more interested in getting good grades than in actual learning. In such a situation "learning the teachers password" might be a short cut to get to your actual goals.

If the claim is that this is a life thing, could some one give me some more non-classroom example? Organized religion counts as classroom.

When I fist heard that light is a wave, then I interpreted that sentence in my brain an gave it meaning. I can't say for sure that I gave it the correct meaning. But I defiantly know that I did not just save a way the sound pattern, as truth. Because I don't think that way, and I can't even imagine thinking that way.

I can, on the other hand, imagine thinking: "If I write 'because of heat conduction' on a test, I have a chance of getting points." This is not how I went though school, because I was interested in accusal learning, but I can model a student who thinks this way.

"If I write 'because of heat conduction' on a test, I have a chance of getting points." is an anticipation controller.

This is a mistake. There is actually a two-electron state in the OP. (And there is no assumption 'that they are independently and individually real.' The claim is merely that the two-electron state is real.)

This is Awesome! Exactly what I want!

Two questions:

1̶)̶ ̶I̶t̶ ̶s̶e̶e̶m̶s̶ ̶l̶i̶k̶e̶ ̶I̶ ̶n̶e̶e̶d̶ ̶t̶o̶ ̶p̶a̶y̶ ̶b̶y̶ ̶p̶a̶y̶ ̶p̶a̶l̶.̶ ̶O̶n̶l̶y̶ ̶I̶ ̶c̶a̶n̶'̶t̶ ̶b̶e̶c̶a̶u̶s̶e̶ ̶a̶p̶p̶a̶r̶e̶n̶t̶l̶y̶ ̶m̶y̶ ̶S̶w̶e̶d̶i̶s̶h̶ ̶p̶h̶o̶n̶e̶ ̶n̶u̶m̶b̶e̶r̶ ̶i̶s̶ ̶n̶o̶t̶ ̶v̶a̶l̶i̶d̶.̶ ̶I̶ ̶t̶r̶i̶e̶d̶ ̶b̶o̶t̶h̶ ̶+̶4̶6̶7̶6̶8̶4̶9̶6̶4̶X̶X̶ ̶a̶n̶d̶ ̶0̶0̶4̶6̶7̶6̶8̶4̶9̶6̶4̶X̶X̶ ̶a̶n̶d̶ ̶n̶e̶t̶h̶e̶r̶ ̶w̶o̶r̶k̶s̶.̶ ̶(̶X̶X̶ ̶=̶ ̶t̶h̶e̶ ̶t̶w̶o̶ ̶l̶a̶s̶t̶ ̶d̶i̶g̶i̶t̶s̶ ̶o̶f̶ ̶m̶y̶ ̶p̶h̶o̶n̶e̶ ̶n̶u̶m̶b̶e̶r̶.̶)̶
Edit: This problem has been solved

2) This post is rather old. Is there more sequences available on podcast by now. How do I find them?