LESSWRONG
LW

Comment Permalink
SilasBarta30

I don't think simulations help. Once you start simulating yourself to arbitrary precision, that being would have the same thoughts as you, including "Hey, I should run a simulation", and then you're back to square one.

More generally, when you think about how to interact with other people, you are simulating them, in a crude sense, using your own mind as a shortcut. See empathic inference.

If you become superintelligent and have lots more computing resources, then your simulations of other minds themselves become minds, with experiences indistinguishable from yours, and make the same decisions, for the same reasons. What's worse, the simulations have the same moral weight! See EY's nonperson predicates.

(This has inspired me to consider "Virtualization Decision Theory", VDT, which says, "Act as though setting the output of yourself in a simulation run by beings deciding how to interact with a realer version of you that you care about more.")

Here are my earlier remarks on the simulated-world / religion parallel.

Reply
Aleksei_Riikonen00

Simulations might be of limited utility (given limited computational resources), but they'd certainly help.

Without simulations, it's very difficult to run complex experiments of how an entity behaves in a series of situations, with the only changing variable being the entity's initial beliefs.

Reply
See in context
by Aleksei_Riikonen
1 min read
71

-16

What must a sane person1 think regarding religion? The naive first approximation is "religion is crap". But let's consider the following:

Humans are imperfectly rational creatures. Our faults include not being psychologically able to maximally operate according to our values. We can e.g. suffer from burn-out if we try to push ourselves too hard.

It is thus important for us to consider, what psychological habits and choices contribute to our being able to work as diligently for our values as we want to (while being mentally healthy). It is a theoretical possibility, a hypothesis that could be experimentally studied, that the optimal2 psychological choices include embracing some form of Faith, i.e. beliefs not resting on logical proof or material evidence.

In other words, it could be that our values mean that Occam's Razor should be rejected (in some cases), since embracing Occam's Razor might mean that we miss out on opportunities to manipulate ourselves psychologically into being more what we want to be.

To a person aware of The Simulation Argument, the above suggests interesting corollaries:

  1. Running ancestor simulations is the ultimate tool to find out, what (if any) form of Faith is most conducive to us being able to live according to our values.
  2. If there is a Creator and we are in fact currently in a simulation being run by that Creator, it would have been rather humorous of them to create our world thus that the above method would yield "knowledge" of their existence.

 

1: Actually, what I've written here assumes we are talking about humans. Persons-in-general may be psychologically different, and theoretically capable of perfect rationality.

2: At least for some individuals, not necessarily all.

Personal Blog

-16

Is it rational to be religious? Simulations are required for answer.
New Comment
71 comments, sorted by
top scoring
Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]cousin_it140

First I was like, "oh no, another one claims that faith could be rational without showing why it is". Then I parsed the simulation part and clicked "upvote" because it showed me a new and scary possibility.

In the year 2020, cousin_it downloads some software for running predictor sims. He wants to optimize his future conduct - test which particular minor mind-hack would be the most beneficial, given his individual psychological makeup. So he runs many ancestor sims of himself poisoned with different irrational mind-hacks to see who wins in the past. I am one such sim, poisoned in an unknown manner; what should I do?

Times like these I regret that I can't write good sci-fi in English because it's not my first language.

Reply
9thomblake
I find that hard to believe; your non-native-speaker status is not apparent from your comments. Clarification: I find it hard to believe that's a limitation on writing good sci-fi, not that English is not your first language.
7Oligopsony
Writing quality fiction takes more facility with language than than writing quality nonfiction (like posts) does. (Not that nativeness is an absolute barrier: English was Nabokov's third language, IIRC.) A cynic could observe that readers of genre fiction are comparatively less demanding in this respect, though.
1Risto_Saarelma
Very off topic, but I've actually often wondered why there don't seem to be any non-native speakers writing commercial fiction in English, given how massively larger the English-speaking market is compared to that of a lot of the smaller European languages. Nabokov is literally the only example I can think of.
1Oligopsony
Well, fiction writing generally isn't the sort of thing one enters into for the wonderful market. You do it because you love it and are somehow good enough at it that it can pay the bills. Probably you write your first novel in your spare time with very low expectations of it ever being published. So why write in anything but your favorite language? And while, proficiency-wise, people like Nabokov and Conrad exist, chances are that you aren't one of them. (That said, there are probably more non-native writers of note than you think. How many of my favorite authors have red hair? I have no idea.)
1Risto_Saarelma
Thing is, I read almost no fiction in Finnish and quite a lot in English. There isn't much of a tradition of speculative fiction in Finnish that isn't just copying stuff already done in English. So if I were to write a SF or a fantasy story, I'd seriously consider whether I could do it in English, because for me those kinds of stories are written in English and then maybe poorly translated into Finnish. I'm sure few people match up to Nabokov or Conrad (whose non-nativeness I didn't know about), but I find it odd that I don't know of any contemporary writers who are even trying to write in English without being native speakers. I'm sure there are ones I don't know about, so any examples of currently active published non-native English fiction writers are welcome.
1gwern
Sounds like an opportunity! I wonder if it would be more valuable to translate (with the royalties that implies) or to just rip off?
3Risto_Saarelma
An opportunity for what? Genre literature is already translated into Finnish, and the publishers with something that isn't Harry Potter or Lord of the Rings are mostly able to stick around, but probably aren't making big profits. Finnish-written SF or fantasy is mostly crap and sinks without a trace. A good indication of the general level of quality is that for the first 20 years of the Atorox prize for the best Finnish SF short story, two outlier authors won ten of the 20 awards. No one has even been able to earn a full-time living writing SF in Finnish, not to mention growing rich. Of course there's nothing preventing someone from writing stuff on par with Ted Chiang and Jeff VanderMeer in Finnish, but why bother? The book could be a cult classic but probably not a mainstream hit, and there aren't enough non-mainstream Finnish-speaking buyers for a book to earn someone a living. I don't even know who buys all the translated Finnish SF books. I've read a bunch of those from the library, but almost all of the books I've bought have been in English. Why bother with translations that are both more expensive than the original paperbacks and have clunkier language?
4DanielLC
Why do you have to write it in English?
1jimrandomh
I have precommitted as strongly as I can to never run simulations of myself which are worse off than the version the simulation was based on. This might fall out as a consequence of UDT, but with a time-varying utility function I'm not really sure. In general, self-copying and self-simulation require extreme care. They might be able to affect subjective experience in a way that goes back in time. The rules of subjective experience, if any, are non-transferrable (you can't learn them from someone else who's figured them out, even in principle) and might not be discoverable at all.
6Will_Newsome
Humans can't easily precommit to anything at all, and even if they could, it'd be incredibly stupid to try without thinking about it for a very very long time. I'm surprised at how many people don't immediately see this.
1cousin_it
I don't believe your decision follows from UDT. If you have a short past and a long future, knowledge gained from sims may improve your future enough to pay off the sims' suffering.
0jimrandomh
This is firmly in the realm of wild speculation and/or science fiction plot ideas. That said - You're right that it does not follow from UDT alone. I do think it follows from a combination of UDT with many common types of utility functions; in particular, if utility is discounted exponentially with time, or if the sims must halt and being halted is sufficiently bad. A lot depends on what happens subjectively after the simulation is halted, or if there are sufficient resources to keep it going indefinitely. In the latter case, most simulated bad things can be easily made up for by altering the simulated universe after the useful data has been extracted. This would imply that if you are living in a sim created by your future self, your future self follows UDT, and your future self has sufficient resources, you'll end up in a heaven of some sort. Actually, if you ever did gain the ability to run long-duration simulations of your past selves, then it seems like UDT implies you should run rescue sims of yourself. Simulations that have to halt after a short duration are very problematic, though; if you anticipate a long life ahead of you, then your past selves probably also have a long life ahead of them, too, which would be cut short by a sim that has to halt. This would probably outweigh the benefits of any information gleaned.
[-]knb100

I think the mistaken assumption here is that you can actually choose to have faith. Certainly you can choose to repeat the words "I have faith". You can even say those words inside your head. You can even attend religious services. That is not the same as actually believing in your religion.

I think this essentially is what Orwell called "Doublethink", and it seems to explain much of the religious behavior I personally have seen.

Reply
0Aleksei_Riikonen
Introspectively, it seems clear to me that it is possible to choose to have faith. It might be, though, that what this "faith" is is a more complicated question than it would initially seem to be. But I can clearly imagine adopting something that very much feels and would seem to work like "faith". (Might as well now mention that I've been an atheist all my life, and still am. I don't currently have any faith I know of, except perhaps the thing about a time-continuous self.)
2knb
I "chose to have faith" when I had a crisis of faith as a kid. However, after my crisis of faith I no longer actually made predictions based on my faith-beliefs. "Choosing faith" and actually believing something are very different.
0Aleksei_Riikonen
Not making predictions based on faith-beliefs sounds just perfect for what I had in mind here. Maybe it can be characterized as "Doublethink" or "belief in belief (without the actual belief)", but if so, that's fine. The goal would not be to use Faith to make predictions, only to psychologically manipulate oneself.
7orthonormal
Relevant LW posts:
[-]thomblake90

This post does not seem to contribute much. As nawitus pointed out, this post does a good enough job of distinguishing between instrumental and epistemic rationality.

While it seems obvious that in some cases, a false belief will have greater utility than a true one (I can set up a contrived example if you need one), it's a devil's bargain. Once you've infected yourself with a belief that cannot respond to evidence, you will (most likely) end up getting the wrong answer on Very Important Problems.

And if you've already had your awakening as a rationalist, ... (read more)

Reply
1thomblake
Yes, the irony in the last statement is intended.
0Aleksei_Riikonen
I did not hypothetize about infecting oneself with a belief that doesn't respond to evidence. The kind of hypothetical faith I spoke of would respond to evidence; evidence of what is conducive to being able to act according to one's values.
0thomblake
In that case, the following is misleading: At any rate, a belief that would not respond to evidence of its truth or falsehood would be sufficiently malign.
3Aleksei_Riikonen
Ah, yes, you're correct. That was poorly written. On your latter point, do you really mean that in the thought experiment of someone wanting to shoot your friend and coming to you to ask for directions, you hope you couldn't make yourself honestly believe falsehoods that you could then convey to the assassin thereby misdirecting him without seeming dishonest?
3thomblake
Indeed. As long as we're asking for superpowers, I'd prefer to have the ability to defeat the assassin, or to credibly lie to him without believing my lie. Given that this situation is not going to happen to me, I'd rather keep the ability to distinguish truth from falsehood without epistemically poisoning myself.
[-]nawitus70

According to these definitions, it could be instrumentally rational to be religious for some subset of people, but not epimestically rational.

Reply
[-]mtraven50

This post is based on the (very common) mistake of equating religious practice and religious faith. Religion is only incidentally about what you believe; the more important components are community and ritual practice. From that perspective, it is a lot easier to believe that religion can be beneficial. What you think about the Trinity, for instance, is less important than the fact that you go to Mass and see other members of your community there and engage in these bizarre activities together.

There is an enormous blindspot about society in the libertarian/rationalist community, of which the above is just one manifestation.

Reply
1Aleksei_Riikonen
No, I very clearly am aware of those two things as separate things. (Though I could have been clearer about this in my post.) It is not obvious that faith couldn't be psychologically useful, also separately from practice.
[-]TobyBartels50

I know some individuals that I believe would be worse off if they were to have a crisis of faith and lose their religion. And while I can't be sure and have never run any tests to find out, I think that they really believe, not just with belief in belief. By the way, none of these are particularly intelligent people.

But I have a hard time imagining someone intelligent and rational who would be better off deceiving themself and gaining faith. Adopting a religion where you are allowed to fake it (like Risto suggests) would almost certainly be better. Som... (read more)

Reply
[-]Will_Newsome30

I didn't downvote this post, but I can't say I endorse seeing more posts like it. The concept of this post is one of the least interesting in a huge conceptspace of decision theory problems, especially decision theory problems in an ensemble universe. To focus on 'having faith' and 'rationality' in particular might seem clever, but it fails to illuminate in the same way that e.g. Nesov's counterfactual mugging does. When you start thinking about various things simulators might do, you're probably wrong about how much measure is going to be taken up by any ... (read more)

Reply
-1Aleksei_Riikonen
Actually, I didn't try to be relevant or interesting to the LW community. I'm just currently genuinely very interested in the kinds of questions this post was about, and selfishly thought I'd get very useful criticism and comments if I'd post here like this (as did indeed happen). Getting downvoted so much is something that I for some reason enjoy very much :) It probably has to do with me thinking, that while there are very valid points on which my post and my decision to post it can be criticized, I suspect that instead of thinking of those valid reasons to dislike what I did, and seeing them as a reason to downvote, the downvoters probably just had a knee-jerk reaction to religion as a topic (probably even suspecting that I'd have religious views that I don't actually have). If this suspicion is true, I would have ended up demonstrating a form of irrationality somewhat widespread within LW readership. Or then the typical voting LW member is smarter than I thought, and they did actually notice that I was largely just looking for criticism and comments useful to me, instead of formulating my thoughts further on my own, and then perhaps later posting on LW something more polished and with a somewhat different focus.
4Will_Newsome
I've found it hard to avoid doing this kind of thing. Luckily I have people at the Singularity Institute to discuss this kind of hypothesis with. If you post it to the Open Thread, it will like as not be ignored, and if you make a top level post about it, then it will like as not be downvoted (but at least you'll get feedback). Perhaps it would be best if a lot of Less Wrongers made blogs and advertised those blogs in a top level post using some sort of endorsement method? I've thought about writing my own blog before, but it'd be annoying to have to ask a lot of people to check it out or subscribe. But if a lot of LWers did it, it wouldn't be nearly as annoying. Then posts like the one you wrote could still get feedback without taking up space in the minds of a multitude of Less Wrong readers who don't care about the decision theory of simulations. This totally does not work unless you have a way of discovering evidence to distinguish between the two hypotheses, and I don't think you have such a method. Commenters are more likely to have more sophisticated reasons for disagreeing than the average LW lurker who sees the word 'faith' in anything but a totally negative light and immediately downvotes, so posting this gave you little evidence. The downvotes are most likely to come from both the least and most sophisticated of LWers: the least because they're allergic to anything to do with religion, the most because they're allergic to hypotheses that fail to carve reality at its joints. If I was going to downvote the post it'd be because of the latter, but I still don't know what the median reason would be for a downvote.
0Unknowns
"the downvoters probably just had a knee-jerk reaction to religion as a topic (probably even suspecting that I'd have religious views that I don't actually have)" With some exceptions when the poster is well known here, my impression is that posts and comments on the topic of religion do get treated this way on a regular basis.
2Will_Newsome
In my experience bringing up religion to make a point is often a bad call (or so community norms suggest) because politics is the mind killer and alienation for no good reason is generally frowned upon, and bringing up religion as a topic of discussion in itself is often done by those who want to bash religion for a reason that is not sophisticated enough.
[-]simplicio30

I'm not convinced that the religious have any particular advantages wrt akrasia and such things.

A conversion is sure to give you a great deal of energy & willpower temporarily, but ultimately that's a finite resource.

The main advantage the religious have is supportive community. That is where rationalists really fall down, although I think LW is a step in the right direction.

Reply
[-]SilasBarta30

I don't think simulations help. Once you start simulating yourself to arbitrary precision, that being would have the same thoughts as you, including "Hey, I should run a simulation", and then you're back to square one.

More generally, when you think about how to interact with other people, you are simulating them, in a crude sense, using your own mind as a shortcut. See empathic inference.

If you become superintelligent and have lots more computing resources, then your simulations of other minds themselves become minds, with experiences indisting... (read more)

Reply
0Aleksei_Riikonen
Simulations might be of limited utility (given limited computational resources), but they'd certainly help. Without simulations, it's very difficult to run complex experiments of how an entity behaves in a series of situations, with the only changing variable being the entity's initial beliefs.
0SilasBarta
I think that underscores the crucial difference about doing simulations in this context: You're simulating a being that can itself do simulations, and this is the defining aspect of that being's cognitive architecture.
0Aleksei_Riikonen
No, it is not particularly relevant to run such simulations where the entities have sufficient technology to run advanced simulations themselves. Almost all that we'd want to find out in this context we can use simulations of entities of a limited technological level for.
0thomblake
If you believe that, then your earlier estimate of "necessary" was very far off-target.
0Aleksei_Riikonen
I also think that what intelligence I or anyone else has is imperfect and therefore "of limited utility", but this doesn't mean that our intelligence wouldn't be "necessary" for a very large number of tasks, or even that it wouldn't be "the ultimate tool" for most of those tasks. So I don't see at all what would be the contradiction you're referring to.
[-]Aleksei_Riikonen30

Belief in the concept of a time-continuous "self" might be an example of an article of Faith that is useful for humans.

(Most people believe in a time-continuous self anyway, they just don't realize it's something that current best physics tells us there's no evidence for the existence of.)

Reply
8Vladimir_Nesov
Don't confuse heuristics with faith.
0Aleksei_Riikonen
Physics has already given us better ideas that we could replace a belief in a time-continuous self with. If we choose not to use these ideas that better reflect what we know of reality, I wouldn't call it a heuristic, but instead choosing faith over what pure reason would tell us.
[-]jimrandomh110

Physics has already given us better ideas that we could replace a belief in a time-continuous self with. If we choose not to use these ideas that better reflect what we know of reality, I wouldn't call it a heuristic, but instead choosing faith over what pure reason would tell us.

But physics has also confirmed that a time-continuous self is a good enough approximation under most circumstances. You wouldn't call choosing Newtonian physics over relativity "faith", and in most cases you wouldn't call it wrong either. It is only when we try to use the approximation in corner cases, like cloning and death, that it becomes a problem.

Reply
-1Aleksei_Riikonen
The analogy would be that relativity says something that demoralizes us. Using Newtonian physics as a heuristic when solving problems doesn't allow us to avoid that demoralizing effect. If we'd still believe that relativity is the model that is actually true, the demoralizing effect would remain.
4thomblake
Calling a model "true" is a category error. Models predict their relevant details of reality to the accuracy and precision necessary for the tasks to which they are appropriately applied, as best they can.
1Vladimir_Nesov
It's well-defined enough when we talk about models of reality, so far as what "reality" is, assumed understood. It's clearly false that speed of light is 10km/s, it's clearly true that speed of light is not 10 km/s.
0thomblake
Yes, the proposition "the speed of light is 10km/s" is false. However, it is entirely possible to have a model which sets the speed of light to 10km/s (to make the math simpler, possibly), that nonetheless churns out accurate predictions.
0Vladimir_Nesov
LCPW (least convenient possible world). I obviously meant that you use a standard physical framework. Accurate predictions = true model, completely wrong predictions = false model. Simple enough.
2thomblake
Eh. If you want to use words that way, I suppose I can't stop you. You could also use the word "true" to mean "valid" with respect to arguments, and I think your intention would be obvious, but I will also call that a category error.
7cousin_it
I'm gonna pull a Nesov on this one and say that belief in a time-continuous self can be thought of as a value/preference rather than belief. You care about your individual organism because evolution made you care about it, not because it is physically real (whatever that means). Of course, similar reasoning can be used to show that observed particle physics is a Darwinian construct :-) Last I talked with Nesov about it, this was a big puzzle. Any news?
0nawitus
The lack of belief in a time-continuos self would give the same moral value to yourself as to other people, but wouldn't eliminate caring about yourself altogether.
1Vladimir_Nesov
Wrong. To see the error, try applying the argument to structures other than people.
1nawitus
Care to give an example then?
-1Vladimir_Nesov
Example of what? You didn't give your argument, only conclusion. I only guessed that this argument, whatever it is, will more visibly crumble in the case I suggested.
-2nawitus
Eh. If you don't know the argument it's irrational to call it wrong. I didn't really argue anything, I just made an observation for those people who possibly believe that time-continuos self is required for morality.
0Vladimir_Nesov
Your conclusion is wrong, therefore the argument must be wrong as well.
-2nawitus
And you don't provide any arguments for your claim either.. Okay, here's one: Even with time-continuos self, humans value other people, even though they personally experience anything other peope do. There's some (moral) value in other persons. Maybe people value themselves more, but that's not even relevant to the argument. So, if time-continuos self doesn't exist, people will value their future selfs as much as any other persons, which is atleast more than nothing. Of course, this assumes that such a person does value other people. May not apply to every single person.
0NihilCredo
In what ways would the world look different to me if my TCS did/did not exist?
0Aleksei_Riikonen
Can't think of any such way. Similarly, the existence or non-existence of some sorts of inactive Gods doesn't affect your observations in any way. Occam's Razor would eliminate both those Gods and a time-continuous self, though. (But personally, I propose that we may have faith in a time-continuous self, if it is sufficiently useful psychologically. And that it's an open question whether there are other supernatural things that at least some should also have faith in.)
[-]PhilGoetz20

The basic idea is sound - If you really think religion is good/bad for X, the best proof would be to run a simulation and observe outcomes for X. I interpret the downvoting to -10 as a strong collective irrational bias against religion, even in a purely instrumental context.

The corollaries are distracting.

Reply
0luminosity
Except I don't read the post as endorsing testing whether religion is good or bad for X, but rather saying that if a simulation showed we'd be better off with religious beliefs we'd be better off adopting them. There's a number of reasons why this seems like a bad idea: * First, your simulation can only predict whether religion is better for you under the circumstances that you simulate. For instance, suppose you run a simulation of the entire planet earth, but nothing much beyond it. The simulation shows better outcomes for you with religious faith. You then modify yourself to have religious faith. An extinction level asteroid comes hurtling towards earth. Previously, you would have tried to work out the best strategy to divert its course. Now you sit and pray that it goes off course instead. * Let's say you simulate yourself for twenty years under various religious beliefs, and under athiesm, and one of the simulations leads to a better outcome. You alter yourself to adopt this faith. You've now poisoned your ability to conduct similar tests in future. Perhaps a certain religion is better for the first year, or five years, or five thousand. Perhaps beyond that time it no longer is. Because you have now altered your beliefs to rest upon faith rather than upon testing you can no longer update yourself out of this religious state because you will no longer test to see whether it is optimal.
0Larks
While true, I doubt the first effect would be significant. You're not very likely to be the one responsible for saving the earth and, singularity aside, terrestrial effects are likely to be far more important to you. Contrawise, if you were capable of running a simulation, the odds of your input being relevant for existential are much higher. You might be running the simulation to help other people decide whether or not to be religious, or whether to persuade others to be religious, but then it becomes a lot more likely that the combined reduction in epistemic rationality would become an existential issue.
[-]Risto_Saarelma20

Don't some interpretations of neopagan magic have a bit of the same idea as the religion thing here? The idea is that there isn't assumed to be any supernatural woo involved, the magic rituals just exist as something that an observing human brain will really glom onto, and will end up doing stuff that it is able to but otherwise might not have done.

I think Eric S. Raymond and Alan Moore have written about magic from this outlook. Chaos magic with its concept of belief as a tool might also be relevant.

Reply
1PhilGoetz
Yes; Anders Sandberg is a paragon of rationality, but is (or was, last time I asked him about it) also a neopagan, in the way just described.
[-]KevinC00

I think the question "Is it rational to be religious?" is one that deserves critical attention and testing, but talk of ancestor simulations completely demolishes the point. Any entity capable of creating an actual ancestor simulation--a fully-modeled "Matrix" populated with genuinely human-equivalent sentient Sims--is an entity for whom the results of such a test would be irrelevant and obsolete. The premise, that some form of Faith might be useful or even necessary for rational humans to maximally act in accordance with their values... (read more)

Reply
[-]Psychohistorian00

This post definitely has problems, but given the fairly interesting discussion it appears to have prompted, does not seem to deserve being in the minus-double digits.

I think the central point is that the practical value of faith is more of an empirical question than a logical one. The central problem, of course, is that for a real person to accept some propositions on faith requires a (likely significant) dent in their overall rationality. The question is not about the value of faith, but about the tradeoffs made to obtain it; a complexity which is not re... (read more)

Reply
[-]Unknowns-40

I predict a score of around negative five for this post.

Reply
1katydee
I don't think this post should be downvoted. Testable predictions are a good idea and I would like to see more of them here, though ideally they would be posted as spoilers so that people wouldn't correct their actions to fulfill (or not fulfill) the predictions.
0NihilCredo
This particular prediction has every sign of being meant as a very indirect formulation of "This post sucks".
0katydee
I didn't read it that way, but even if it was intended as such, I think it would still be a valid comment.
Moderation Log
More from Aleksei_Riikonen
View more
Curated and popular this week
71Comments