Irrationality Game
For reasons related to Godel's incompleteness theorems and mathematically proven minimum difficulties for certain algorithms, I believe there is an upper limit on how intelligent an agent can be. (90%)
I believe that human hardware can - in principle - be as intelligent as it is possible to be. (60%) To be clear, this doesn't actually occur in the real world we currently live in.
Edit: In deference to social norms in the community, Retracted.
Upvoted for significant overconfidence on your second claim, assuming some plausible understanding of the phrase "human hardware". I'm also interested in your reasoning.
I'm not sure what you mean.
For reasons related to Godel's incompleteness theorems and mathematically proven minimum difficulties for certain algorithms, I believe there is an upper limit on how intelligent an agent can be.
We already know the upper limit on intelligence. It takes one bit of evidence to narrow down the possibilities by a factor of two.
I believe that human hardware can - in principle - be as intelligent as it is possible to be.
By "human hardware" do you mean an actual human brain, in the shape it normally forms, or just anything made out of neurons? If you mean the former, this is obviously false. We have a limited memory and thus a limited intelligence. If you mean the latter, we already know neurons are Turing complete, though you could still build a more efficient computer that does it faster and with less energy.
Do you mean that a human brain could, in principle, come very close to the upper limit of effective intelligence? That is, you might not be able to memorize 10^50 digits, but you could still answer any question you'd reasonably come across just as well?
Also, are you talking about just training a normal human, or something where their neurons have to just happen to be wired exactly right?
Also, there's the question of how to measure intelligence. Is it just how likely we are to set off a utilitronium shockwave, and how accurately it follows our goals?
This is an Irrationality Game comment.
We are living in a time of relative technological stagnation outside of computers as argued by Peter Thiel and others. 70%
If we are he is right about the reasons for this. 60%
It's mostly a definitional matter. I think we are progressing quickly in many fields, but we're mainly doing so by using computers, not by inventing new unrelated tech.
Retracted. Do not feed the trolls.
This is an Irrationality Game comment.
No less than 15% of the population could gain expected net benefits to overall wellbeing through carefully planned and executed anabolic steroid use. 80%.
This is an Irrationality Game comment.
We have not been experiencing moral progress in the past 250 years. Moral change? Sure. I'd also be ok with calling it value drift. 90%
Edit: I talked about this previously in some detail here.
Edit: Apparently the OP was a troll account, retracting all contributions to the thread.
Do you believe that there is no non-arbitrary way to define "moral progress", or you think that "moral progress" is a coherent concept, just we haven't experienced it?
(Retracted for the same reasons as other comments in this thread.)
I think moral progress is a coherent concept, I'm inclined to argue no human society so far has experience it, though obviously I can't rule out some outliers that did do so in certain time periods since this is such a huge set. we have so little data and there seems to be great variance in the kinds of values we seen in them.
This is an Irrationality Game comment. (Though I'm actually not sure how it will score).
"Moral progress" simply describes moral change or value drift in the speaker's preferred direction. Very confident (~95%).
I don't use it that way. I like lots of moral changes in the past 250 years but feel the process behind it isn't something I want to outsource morality to. Just like I like having opposable thumbs but feel uncomfortable letting evolution shape humans any further. We should do that ourselves so it doesn't grind down our complex values.
There are lots of people running around who think society in 1990 is somehow morally superior to society in 1890 on some metric of rightness beyond the similarity of their values to our own. This is the difference between someone being on the "wrong side of history" being merely a mistake in reasoning they should get over as soon as possible and it being a tragedy for them. A tragedy that perhaps kept repeating for every human society and individual in existence for nearly all of history.
This also suggests different strategies are appropriate for dealing with future moral change. I think we should be very cautious since I'm sure we don't understand the process. Modern Western civilization doesn't have narrative of "over time values became more and more like our own", but "over time morality got better and better and this gives our society meaning!". Its the difference between seeing "God guiding evolution" and confronting the full horror of Azathoth.
If you can't produce evidence that moral progress ever happened and believe that it definitely hasn't happened in the recent past, why do you think that moral progress is a coherent concept?
I didn't say I had great confidence in moral progress being a coherent concept. But it seems plausible to me that acquiring more true beliefs and thinking about them clearly might lead to discovering some values are incoherent or unreachable and thus stop pursuing them.
Hard to say, history is blurry, we do know the past 300 years well enough that I'm ok with this level certainty.
I'm far from comfortable saying that there was no moral progress in say some Medieval European societies. Not perhaps from our perspective, but from a sort of CEV-of-700 AD values looking at 1100 AD one, who knows? I don't know enough to have a reasonable estimate.
There was also useful progress in philosophy made before the "Enlightenment" that sometimes captured previous values and preferences and fixed them up. But again nearly any society for which that is true there was also lots of harmful philosophy that mutated values in responses to various pressures.
Upvoted in disagreement. The trend to moral progress has been one of less accepting of violence, less acceptance of nonconsensual interaction, less victim blaming, and less standing by while terrible things happen to others (or at least looking indignant at past instances of this).
This leads to a falsifiable prediction. In the next one to four centuries, vegetarianism will increase to a majority, jails will be seen as unnecessarily, brutally, unjustifiably harsh, "the poor" will be less of an Acceptable Target (c.f. delusions that they are "just lazy" and so on), and a condemnation of the present generation for being so terrible at donating in general and at donating to the right causes. If all of those things happen, moral progress will have been flat-out confirmed.
I don't think I should be a vegetarian. Thus at best I feel uneasy that people in four centuries thinking vegetarianism should be compulsory and at worst I'll be dismayed them spending time on activities related to that instead of things i value. If I thought that was great I'd already be vegetarian, duh.
Also I think I like some violence to be ok. Completely non-violent minds would be rather inhuman, and violence has some neat properties if viewed from the perspective of fun theory. In any case I strongly suspect the general non-violence trend (document by Pinker) in the past few thousand years was due to biological changes in humans because of our self-domestication. Your point on consent is questionable. Victim blaming as well since especially in the 20th century I would think all we saw was one set of scapegoats being swapped for another one.
This leads me to suspect Homer's FAI is probably different from my own FAI, is different from the FAI of 2400 AD values. If FAI2400 gets to play with the universe around forever, instead of FAI2012 I'd be rather pissed. Just because you see a trend line in moral change dosen't mean there is any reason to outsource your future value edits to. Isn't this the classical mistake of confusing is for should?
But if it was as you say then all our worries about CEV and FAI would be silly, since our society apparently already automagically is something very similar to what we want, we just need to figure out how to design it so that we can include emulated human minds into it while it continues working its thing.
Yay positive singularity problem solved!
In any case I strongly suspect the general non-violence trend (document by Pinker) in the past few thousand years was due to biological changes in humans because of our self-domestication.
They cite evidence of "moderate to strong heritability" of male aggressiveness. Shouldn't strong selection pressures use up variance and thus lower heritability?
Not in this case. At least not if Gregory Cochran and Henry Harpending are right and we have more new mutations tested due to a large population than we would otherwise.
This is an Irrationality Game comment.
We have not been experiencing moral progress in the past 250 years. Moral change? Sure. I'd also be ok with calling it value drift. 90%
Edit: I talked about this previously in some detail here.
Edit: Apparently the OP was a troll account, retracting all contributions to the thread.
Upvoted in disagreement. The trend to moral progress has been one of less accepting of violence, less acceptance of nonconsensual interaction, less victim blaming, and less standing by while terrible things happen to others (or at least looking indignant at past instances of this).
This leads to a falsifiable prediction. In the next one to four centuries, vegetarianism will increase to a majority, jails will be seen as unnecessarily, brutally, unjustifiably harsh, "the poor" will be less of an Acceptable Target (c.f. delusions that they are "...
I thought it would be good to play the irrationality game again. Let's do it!
Please read the post before voting on the comments, as this is a game where voting works differently.
Warning: the comments section of this post will look odd. The most reasonable comments will have lots of negative karma. Do not be alarmed, it's all part of the plan. In order to participate in this game you should disable any viewing threshold for negatively voted comments.
Here's an irrationalist game meant to quickly collect a pool of controversial ideas for people to debate and assess. It kinda relies on people being honest and not being nitpickers, but it might be fun.
Write a comment reply to this post describing a belief you think has a reasonable chance of being true relative to the the beliefs of other Less Wrong folk. Jot down a proposition and a rough probability estimate or qualitative description, like 'fairly confident'.
Example (not my true belief): "The U.S. government was directly responsible for financing the September 11th terrorist attacks. Very confident. (~95%)."
If you post a belief, you have to vote on the beliefs of all other comments. Voting works like this: if you basically agree with the comment, vote the comment down. If you basically disagree with the comment, vote the comment up. What 'basically' means here is intuitive; instead of using a precise mathy scoring system, just make a guess. In my view, if their stated probability is 99.9% and your degree of belief is 90%, that merits an upvote: it's a pretty big difference of opinion. If they're at 99.9% and you're at 99.5%, it could go either way. If you're genuinely unsure whether or not you basically agree with them, you can pass on voting (but try not to). Vote up if you think they are either overconfident or underconfident in their belief: any disagreement is valid disagreement.
That's the spirit of the game, but some more qualifications and rules follow.
If the proposition in a comment isn't incredibly precise, use your best interpretation. If you really have to pick nits for whatever reason, say so in a comment reply.
The more upvotes you get, the more irrational Less Wrong perceives your belief to be. Which means that if you have a large amount of Less Wrong karma and can still get lots of upvotes on your crazy beliefs then you will get lots of smart people to take your weird ideas a little more seriously.
Some poor soul is going to come along and post "I believe in God". Don't pick nits and say "Well in a a Tegmark multiverse there is definitely a universe exactly like ours where some sort of god rules over us..." and downvote it. That's cheating. You better upvote the guy. For just this post, get over your desire to upvote rationality. For this game, we reward perceived irrationality.
Try to be precise in your propositions. Saying "I believe in God. 99% sure." isn't informative because we don't quite know which God you're talking about. A deist god? The Christian God? Jewish?
Y'all know this already, but just a reminder: preferences ain't beliefs. Downvote preferences disguised as beliefs. Beliefs that include the word "should" are are almost always imprecise: avoid them.
That means our local theists are probably gonna get a lot of upvotes. Can you beat them with your confident but perceived-by-LW-as-irrational beliefs? It's a challenge!Additional rules:
- Generally, no repeating an altered version of a proposition already in the comments unless it's different in an interesting and important way. Use your judgement.
- If you have comments about the game, please reply to my comment below about meta discussion, not to the post itself. Only propositions to be judged for the game should be direct comments to this post.
- Don't post propositions as comment replies to other comments. That'll make it disorganized.
- You have to actually think your degree of belief is rational. You should already have taken the fact that most people would disagree with you into account and updated on that information. That means that any proposition you make is a proposition that you think you are personally more rational about than the Less Wrong average. This could be good or bad. Lots of upvotes means lots of people disagree with you. That's generally bad. Lots of downvotes means you're probably right. That's good, but this is a game where perceived irrationality wins you karma. The game is only fun if you're trying to be completely honest in your stated beliefs. Don't post something crazy and expect to get karma. Don't exaggerate your beliefs. Play fair.
- Debate and discussion is great, but keep it civil. Linking to the Sequences is barely civil -- summarize arguments from specific LW posts and maybe link, but don't tell someone to go read something. If someone says they believe in God with 100% probability and you don't want to take the time to give a brief but substantive counterargument, don't comment at all. We're inviting people to share beliefs we think are irrational; don't be mean about their responses.
- No propositions that people are unlikely to have an opinion about, like "Yesterday I wore black socks. ~80%" or "Antipope Christopher would have been a good leader in his latter days had he not been dethroned by Pope Sergius III. ~30%." The goal is to be controversial and interesting.
- Multiple propositions are fine, so long as they're moderately interesting.
- You are encouraged to reply to comments with your own probability estimates, but comment voting works normally for comment replies to other comments. That is, upvote for good discussion, not agreement or disagreement.
- In general, just keep within the spirit of the game: we're celebrating LW-contrarian beliefs for a change!