LESSWRONG
LW

Comment Permalink
Wei Dai370

In my posts, I've argued that indexical uncertainty like this shouldn't be represented using probabilities. Instead, I suggest that you consider yourself to be all of the many copies of you, i.e., both the ones in the ancestor simulations and the one in 2010, making decisions for all of them. Depending on your preferences, you might consider the consequences of the decisions of the copy in 2010 to be the most important and far-reaching, and therefore act mostly as if that was the only copy.

Reply
cousin_it10

Coming back to this comment, it seems to be another example of UDT giving a technically correct but incomplete answer.

Imagine you have a device that will tell you, tomorrow at 12am, whether you are in a simulation or in the base layer. (It turns out that all simulations are required by multiverse law to have such devices.) There's probably not much you can do before 12am tomorrow that can cause important and far-reaching consequences. But fortunately you also have another device that you can hook up to the first. The second device generates moments of ple... (read more)

Reply
6Eliezer Yudkowsky
BTW, I agree with this.
1gwern
Have you pulled it all together anywhere? I've sometimes seen & thought this Pascal's wager-like logic before (act as if your choices matter because if they don't...), but I've always been suspicious precisely because it looks too much to me like Pascal's wager.
See in context
by MichaelGR
1 min read
100

49

On October 29th, I asked Eliezer and the LW community if they were interested in doing a video Q&A. Eliezer agreed and a majority of commenters were in favor of the idea, so on November 11th, I created a thread where LWers could submit questions. Dozens of questions were asked, generating a total of over 650 comments. The questions were then ranked using the LW voting system.

On December 11th, Eliezer filmed his replies to the top questions (skipping some), and sent me the videos on December 22nd. Because voting continued after that date, the order of the top questions in the original thread has changed a bit, but you can find the original question for each video (and the discussion it generated, if any) by following the links below.

Thanks to Eliezer and everybody who participated.

Update: If you prefer to download the videos, they are available here (800 MB, .wmw format, sort the files by 'date created').

Link to question #1.

Link to question #2.

Link to question #3.

Link to question #4.

Eliezer Yudkowsky - Less Wrong Q&A (5/30) from MikeGR on Vimeo.

Link to question #5.

(Video #5 is on Vimeo because Youtube doesn't accept videos longer than 10 minutes and I only found out after uploading about a dozen. I would gladly have put them all on Vimeo, but there's a 500 MB/week upload limit and these videos add up to over 800 MB.)

Link to question #6.

Link to question #7.

Link to question #8.

Link to question #9.

Link to question #10.

Link to question #11.

Link to question #12.

Link to question #13.

Link to question #14.

Link to question #15.

Link to question #16.

Link to question #17.

Link to question #18.

Link to question #19.

Link to question #20.

Link to question #21.

Link to question #22.

Link to question #23.

Link to question #24.

Link to question #25.

Link to question #26.

Link to question #27.

Link to question #28.

Link to question #29.

Link to question #30.

If anything is wrong with the videos or links, let me know in the comments or via private message.

Machine Intelligence Research Institute (MIRI)Q&A (format)Community
Frontpage

49

Mentioned in
Load More (5/7)
Less Wrong Q&A with Eliezer Yudkowsky: Video Answers
New Comment
100 comments, sorted by
top scoring
Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]MichaelGR210

Bonus feature: If you 'pagedown' rapidly through all the videos, you get an Eliezer flipbook.

Reply
[-]FAWS150

I wonder why Eliezer doesn't want to say anything concrete about his work with Marcello? ("Most of the real progress that has been made when I sit down and actually work on the problem is things I'd rather not talk about")

There seem to be only two plausible reasons:

  1. Someone else might use his work in ways he doesn't want them to.
  2. It would somehow hurt him, the SIAI or the cause of Friendly AI.

For 1. someone else stealing his work and finishing a provably friendly AI first would be a good thing, would it not? Losing the chance to do it himself shouldn't matter as much as the fate of the future intergalactic civilization to an altruist like him. Maybe his work on provable friendliness would reveal ideas on AI design that could be used to produce an unfriendly AI? But even then the ideas would probably only help AI researchers who work on transparent design, are aware of the friendliness problem and take friendliness serious enough to mine the work on friendliness of the main proponent of friendliness for useful ideas. Wouldn't giving these people a relative advantage compared to e. g. connectivists be a good thing? Unless he thinks that AGI would then suddenly be very ... (read more)

Reply
[-]Eliezer Yudkowsky110

Maybe his work on provable friendliness would reveal ideas on AI design that could be used to produce an unfriendly AI? But even then the ideas would probably only help AI researchers who work on transparent design

All FAIs are AGIs, most of the FAI problem is solving the AGI problem in particular ways.

Reply
[-]Bo102010120

What would be way cool is a description of the question along with the link, though I realize that might be a bit of work.

Reply
[-]Wei Dai380

1.

What is your information diet like? Do you control it deliberately (do you have a method; is it, er, intelligently designed), or do you just let it happen naturally.

By that I mean things like: Do you have a reading schedule (x number of hours daily, etc)? Do you follow the news, or try to avoid information with a short shelf-life? Do you frequently stop yourself from doing things that you enjoy (f.ex reading certain magazines, books, watching films, etc) to focus on what is more important? etc.

2.

Your "Bookshelf" page is 10 years old (and contains a warning sign saying it is obsolete):

http://yudkowsky.net/obsolete/bookshelf.html

Could you tell us about some of the books and papers that you've been reading lately? I'm particularly interested in books that you've read since 1999 that you would consider to be of the highest quality and/or importance (fiction or not).

3.

What is a typical EY workday like? How many hours/day on average are devoted to FAI research, and how many to other things, and what are the other major activities that you devote your time to?

4.

Could you please tell us a little about your brain? For example, what is your IQ, at what age did you learn calculus,... (read more)

Reply
3Technologos
You repeat #10 as #11; the question as cited by Eliezer is as follows:
3MatthewB
In response to Eliezer's response on Video #5, indicating that smart people should be working on AI, and not String Theory. I tend to agree, as those are fields which are not likely going to give us any new technologies that are going to make the world a safer place... and Any work that speeds the arrival of AI will also speed the solution to any problems in sciences such as String Theory, as a recursively improving intelligence will be able to aid in the discovery of solutions much more rapidly than the addition of five or ten really smart people will aid in the discovery of solutions.
4Jack
Shouldn't we hedge our bets a little? I don't know what the probability is that the Singularity Institute succeeds in building an FAI in time to prevent any existential disasters that would otherwise occur but it isn't 1. Any work done to reduce existential risk in the meantime (and in possible futures where no Friendly AI exists) seems to me worthwhile. Am I wrong?
[-]Kevin80

20: What is the probability that this is the ultimate base layer of reality?

Eliezer gave the joke answer to this question, because this is something that seems impossible to know.

However, I myself assign a significant probability that this is not the base level of reality. Theuncertainfuture.com tells me that I assign a 99% probability of AI by 2070 and it starts approaching .99 before 2070. So why would I be likely to be living as an original human circa 2000 when transhumans will be running ancestor simulations? I suppose it's possible that transhumans w... (read more)

Reply
[-]Wei Dai370

In my posts, I've argued that indexical uncertainty like this shouldn't be represented using probabilities. Instead, I suggest that you consider yourself to be all of the many copies of you, i.e., both the ones in the ancestor simulations and the one in 2010, making decisions for all of them. Depending on your preferences, you might consider the consequences of the decisions of the copy in 2010 to be the most important and far-reaching, and therefore act mostly as if that was the only copy.

Reply
6Eliezer Yudkowsky
BTW, I agree with this.
1cousin_it
Coming back to this comment, it seems to be another example of UDT giving a technically correct but incomplete answer. Imagine you have a device that will tell you, tomorrow at 12am, whether you are in a simulation or in the base layer. (It turns out that all simulations are required by multiverse law to have such devices.) There's probably not much you can do before 12am tomorrow that can cause important and far-reaching consequences. But fortunately you also have another device that you can hook up to the first. The second device generates moments of pleasure or pain for the user. More precisely, it gives you X pleasure/pain if you turn out to be in a sim, and Y pleasure/pain if you are in the base layer (presumably X and Y have different signs). Depending on X and Y, how do you decide whether to turn the second device on?
1gwern
Have you pulled it all together anywhere? I've sometimes seen & thought this Pascal's wager-like logic before (act as if your choices matter because if they don't...), but I've always been suspicious precisely because it looks too much to me like Pascal's wager.
2Wei Dai
I've thought about writing a post on the application of TDT/UDT to the Simulation Argument, but I could't think of much to say. But to expand a bit more on what I wrote in the grandparent, in the Simulation Argument, the decision of the original you interacts with the decisions of the simulations. If you make the wrong decision, your simulations might end up not existing at all, so it doesn't make sense to put a probability on "being in a simulation". (This is like in the absent-minded driver problem, where your decision at the first exit determines whether you get to the second exit.) I'm not sure I see what you mean by "Pascal's wager-like logic". Can you explain a bit more?
4Kevin
A top-level post on the application of TDT/UDT to the Simulation Argument would be worthwhile even if it was just a paragraph or two long.
1wedrifid
A top level post telling me whether TDT and UDT are supposed to be identical or different (or whether they are the same but at different levels of development) would also be handy!
3gwern
I think that's enough. I feel I understand the SA very well, but not TDT or UDT much at all; approaching the latter from the former might make things click for me. I mean that I read Pascal's Wager as basically 'p implies x reward for believing in p, and ~p implies no reward (either positive or negative); thus, best to believe in p regardless of the evidence for p'. (Clumsy phrasing, I'm afraid.) Your example sounds like that: 'believing you-are-not-being-simulated implies x utility (motivation for one's actions & efforts), and if ~you-are-not-being-simulated then your utility to the real world is just 0; so believe you-are-not-being-simulated.' This seems to be a substitution of 'not-being-simulated' into the PW schema.
4Thomas
If the probability, that you are inside a simulation is p, what's the probability that your master simulator is also simulated? How tall is this tower, most likely?
1Cyan
Being in a simulation within a simulation (nested to any level) implies being in a simulation. The proper decomposition is p = sum over all positive N of (probability of simulation nested to level N)
3Thomas
The top simulator has N operations to execute before his free enthalpy basin is empty. Every level down, this number is smaller. Before long, there is impossible to create a nontrivial simulation inside the current. This is the bottom one. This simulation tower is just a great way to squander all the free enthalpy you have. Is the top simulation master that stupid? I doubt it.
0Kevin
In that sense, there's actually a significant risk to the singularity. Why should the simulation master (I usually facetiously use the phrase "our overlords" when referring to this entity) let us ever run a simulation that is likely to result in an infinitely nested simulation? Maybe that's why the LHC keeps blowing up.
1DanArmak
You also need to include scenarios for infinitely-high towers, or closed-loop towers, or branching and merging networks, or one simulation being run in several (perhaps infinitely many) simulating worlds, or the other way around... I don't think we can assign a meaningful prior to any of these, and so we can't calculate the probability of being in a simulation.
0Kevin
I don't think the probability calculation is meaningful because the infinities mess it up. But you still need to ask, are you in the original 2010 or one of infinitely many possible ways to be in a simulated 2010? I can't assign a probability; but I have a strong intuition when comparing one to infinite.
2ArisKatsaris
The Zero-One-Infinity Rule hasn't been shown to apply to our reality, and even if it applied to our reality it would also permit "One". Can you give us a list of most-to-least interesting periods in human history? You have an anglo name, and I think you're living in a particularly boring period of Anglo-American history. (If you had an Arab name, this might be an interesting period though, though not as interesting as if you were an Arab in the period of Mohammed or the first few Caliphs) You don't actually know what you would want with a transhuman mind. If simulations are fully conscious (the only sort of simulation relevant to our argument) I think that would be a particularly cruel thing for a transhuman mind to want.
0rortian
You are suggesting a world with much more energy then the one that we know. It seems you should assign a lower probability to there being a much higher energy universe.
0[anonymous]
By the zero one infinity rule, I also think it likely that there are infinite spacial dimensions. Just a few extra spacial dimensions should give you plenty of computing power to run a lower dimensional universe.
-2Kevin
By the zero one infinity rule, I also think it likely that there are infinite spacial dimensions. Just a few extra spacial dimensions should give you plenty of computing power to run a lower dimensional universe.
0rortian
Wow, I really am curious why you think this would apply to spacial dimensions.
-6Kevin
[-]CronoDAS80

Your answer to question #8 doesn't mention how you convinced your parents to let you drop out of school at age 12...

Reply
[-]arundelo60

I couldn't figure out a way to "play all", so I put everything but the Vimeo one on a YouTube playlist.

Reply
[-]Alex Flint60

Thanks for putting all this together! It would be great if you could put the question text above each of the videos in the post so readers can scan through and find questions they're most interested in.

Reply
[-]Wei Dai60

Re: autodidacticism & Bayesian enlightenment

For comparison, I did a lot of self-education, but also had a conventional education (ending with a BA in Computer Science). I think I was introduced to Bayesianism in a probability class in college, and it was also the background assumption in a couple of economics courses that I took for fun (Game Theory and Industrial Organization). It seems to me that choosing pure autodidacticism probably delayed Eliezer's Bayesian enlightenment by at least a couple of years.

Reply
[-]JulianMorrison60

Society is supported by "hydraulic pressure", a myriad flows of wealth/matter/energy/information and human effort each holding the others up. It's a layered, cyclic graph - technology depends on the surplus food of agriculture, agriculture depends on the efficiencies of technology. It's a massively connected graph. It has non-obvious dependencies even at short range - think what computer gamers have done for Moore's law, or music pirates for broadband. It has dependencies across time. It has a lot of dependencies in which the supporter does not ... (read more)

Reply
[-]gelisam60

Oh, so that's what Eliezer looks like! I had imagined him as a wise old man with long white hair and beard. Like Tellah the sage, in Final Fantasy IV.

Reply
8Eliezer Yudkowsky
I'll have you know that I work hard at not going down that road.
0[anonymous]
Do you mean the beard part or the getting old part?
3khafra
I believe Steve Rayhawk is SIAI's designated "Tellah the Sage."
2Corey_Newsome
Speaking of appearances, Eliezer makes me feel self-conscious about how un-white my teeth are.
[-]Zack_M_Davis60

Re #11, whatever happened with Michael Wilson?

Reply
4Kazuo_Thow
He's currently the technical director at Bitphase AI. From talking to him, it seems that his strategy is to make tools for speeding up eventual FAI development/implementation and also commercialize those tools to gain funding for FAI research.
2Tyrrell_McAllister
Who's Michael Wilson?
3Kaj_Sotala
The writer of this mini-FAQ on AI, among other things. "Further back, I was a research associate at the Singularity Institute for AI for a while, late 2004 to late 2005ish, I'm not involved with them at present but I wish them well."
1Vladimir_Nesov
Probably a True Michael.
0whpearson
He was active on SL4 back in ye olde days.
[-]Stuart_Armstrong50

Thanks for the answers.

Reply
[-]Furcas50

Wow, thank you for this!

Don't forget to rate each video as you're watching them, people!

Reply
[-]roland40

From answer 5 there is a great quote from Eliezer:

Reality is one thing... your emotions are another.

About how we don't feel the importance of the singularity.

Reply
[-]Paul Crowley40

I'd find it incredibly useful to be able to download these videos, so I can watch them on my TV rather than on the PC. I'm doing so one by one via a rather painful process that doesn't work for Vimeo at the moment; if anyone can make it easier that would be wonderful!

EDIT: A torrent of the videos would seem the most straightforward way.

Reply
8MichaelGR
All the videos are available here (in their original .wmw format): http://www.megaupload.com/?d=1Q35MN2F Sort the files by "date created" to have them in order.
0Paul Crowley
Brilliant - thanks!
[-]MatthewB30

In Video 12, Eliezer says that the SIAI is probably not going to be funding any Ad Hoc AI programs that may or may not produce any lightning bolts of AH-HA! or Eureka Moments.

He also says that he believes that any recursive self-improving AI that is created must be done so (created) to very high standards of precision (so that we don't die in the process)...

Given these two things. what exactly is the SIAI going to be funding?

Reply
6Kutta
These projects, for example...
4drcode
Hmm... that list of projects worries me a little... It uncomfortably reminds me of preachers on TV/radio who spend all their air time trying to convert new people as opposed to answering the question "OK, I'm a Christian, now what should I do?" The fact that they don't address any follow up questions really hurts their credibility. Many of these projects seem to address peripheral/marketing issues instead of addressing the central, nitty-gritty technical details required for developing GAI. That worries me a bit.
[-]Christian_Szegedy120

Working on papers submitted to peer-reviewed scientific journals is not marketing but research.

If SIAI wants to build some credibility then it needs some publications in scientific journals. Doing so could help to ensure further funding and development of actual implementations.

I think that it is a very good idea to first formulate and publish the theoretical basis for the work they intend to do, rather than just saying: we need money to develop component X of our friendly AI.

Of course a possible outcome will be that the scientific community will deem the research shallow, unoriginal or unrealistic to implement. However, it is necessary to publish the ideas before they can be reviewed.

So my take on this is that SIAI is merely asking for a chance to demonstrate their skills rather than for blind commitment.

Reply
3SoullessAutomaton
I expect that developing AI to the desired standards is not currently a project that can be moved forward by throwing money at it (at least not money at the scale SIAI has to work with). I can't speak for SIAI, but were I personally tasked with "arrange the creation an AI that will start a positive singularity" my strategy for the next several years at least would center on publicity and recruiting.
2MatthewB
I do not think I am as pessimistic as drcode about the work that I see the SIAI doing. At first, it did strike me as similar to the televangelist, but then I began thinking that all of the works on the SIAI projects list could very well influence people who are going to be doing the hard work of putting code to machine (Hopefully, as I will be doing eventually). I think it was Soulless Automaton below who suggested that the SIAI is probably not yet to the point where they can make grants to doing the actual work of creating AGI/FAI.
[-]Sting20

None of the YouTube videos seem to be linked in the post, but they are available here: https://www.youtube.com/@MichaelGrahamRichard/videos

Reply
[-]Kevin20

Specifically in response to #11, it sounds like you really need more help but can't find anyone right now. What about more broadly reaching out to mathematicians of sufficient caliber?

One idea: throw a mini-conference for super-genius level mathematicians. Whether or not they believe in the possibility of AI, a lot of them would probably be delighted to come if you gave them free airfare, hotel stay, and continental breakfast. Would this be productive?

Reply
[-]JulianMorrison20

On UFAI, you should liaise with Shane Legg, his recent estimate for brain-structure-copying AI of human level but not subject to FAI style proofs - he puts the peak chance around 2028. This would be AI that duplicates brain algorithms with similar conventional AI algorithms, not a neuron-for-neuron copy.

Reply
[-]bogdanb10

I'm really curious, why exactly was this interview made via video?

It seems much less useful than, well, posts and textual comments.

Reply
4LucasSloan
Video takes more time to consume, but it is more natural for humans to consume. It makes the material more friendly or somesuch. We get to take advantage of all the channels of communication that aren't just the text.
[-]MatthewB10

Just a quick question here...

While I agree with everything that Eliezer is saying (in the videos up to #5. I have not yet watched the remaining 25 videos yet), I think that some of his comments could be taken hugely out of context if care is not given to think of this ahead of time.

For instance, he, rightly, makes the claim that this point in history is crunch time for our species (although I have some specific questions about the specific consequences he believes might befall us if we fail), and for the inter-galactic civilization to which we will eventua... (read more)

Reply
[-]Eliezer Yudkowsky160

People who want to quote me out of context already have plenty of ammunition. I say screw it.

Reply
0MatthewB
Well... OK Then. I think my whole point was that you/we/the Singularity movement in general, needs to be prepared for an eventual use of quotes taken out of context. To be prepared for it. I have no problem with outlining eventual goals, and their reasoning, even if it sounds insane (to an uneducated listener), yet it would be a good idea to have the groundwork prepared for such an eventuality. I was hoping that such groundwork was on someone's mind, is this the case?
3Kevin
I think you're right to point out how crazy this seems to outsiders. This website reads like nonsense to most people.
7MichaelGR
That's why FAQs and About pages and such should be written with newcomers in mind, and address the "Yes it sounds crazy, but here's why it might not be" question that they will first ask.
3[anonymous]
I'm actually more worried about very high status reasonably intelligent individuals in positions of power, who will use out of context quotes to preserve their self-image of being a good and moral persons, by refusing to re-evaluate priorities because that would violate their tribal identity and their rationale for why they have so far "deserved" all the high status that they have. Imagine a supreme court judge, in fact imagine the outlier who is closest to the ideal from the current set of all judges ever, the best possible judge that could stumble into the position by currently existing social structures, trying to decide if something related to the FAI project is legal or not. Frankly, that scares the s**t out of me.
1MatthewB
I am curious as to why the above comment was down-voted. I do not understand what was either irrational or possibly offensive to anyone within the comment.
6Vladimir_Nesov
I downvoted the comment for stating the overly obvious: not because it makes any particular mistake, but to signal that I don't want many comments like this to appear. Correspondingly, it's a weak signal, and typically one should wait several hours for the rating disagreement on comments to settle, for example your comment is likely to be voted up again if someone thinks it is a kind of comment that shouldn't be discouraged.
1MatthewB
You don't want to see comments asking about the possible repercussions of certain forms of language? I did do some editorializing at the end of the comment, but the majority of the comment was meant as a question about publicizing the need for friendly AI due to the need to be responsible for a possible inter-galactic civilization. As this would tend to portray us as lunatics, even if there is a very good rationale behind it (Eliezer's and other's arguments about the potential of friendly AI and the intelligence explosion that results from it are very sound, and the arguments for intelligence expanding from Earth as we make our way outward are just as sound). My point was more along the lines of: Couldn't this be communicated in a way that will not sound insane to the Normals?
2Vladimir_Nesov
This is an obvious concern, and much more general and salient than this particular situation, so just stating it explicitly doesn't seem to contribute anything. Relevant links: Absurdity heuristic, Illusion of transparency.
0MatthewB
I had thought that the implicature in that question was more than just rhetorical stating of something that I hoped would be obvious. It was meant to be a way of politely asking about things such as: Was this video meant just for LW, or do random people come by the videos on YouTube, or where-ever else they might wind up linked? How popular is this blog and do I need to be more careful about mentioning such things due to lurkers? Shouldn't someone be worrying explicitly about public image (and if there is, what are they doing about it)? Etc. Lastly, I read the link on the Absurdity Heuristic, yet, I am not so certain why it is relevant; The importance of the Absurd in learning or discovery?
0Cyan
Maybe Searle's a lurker? I think the pranks are the problem (ETA: nope), although I personally find them hilarious.
0MatthewB
I think that the Searle comment was on a different thread, which shouldn't have any bearing on this one. And, looking back... I can see why someone may have objected.
1Cyan
Dur, I'm an idiot.
[-]Cyan00

Eliezer and I continue to look rather alike. I still don't have a full beard, but I put on some weight last year and my face pudged up a bit, accentuating the similarity. I took a short vid of myself with a Flip camcorder and ran it next to my laptop screen while running one of the YouTube vids, and it was pretty uncanny. Incidentally, elizombies.jpg is nowhere to be found... :-( .

Reply
2Tyrrell_McAllister
It still shows in the post Zombies: The Movie. Here is the link straight to the picture: http://lesswrong.com/static/imported/2008/04/19/elizombies.jpg
0Cyan
Thanks. I have no Google-fu, apparently.
[-]NancyLebovitz00

Question 25: I'm surprised that Orthodox Judaism would disincline people to choose cryonics-- I thought it's a religion which is strongly oriented towards living this life well rather than towards an afterlife.

I read about an ethics of life extension conference where the only people who were unambiguously in favor of life extention were the Orthodox Jews.

What am I missing?

Reply
9Psy-Kosh
What you're missing is "...blah blah blah, proper Jewish burial, in accordance with the will of god... blah blah blah... no 'disrespecting' dead bodies ... blah blah blah... moshiach ("the messiah") will come 'real soon now', and bring back the dead, their bodies being regrown from a tiny indestructible bone that exists at the base of the spine..." That should give you a small sample. :P
0[anonymous]
Well, yes, exactly. Cryonics is about living this life well, not about an afterlife. An afterlife is what happens after you're gone. The more you cared about an afterlife, the less you'd be inclined to extend your life or be immortal. This might be confusing because it looks like you're cryonically preserved after you "die", but really, you haven't actually died yet. Death means you can no longer be resuscitated.
[-]JulianMorrison00

I'm surprised you found my "success creating rationalists" Q confusing. What are the factors of success? How many, how good, how successful are the teaching techniques, can the techniques scale to more than just a clique (or to trees-of-cliques), is the teacher-pupil-teacher cycle properly closed, and so on.

Reply
2MichaelGR
Here's the entirety of your original question: The precisions that you added here would certainly have helped make things clearer.
0JulianMorrison
I thought the implications of "success" in the context of "create rationalists" were clear. Or, that a person setting out to generate implications would produce a stochastic approximation of the ones that interested me. (And I was also interested in the shape of that approximation.)
[-]CronoDAS-40

Regarding #5, I think the world needs more tuberculosis drugs more urgently than it needs more FAI research. The future will take care of itself; I don't expect to be around to see it anyway.

Reply
8Nick_Tarleton
We've been over this before.
1MichaelGR
What if both those things are false? (The first because of survivorship bias, the second because of advances in medical science (ie. curing the diseases of aging, SENS) or because of possible breakthroughs in AGI/brain scanning/etc making things happen quickly)
3CronoDAS
Well, if the future doesn't take care of itself, then I definitely won't be around to see it. ;) And I don't know if my being around to see it would be a good thing. I can't imagine the distant future needing me any more than the present needs men like Nathan Bedford Forrest or any random ancient Roman gladiator. What would the average educated person from 1800 think about today? How many things would they be horrified by? Let's see... Interracial marriages? Divorce being commonplace and accepted? The Bible not being taught in schools? Children talking back to their parents? Pornography? Women in the workforce? Gay rights? I'm sure that the list could go on and on, and I'd also expect that I'd be as horrified by our future as our ancestors would be by our present.
8DanArmak
Incidentally an ancient Roman, gladiator or otherwise, would not be very surprised by any of the things you listed.
2CronoDAS
Yes, the Romans wouldn't be very disturbed by most of the above. Except the things about children and women, perhaps. They'd probably consider us too soft in a lot of ways...
4MichaelGR
My point is that it might or might not "take care of itself", we shouldn't be so sure either way, which is why we should do what we can to nudge it in the right direction (by, f.ex., working on existential risks and FAI, among other things). And how many things would they find amazing and worth living for (many of which we take for granted and don't even notice anymore)? As Kutta says, this isn't a time machine scenario (unless cryonics are involved, I suppose). The future would come one day at a time, as it has always done throughout your life.
0[anonymous]
But the whole point of trying to reduce existential risk is that this may not be true.
0Kutta
There is no time machine utilized here; you just live into the future normally. Aside from that, you should be able to explain most of those horrifying things to Roman Gladiators as good things given there's enough time and effort. If I'm teleported to the future and see all kind of horrifying things around me, this evidence that the future is a bad future is somewhat discounted because first I have to rule out the possibility that the "horrors" I see are manifestations or side effects of moral progress.
0CronoDAS
Well, I think the most plausible way for me to live to see more than 120 years after my date of birth involves cryonics - and that might as well be time travel into the future. I would think so too; if all goes reasonably well, the future would be better for those that live in it, but that doesn't mean I won't be disturbed by it. And people in the future would probably judge me guilty of either contributing to or failing to prevent horrible crimes, much the same as we consider the ancient Romans to have been responsible for many horrible things. I don't want to be put on trial for eating factory farmed meat, for example.
3[anonymous]
Behold the radiant beauty that is nullum crimen sine lege (specifically forbidding ex post facto laws as many modern legal systems do). Of course while this is pretty widely embraced by most decent places to live, it in practice isn't really robust since we've seen violations of this principle on a massive scale in recent history. But a future that upheld it consistently would be pretty neat. Or so it seems to me when naively looking at it.
0[anonymous]
http://lesswrong.com/lw/uk/beyond_the_reach_of_god/
Moderation Log
100Comments