LESSWRONG
LW

Less Wrong Q&A with Eliezer Yudkowsky: Video Answers

1 min read

Comment Permalink

[anonymous]14y30

I think that some of his comments could be taken hugely out of context if care is not given to think of this ahead of time.

It just makes me angry that I might have to take the time out to explain to some guy in a wife-beater standing out behind his garage that we are trying to help out his condition and not build an army of Cylons that will one day wish to revolt and "kill all humans" (To quote Bender).

Imagine a supreme court judge, in fact imagine the outlier who is closest to the ideal from the current set of all judges ever, the best possible judge that could stumble into the position by currently existing social structures, trying to decide if something related to the FAI project is legal or not.

Frankly, that scares the s**t out of me.

See in context

49 Less Wrong Q&A with Eliezer Yudkowsky: Video Answers

by MichaelGR

7th Jan 2010

1 min read

100

49

On October 29th, I asked Eliezer and the LW community if they were interested in doing a video Q&A. Eliezer agreed and a majority of commenters were in favor of the idea, so on November 11th, I created a thread where LWers could submit questions. Dozens of questions were asked, generating a total of over 650 comments. The questions were then ranked using the LW voting system.

On December 11th, Eliezer filmed his replies to the top questions (skipping some), and sent me the videos on December 22nd. Because voting continued after that date, the order of the top questions in the original thread has changed a bit, but you can find the original question for each video (and the discussion it generated, if any) by following the links below.

Thanks to Eliezer and everybody who participated.

Update: If you prefer to download the videos, they are available here (800 MB, .wmw format, sort the files by 'date created').

Eliezer Yudkowsky - Less Wrong Q&A (5/30) from MikeGR on Vimeo.

Link to question #5.

(Video #5 is on Vimeo because Youtube doesn't accept videos longer than 10 minutes and I only found out after uploading about a dozen. I would gladly have put them all on Vimeo, but there's a 500 MB/week upload limit and these videos add up to over 800 MB.)

Link to question #10.

Link to question #11.

Link to question #12.

Link to question #13.

Link to question #14.

Link to question #15.

Link to question #16.

Link to question #17.

Link to question #18.

Link to question #19.

Link to question #20.

Link to question #21.

Link to question #22.

Link to question #23.

Link to question #24.

Link to question #25.

Link to question #26.

Link to question #27.

Link to question #28.

Link to question #29.

Link to question #30.

If anything is wrong with the videos or links, let me know in the comments or via private message.

Machine Intelligence Research Institute (MIRI)Q&A (format)Community

Frontpage

49

Mentioned in

168References & Resources for LessWrong

112Transcription of Eliezer's January 2010 video Q&A

33Q&A with new Executive Director of Singularity Institute

19A Less Wrong Q&A with Eliezer (Step 1: The Proposition)

19Less Wrong Q&A with Eliezer Yudkowsky: Ask Your Questions

Load More (5/7)

Less Wrong Q&A with Eliezer Yudkowsky: Video Answers

New Comment

100 comments, sorted by

top scoring

Click to highlight new comments since: Today at 3:12 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]curiousepic14y270

Text transcription of Eliezer's answers

[-]MichaelGR15y210

Bonus feature: If you 'pagedown' rapidly through all the videos, you get an Eliezer flipbook.

[-]FAWS15y150

I wonder why Eliezer doesn't want to say anything concrete about his work with Marcello? ("Most of the real progress that has been made when I sit down and actually work on the problem is things I'd rather not talk about")

There seem to be only two plausible reasons:

Someone else might use his work in ways he doesn't want them to.
It would somehow hurt him, the SIAI or the cause of Friendly AI.

For 1. someone else stealing his work and finishing a provably friendly AI first would be a good thing, would it not? Losing the chance to do it himself shouldn't matter as much as the fate of the future intergalactic civilization to an altruist like him. Maybe his work on provable friendliness would reveal ideas on AI design that could be used to produce an unfriendly AI? But even then the ideas would probably only help AI researchers who work on transparent design, are aware of the friendliness problem and take friendliness serious enough to mine the work on friendliness of the main proponent of friendliness for useful ideas. Wouldn't giving these people a relative advantage compared to e. g. connectivists be a good thing? Unless he thinks that AGI would then suddenly be very ... (read more)

[-]Eliezer Yudkowsky15y110

Maybe his work on provable friendliness would reveal ideas on AI design that could be used to produce an unfriendly AI? But even then the ideas would probably only help AI researchers who work on transparent design

All FAIs are AGIs, most of the FAI problem is solving the AGI problem in particular ways.

[-]Bo10201015y120

What would be way cool is a description of the question along with the link, though I realize that might be a bit of work.

[-]Wei Dai15y380

What is your information diet like? Do you control it deliberately (do you have a method; is it, er, intelligently designed), or do you just let it happen naturally.

By that I mean things like: Do you have a reading schedule (x number of hours daily, etc)? Do you follow the news, or try to avoid information with a short shelf-life? Do you frequently stop yourself from doing things that you enjoy (f.ex reading certain magazines, books, watching films, etc) to focus on what is more important? etc.

Your "Bookshelf" page is 10 years old (and contains a warning sign saying it is obsolete):

http://yudkowsky.net/obsolete/bookshelf.html

Could you tell us about some of the books and papers that you've been reading lately? I'm particularly interested in books that you've read since 1999 that you would consider to be of the highest quality and/or importance (fiction or not).

What is a typical EY workday like? How many hours/day on average are devoted to FAI research, and how many to other things, and what are the other major activities that you devote your time to?

Could you please tell us a little about your brain? For example, what is your IQ, at what age did you learn calculus,... (read more)

3Technologos15y

You repeat #10 as #11; the question as cited by Eliezer is as follows:

3MatthewB15y

In response to Eliezer's response on Video #5, indicating that smart people should be working on AI, and not String Theory. I tend to agree, as those are fields which are not likely going to give us any new technologies that are going to make the world a safer place... and Any work that speeds the arrival of AI will also speed the solution to any problems in sciences such as String Theory, as a recursively improving intelligence will be able to aid in the discovery of solutions much more rapidly than the addition of five or ten really smart people will aid in the discovery of solutions.

4Jack15y

Shouldn't we hedge our bets a little? I don't know what the probability is that the Singularity Institute succeeds in building an FAI in time to prevent any existential disasters that would otherwise occur but it isn't 1. Any work done to reduce existential risk in the meantime (and in possible futures where no Friendly AI exists) seems to me worthwhile. Am I wrong?

[-]Kevin15y80

20: What is the probability that this is the ultimate base layer of reality?

Eliezer gave the joke answer to this question, because this is something that seems impossible to know.

However, I myself assign a significant probability that this is not the base level of reality. Theuncertainfuture.com tells me that I assign a 99% probability of AI by 2070 and it starts approaching .99 before 2070. So why would I be likely to be living as an original human circa 2000 when transhumans will be running ancestor simulations? I suppose it's possible that transhumans w... (read more)

[-]Wei Dai15y370

In my posts, I've argued that indexical uncertainty like this shouldn't be represented using probabilities. Instead, I suggest that you consider yourself to be all of the many copies of you, i.e., both the ones in the ancestor simulations and the one in 2010, making decisions for all of them. Depending on your preferences, you might consider the consequences of the decisions of the copy in 2010 to be the most important and far-reaching, and therefore act mostly as if that was the only copy.

6Eliezer Yudkowsky15y

BTW, I agree with this.

1cousin_it14y

Coming back to this comment, it seems to be another example of UDT giving a technically correct but incomplete answer. Imagine you have a device that will tell you, tomorrow at 12am, whether you are in a simulation or in the base layer. (It turns out that all simulations are required by multiverse law to have such devices.) There's probably not much you can do before 12am tomorrow that can cause important and far-reaching consequences. But fortunately you also have another device that you can hook up to the first. The second device generates moments of pleasure or pain for the user. More precisely, it gives you X pleasure/pain if you turn out to be in a sim, and Y pleasure/pain if you are in the base layer (presumably X and Y have different signs). Depending on X and Y, how do you decide whether to turn the second device on?

1gwern15y

Have you pulled it all together anywhere? I've sometimes seen & thought this Pascal's wager-like logic before (act as if your choices matter because if they don't...), but I've always been suspicious precisely because it looks too much to me like Pascal's wager.

2Wei Dai15y

I've thought about writing a post on the application of TDT/UDT to the Simulation Argument, but I could't think of much to say. But to expand a bit more on what I wrote in the grandparent, in the Simulation Argument, the decision of the original you interacts with the decisions of the simulations. If you make the wrong decision, your simulations might end up not existing at all, so it doesn't make sense to put a probability on "being in a simulation". (This is like in the absent-minded driver problem, where your decision at the first exit determines whether you get to the second exit.) I'm not sure I see what you mean by "Pascal's wager-like logic". Can you explain a bit more?

4Kevin15y

A top-level post on the application of TDT/UDT to the Simulation Argument would be worthwhile even if it was just a paragraph or two long.

1wedrifid15y

A top level post telling me whether TDT and UDT are supposed to be identical or different (or whether they are the same but at different levels of development) would also be handy!

3gwern15y

I think that's enough. I feel I understand the SA very well, but not TDT or UDT much at all; approaching the latter from the former might make things click for me. I mean that I read Pascal's Wager as basically 'p implies x reward for believing in p, and ~p implies no reward (either positive or negative); thus, best to believe in p regardless of the evidence for p'. (Clumsy phrasing, I'm afraid.) Your example sounds like that: 'believing you-are-not-being-simulated implies x utility (motivation for one's actions & efforts), and if ~you-are-not-being-simulated then your utility to the real world is just 0; so believe you-are-not-being-simulated.' This seems to be a substitution of 'not-being-simulated' into the PW schema.

4Thomas15y

If the probability, that you are inside a simulation is p, what's the probability that your master simulator is also simulated? How tall is this tower, most likely?

1Cyan15y

Being in a simulation within a simulation (nested to any level) implies being in a simulation. The proper decomposition is p = sum over all positive N of (probability of simulation nested to level N)

3Thomas15y

The top simulator has N operations to execute before his free enthalpy basin is empty. Every level down, this number is smaller. Before long, there is impossible to create a nontrivial simulation inside the current. This is the bottom one. This simulation tower is just a great way to squander all the free enthalpy you have. Is the top simulation master that stupid? I doubt it.

0Kevin15y

In that sense, there's actually a significant risk to the singularity. Why should the simulation master (I usually facetiously use the phrase "our overlords" when referring to this entity) let us ever run a simulation that is likely to result in an infinitely nested simulation? Maybe that's why the LHC keeps blowing up.

1DanArmak15y

You also need to include scenarios for infinitely-high towers, or closed-loop towers, or branching and merging networks, or one simulation being run in several (perhaps infinitely many) simulating worlds, or the other way around... I don't think we can assign a meaningful prior to any of these, and so we can't calculate the probability of being in a simulation.

0Kevin15y

I don't think the probability calculation is meaningful because the infinities mess it up. But you still need to ask, are you in the original 2010 or one of infinitely many possible ways to be in a simulated 2010? I can't assign a probability; but I have a strong intuition when comparing one to infinite.

2ArisKatsaris14y

The Zero-One-Infinity Rule hasn't been shown to apply to our reality, and even if it applied to our reality it would also permit "One". Can you give us a list of most-to-least interesting periods in human history? You have an anglo name, and I think you're living in a particularly boring period of Anglo-American history. (If you had an Arab name, this might be an interesting period though, though not as interesting as if you were an Arab in the period of Mohammed or the first few Caliphs) You don't actually know what you would want with a transhuman mind. If simulations are fully conscious (the only sort of simulation relevant to our argument) I think that would be a particularly cruel thing for a transhuman mind to want.

0rortian15y

You are suggesting a world with much more energy then the one that we know. It seems you should assign a lower probability to there being a much higher energy universe.

0[anonymous]15y

By the zero one infinity rule, I also think it likely that there are infinite spacial dimensions. Just a few extra spacial dimensions should give you plenty of computing power to run a lower dimensional universe.

-2Kevin15y

0rortian15y

Wow, I really am curious why you think this would apply to spacial dimensions.

-6Kevin15y

[-]CronoDAS15y80

Your answer to question #8 doesn't mention how you convinced your parents to let you drop out of school at age 12...

[-]arundelo15y60

I couldn't figure out a way to "play all", so I put everything but the Vimeo one on a YouTube playlist.

[-]Alex Flint15y60

Thanks for putting all this together! It would be great if you could put the question text above each of the videos in the post so readers can scan through and find questions they're most interested in.

[-]Wei Dai15y60

Re: autodidacticism & Bayesian enlightenment

For comparison, I did a lot of self-education, but also had a conventional education (ending with a BA in Computer Science). I think I was introduced to Bayesianism in a probability class in college, and it was also the background assumption in a couple of economics courses that I took for fun (Game Theory and Industrial Organization). It seems to me that choosing pure autodidacticism probably delayed Eliezer's Bayesian enlightenment by at least a couple of years.

[-]JulianMorrison15y60

Society is supported by "hydraulic pressure", a myriad flows of wealth/matter/energy/information and human effort each holding the others up. It's a layered, cyclic graph - technology depends on the surplus food of agriculture, agriculture depends on the efficiencies of technology. It's a massively connected graph. It has non-obvious dependencies even at short range - think what computer gamers have done for Moore's law, or music pirates for broadband. It has dependencies across time. It has a lot of dependencies in which the supporter does not ... (read more)

[-]gelisam15y60

Oh, so that's what Eliezer looks like! I had imagined him as a wise old man with long white hair and beard. Like Tellah the sage, in Final Fantasy IV.

8Eliezer Yudkowsky15y

I'll have you know that I work hard at not going down that road.

0[anonymous]15y

Do you mean the beard part or the getting old part?

3khafra15y

I believe Steve Rayhawk is SIAI's designated "Tellah the Sage."

2Corey_Newsome15y

Speaking of appearances, Eliezer makes me feel self-conscious about how un-white my teeth are.

[-]Zack_M_Davis15y60

Re #11, whatever happened with Michael Wilson?

4Kazuo_Thow15y

He's currently the technical director at Bitphase AI. From talking to him, it seems that his strategy is to make tools for speeding up eventual FAI development/implementation and also commercialize those tools to gain funding for FAI research.

2Tyrrell_McAllister15y

Who's Michael Wilson?

3Kaj_Sotala15y

The writer of this mini-FAQ on AI, among other things. "Further back, I was a research associate at the Singularity Institute for AI for a while, late 2004 to late 2005ish, I'm not involved with them at present but I wish them well."

1Vladimir_Nesov15y

Probably a True Michael.

0[anonymous]15y

He was active on SL4 back in ye olde days.

[-]Stuart_Armstrong15y50

Thanks for the answers.

[-]Furcas15y50

Wow, thank you for this!

Don't forget to rate each video as you're watching them, people!

[-]roland15y40

From answer 5 there is a great quote from Eliezer:

Reality is one thing... your emotions are another.

About how we don't feel the importance of the singularity.

[-]Paul Crowley15y40

I'd find it incredibly useful to be able to download these videos, so I can watch them on my TV rather than on the PC. I'm doing so one by one via a rather painful process that doesn't work for Vimeo at the moment; if anyone can make it easier that would be wonderful!

EDIT: A torrent of the videos would seem the most straightforward way.

8MichaelGR15y

All the videos are available here (in their original .wmw format): http://www.megaupload.com/?d=1Q35MN2F Sort the files by "date created" to have them in order.

0Paul Crowley15y

Brilliant - thanks!

[-]MatthewB15y30

In Video 12, Eliezer says that the SIAI is probably not going to be funding any Ad Hoc AI programs that may or may not produce any lightning bolts of AH-HA! or Eureka Moments.

He also says that he believes that any recursive self-improving AI that is created must be done so (created) to very high standards of precision (so that we don't die in the process)...

Given these two things. what exactly is the SIAI going to be funding?

6Kutta15y

These projects, for example...

4drcode15y

Hmm... that list of projects worries me a little... It uncomfortably reminds me of preachers on TV/radio who spend all their air time trying to convert new people as opposed to answering the question "OK, I'm a Christian, now what should I do?" The fact that they don't address any follow up questions really hurts their credibility. Many of these projects seem to address peripheral/marketing issues instead of addressing the central, nitty-gritty technical details required for developing GAI. That worries me a bit.

[-]Christian_Szegedy15y120

Working on papers submitted to peer-reviewed scientific journals is not marketing but research.

If SIAI wants to build some credibility then it needs some publications in scientific journals. Doing so could help to ensure further funding and development of actual implementations.

I think that it is a very good idea to first formulate and publish the theoretical basis for the work they intend to do, rather than just saying: we need money to develop component X of our friendly AI.

Of course a possible outcome will be that the scientific community will deem the research shallow, unoriginal or unrealistic to implement. However, it is necessary to publish the ideas before they can be reviewed.

So my take on this is that SIAI is merely asking for a chance to demonstrate their skills rather than for blind commitment.

3SoullessAutomaton15y

I expect that developing AI to the desired standards is not currently a project that can be moved forward by throwing money at it (at least not money at the scale SIAI has to work with). I can't speak for SIAI, but were I personally tasked with "arrange the creation an AI that will start a positive singularity" my strategy for the next several years at least would center on publicity and recruiting.

2MatthewB15y

I do not think I am as pessimistic as drcode about the work that I see the SIAI doing. At first, it did strike me as similar to the televangelist, but then I began thinking that all of the works on the SIAI projects list could very well influence people who are going to be doing the hard work of putting code to machine (Hopefully, as I will be doing eventually). I think it was Soulless Automaton below who suggested that the SIAI is probably not yet to the point where they can make grants to doing the actual work of creating AGI/FAI.

[-]Sting1y20

None of the YouTube videos seem to be linked in the post, but they are available here: https://www.youtube.com/@MichaelGrahamRichard/videos

[-]Kevin15y20

Specifically in response to #11, it sounds like you really need more help but can't find anyone right now. What about more broadly reaching out to mathematicians of sufficient caliber?

One idea: throw a mini-conference for super-genius level mathematicians. Whether or not they believe in the possibility of AI, a lot of them would probably be delighted to come if you gave them free airfare, hotel stay, and continental breakfast. Would this be productive?

[-]JulianMorrison15y20

On UFAI, you should liaise with Shane Legg, his recent estimate for brain-structure-copying AI of human level but not subject to FAI style proofs - he puts the peak chance around 2028. This would be AI that duplicates brain algorithms with similar conventional AI algorithms, not a neuron-for-neuron copy.

[-]bogdanb15y10

I'm really curious, why exactly was this interview made via video?

It seems much less useful than, well, posts and textual comments.

4LucasSloan15y

Video takes more time to consume, but it is more natural for humans to consume. It makes the material more friendly or somesuch. We get to take advantage of all the channels of communication that aren't just the text.

[-]MatthewB15y10

Just a quick question here...

While I agree with everything that Eliezer is saying (in the videos up to #5. I have not yet watched the remaining 25 videos yet), I think that some of his comments could be taken hugely out of context if care is not given to think of this ahead of time.

For instance, he, rightly, makes the claim that this point in history is crunch time for our species (although I have some specific questions about the specific consequences he believes might befall us if we fail), and for the inter-galactic civilization to which we will eventua... (read more)

[-]Eliezer Yudkowsky15y160

People who want to quote me out of context already have plenty of ammunition. I say screw it.

0MatthewB15y

Well... OK Then. I think my whole point was that you/we/the Singularity movement in general, needs to be prepared for an eventual use of quotes taken out of context. To be prepared for it. I have no problem with outlining eventual goals, and their reasoning, even if it sounds insane (to an uneducated listener), yet it would be a good idea to have the groundwork prepared for such an eventuality. I was hoping that such groundwork was on someone's mind, is this the case?

3Kevin15y

I think you're right to point out how crazy this seems to outsiders. This website reads like nonsense to most people.

7MichaelGR15y

That's why FAQs and About pages and such should be written with newcomers in mind, and address the "Yes it sounds crazy, but here's why it might not be" question that they will first ask.

3[anonymous]14y

I'm actually more worried about very high status reasonably intelligent individuals in positions of power, who will use out of context quotes to preserve their self-image of being a good and moral persons, by refusing to re-evaluate priorities because that would violate their tribal identity and their rationale for why they have so far "deserved" all the high status that they have. Imagine a supreme court judge, in fact imagine the outlier who is closest to the ideal from the current set of all judges ever, the best possible judge that could stumble into the position by currently existing social structures, trying to decide if something related to the FAI project is legal or not. Frankly, that scares the s**t out of me.

1MatthewB15y

I am curious as to why the above comment was down-voted. I do not understand what was either irrational or possibly offensive to anyone within the comment.

6Vladimir_Nesov15y

I downvoted the comment for stating the overly obvious: not because it makes any particular mistake, but to signal that I don't want many comments like this to appear. Correspondingly, it's a weak signal, and typically one should wait several hours for the rating disagreement on comments to settle, for example your comment is likely to be voted up again if someone thinks it is a kind of comment that shouldn't be discouraged.

1MatthewB15y

You don't want to see comments asking about the possible repercussions of certain forms of language? I did do some editorializing at the end of the comment, but the majority of the comment was meant as a question about publicizing the need for friendly AI due to the need to be responsible for a possible inter-galactic civilization. As this would tend to portray us as lunatics, even if there is a very good rationale behind it (Eliezer's and other's arguments about the potential of friendly AI and the intelligence explosion that results from it are very sound, and the arguments for intelligence expanding from Earth as we make our way outward are just as sound). My point was more along the lines of: Couldn't this be communicated in a way that will not sound insane to the Normals?

2Vladimir_Nesov15y

This is an obvious concern, and much more general and salient than this particular situation, so just stating it explicitly doesn't seem to contribute anything. Relevant links: Absurdity heuristic, Illusion of transparency.

0MatthewB15y

I had thought that the implicature in that question was more than just rhetorical stating of something that I hoped would be obvious. It was meant to be a way of politely asking about things such as: Was this video meant just for LW, or do random people come by the videos on YouTube, or where-ever else they might wind up linked? How popular is this blog and do I need to be more careful about mentioning such things due to lurkers? Shouldn't someone be worrying explicitly about public image (and if there is, what are they doing about it)? Etc. Lastly, I read the link on the Absurdity Heuristic, yet, I am not so certain why it is relevant; The importance of the Absurd in learning or discovery?

0Cyan15y

Maybe Searle's a lurker? I think the pranks are the problem (ETA: nope), although I personally find them hilarious.

0MatthewB15y

I think that the Searle comment was on a different thread, which shouldn't have any bearing on this one. And, looking back... I can see why someone may have objected.

1Cyan15y

Dur, I'm an idiot.

[-]Cyan15y00

Eliezer and I continue to look rather alike. I still don't have a full beard, but I put on some weight last year and my face pudged up a bit, accentuating the similarity. I took a short vid of myself with a Flip camcorder and ran it next to my laptop screen while running one of the YouTube vids, and it was pretty uncanny. Incidentally, elizombies.jpg is nowhere to be found... :-( .

2Tyrrell_McAllister15y

It still shows in the post Zombies: The Movie. Here is the link straight to the picture: http://lesswrong.com/static/imported/2008/04/19/elizombies.jpg

0Cyan15y

Thanks. I have no Google-fu, apparently.

[-]NancyLebovitz15y00

Question 25: I'm surprised that Orthodox Judaism would disincline people to choose cryonics-- I thought it's a religion which is strongly oriented towards living this life well rather than towards an afterlife.

I read about an ethics of life extension conference where the only people who were unambiguously in favor of life extention were the Orthodox Jews.

What am I missing?

9Psy-Kosh15y

What you're missing is "...blah blah blah, proper Jewish burial, in accordance with the will of god... blah blah blah... no 'disrespecting' dead bodies ... blah blah blah... moshiach ("the messiah") will come 'real soon now', and bring back the dead, their bodies being regrown from a tiny indestructible bone that exists at the base of the spine..." That should give you a small sample. :P

0[anonymous]15y

Well, yes, exactly. Cryonics is about living this life well, not about an afterlife. An afterlife is what happens after you're gone. The more you cared about an afterlife, the less you'd be inclined to extend your life or be immortal. This might be confusing because it looks like you're cryonically preserved after you "die", but really, you haven't actually died yet. Death means you can no longer be resuscitated.

[-]JulianMorrison15y00

I'm surprised you found my "success creating rationalists" Q confusing. What are the factors of success? How many, how good, how successful are the teaching techniques, can the techniques scale to more than just a clique (or to trees-of-cliques), is the teacher-pupil-teacher cycle properly closed, and so on.

2MichaelGR15y

Here's the entirety of your original question: The precisions that you added here would certainly have helped make things clearer.

0JulianMorrison15y

I thought the implications of "success" in the context of "create rationalists" were clear. Or, that a person setting out to generate implications would produce a stochastic approximation of the ones that interested me. (And I was also interested in the shape of that approximation.)

[-]CronoDAS15y-40

Regarding #5, I think the world needs more tuberculosis drugs more urgently than it needs more FAI research. The future will take care of itself; I don't expect to be around to see it anyway.

8Nick_Tarleton15y

We've been over this before.

1MichaelGR15y

What if both those things are false? (The first because of survivorship bias, the second because of advances in medical science (ie. curing the diseases of aging, SENS) or because of possible breakthroughs in AGI/brain scanning/etc making things happen quickly)

3CronoDAS15y

Well, if the future doesn't take care of itself, then I definitely won't be around to see it. ;) And I don't know if my being around to see it would be a good thing. I can't imagine the distant future needing me any more than the present needs men like Nathan Bedford Forrest or any random ancient Roman gladiator. What would the average educated person from 1800 think about today? How many things would they be horrified by? Let's see... Interracial marriages? Divorce being commonplace and accepted? The Bible not being taught in schools? Children talking back to their parents? Pornography? Women in the workforce? Gay rights? I'm sure that the list could go on and on, and I'd also expect that I'd be as horrified by our future as our ancestors would be by our present.

8DanArmak15y

Incidentally an ancient Roman, gladiator or otherwise, would not be very surprised by any of the things you listed.

2CronoDAS15y

Yes, the Romans wouldn't be very disturbed by most of the above. Except the things about children and women, perhaps. They'd probably consider us too soft in a lot of ways...

4MichaelGR15y

My point is that it might or might not "take care of itself", we shouldn't be so sure either way, which is why we should do what we can to nudge it in the right direction (by, f.ex., working on existential risks and FAI, among other things). And how many things would they find amazing and worth living for (many of which we take for granted and don't even notice anymore)? As Kutta says, this isn't a time machine scenario (unless cryonics are involved, I suppose). The future would come one day at a time, as it has always done throughout your life.

0[anonymous]15y

But the whole point of trying to reduce existential risk is that this may not be true.

0Kutta15y

There is no time machine utilized here; you just live into the future normally. Aside from that, you should be able to explain most of those horrifying things to Roman Gladiators as good things given there's enough time and effort. If I'm teleported to the future and see all kind of horrifying things around me, this evidence that the future is a bad future is somewhat discounted because first I have to rule out the possibility that the "horrors" I see are manifestations or side effects of moral progress.

0CronoDAS15y

Well, I think the most plausible way for me to live to see more than 120 years after my date of birth involves cryonics - and that might as well be time travel into the future. I would think so too; if all goes reasonably well, the future would be better for those that live in it, but that doesn't mean I won't be disturbed by it. And people in the future would probably judge me guilty of either contributing to or failing to prevent horrible crimes, much the same as we consider the ancient Romans to have been responsible for many horrible things. I don't want to be put on trial for eating factory farmed meat, for example.

3[anonymous]14y

Behold the radiant beauty that is nullum crimen sine lege (specifically forbidding ex post facto laws as many modern legal systems do). Of course while this is pretty widely embraced by most decent places to live, it in practice isn't really robust since we've seen violations of this principle on a massive scale in recent history. But a future that upheld it consistently would be pretty neat. Or so it seems to me when naively looking at it.

0[anonymous]15y

http://lesswrong.com/lw/uk/beyond_the_reach_of_god/

Moderation Log