If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.

Stephen Hawking famously said that aliens are one of the main risks to human existence. In this map I will try to show all rational ways how aliens could result in human extinction. Paradoxically, even if aliens don’t exist, we may be even in bigger danger.

1.No aliens exist in our past light cone

1a. Great Filter is behind us. So Rare Earth is true. There are natural forces in our universe which are against life on Earth, but we don’t know if they are still active. We strongly underestimate such forces because of anthropic shadow. Such still active forces could be: gamma-ray bursts (and other types of cosmic explosions like magnitars), the instability of Earth’s atmosphere,  the frequency of large scale volcanism and asteroid impacts. We may also underestimate the fragility of our environment in its sensitivity to small human influences, like global warming becoming runaway global warming.

1b. Great filter is ahead of us (and it is not UFAI). Katja Grace shows that this is a much more probable solution to the Fermi paradox because of one particular version of the Doomsday argument, SIA. All technological civilizations go extinct before they become interstellar supercivilizations, that is in something like the next century on the scale of Earth’s timeline. This is in accordance with our observation that new technologies create stronger and stronger means of destruction which are available to smaller groups of people, and this process is exponential. So all civilizations terminate themselves before they can create AI, or their AI is unstable and self terminates too (I have explained elsewhere why this could happen ). 

2.      Aliens still exist in our light cone.

a)      They exist in the form of a UFAI explosion wave, which is travelling through space at the speed of light. EY thinks that this will be a natural outcome of evolution of AI. We can’t see the wave by definition, and we can find ourselves only in the regions of the Universe, which it hasn’t yet reached. If we create our own wave of AI, which is capable of conquering a big part of the Galaxy, we may be safe from alien wave of AI. Such a wave could be started very far away but sooner or later it would reach us. Anthropic shadow distorts our calculations about its probability.

b)      SETI-attack. Aliens exist very far away from us, so they can’t reach us physically (yet) but are able to send information. Here the risk of a SETI-attack exists, i.e. aliens will send us a description of a computer and a program, which is AI, and this will convert the Earth into another sending outpost. Such messages should dominate between all SETI messages. As we get stronger and stronger radio telescopes and other instruments, we have more and more chances of finding messages from them.

c)      Aliens are near (several hundred light years), and know about the Earth, so they have already sent physical space ships (or other weapons) to us, as they have found signs of our technological development and don’t want to have enemies in their neighborhood. They could send near–speed-of-light projectiles or beams of particles on an exact collision course with Earth, but this seems improbable, because if they are so near, why haven’t they didn’t reached Earth yet?

d)      Aliens are here. Alien nanobots could be in my room now, and there is no way I could detect them. But sooner or later developing human technologies will be able to find them, which will result in some form of confrontation. If there are aliens here, they could be in “Berserker” mode, i.e. they wait until humanity reaches some unknown threshold and then attack. Aliens may be actively participating in Earth’s progress, like “progressors”, but the main problem is that their understanding of a positive outcome may be not aligned with our own values (like the problem of FAI).

e)      Deadly remains and alien zombies. Aliens have suffered some kind of existential catastrophe, and its consequences will affect us. If they created vacuum phase transition during accelerator experiments, it could reach us at the speed of light without warning. If they created self-replicating non sentient nanobots (grey goo), it could travel as interstellar stardust and convert all solid matter in nanobots, so we could encounter such a grey goo wave in space. If they created at least one von Neumann probe, with narrow AI, it still could conquer the Universe and be dangerous to Earthlings. If their AI crashed it could have semi-intelligent remnants with a random and crazy goal system, which roams the Universe. (But they will probably evolve in the colonization wave of von Neumann probes anyway.) If we find their planet or artifacts they still could carry dangerous tech like dormant AI programs, nanobots or bacteria. (Vernor Vinge had this idea as the starting point of the plot in his novel “Fire Upon the Deep”)

f)       We could attract the attention of aliens by METI. Sending signals to stars in order to initiate communication we could tell potentially hostile aliens our position in space. Some people advocate for it like Zaitsev, others are strongly opposed. The risks of METI are smaller than SETI in my opinion, as our radiosignals can only reach the nearest hundreds of light years before we create our own strong AI. So we will be able repulse the most plausible ways of space aggression, but using SETI we able to receive signals from much further distances, perhaps as much as one billion light years, if aliens convert their entire home galaxy to a large screen, where they draw a static picture, using individual stars as pixels. They will use vN probes and complex algorithms to draw such picture, and I estimate that it could present messages as large as 1 Gb and will visible by half of the Universe. So SETI is exposed to a much larger part of the Universe (perhaps as much as 10 to the power of 10 more times the number of stars), and also the danger of SETI is immediate, not in a hundred years from now.

g)      Space war. During future space exploration humanity may encounter aliens in the Galaxy which are at the same level of development and it may result in classical star wars.

h)      They will not help us. They are here or nearby, but have decided not to help us in x-risks prevention, or not to broadcast (if they are far) information about most the important x-risks via SETI and about proven ways of preventing them. So they are not altruistic enough to save us from x-risks.

3. If we are in a simulation, then the owners of the simulations are aliens for us and they could switch the simulation off. Slow switch-off is possible and in some conditions it will be the main observable way of switch-off. 

4. False beliefs in aliens may result in incorrect decisions. Ronald Reagan saw something which he thought was a UFO (it was not) and he also had early onset Alzheimer’s, which may be one of the reasons he invested a lot into the creation of SDI, which also provoked a stronger confrontation with the USSR. (BTW, it is only my conjecture, but I use it as illustration how false believes may result in wrong decisions.)

5. Prevention of the x-risks using aliens:

1.      Strange strategy. If all rational straightforward strategies to prevent extinction have failed, as implied by one interpretation of the Fermi paradox, we should try a random strategy.

2.      Resurrection by aliens. We could preserve some information about humanity hoping that aliens will resurrect us, or they could return us to life using our remains on Earth. Voyagers already have such information, and they and other satellites may have occasional samples of human DNA. Radio signals from Earth also carry a lot of information.

3.      Request for help. We could send radio messages with a request for help. (Very skeptical about this, it is only a gesture of despair, if they are not already hiding in the solar system)

4.      Get advice via SETI. We could find advice on how to prevent x-risks in alien messages received via SETI.

5.      They are ready to save us. Perhaps they are here and will act to save us, if the situation develops into something really bad.

6.      We are the risk.  We will spread through the universe and colonize other planets, preventing the existence of many alien civilizations, or change their potential and perspectives permanently. So we will be the existential risk for them.

6. We are the risks for future aleins.

In total, there is several significant probability things, mostly connected with Fermi paradox solutions. No matter where is Great filter, we are at risk. If we had passed it, we live in fragile universe, but most probable conclusion is that Great Filter is very soon.

Another important thing is risks of passive SETI, which is most plausible way we could encounter aliens in near–term future.

Also there are important risks that we are in simulation, but that it is created not by our possible ancestors, but by aliens, who may have much less compassion to us (or by UFAI). In the last case the simulation be modeling unpleasant future, including large scale catastrophes and human sufferings.

The pdf is here: 

"The XAI program will focus the development of multiple systems on addressing challenges problems in two areas: (1) machine learning problems to classify events of interest in heterogeneous, multimedia data; and (2) machine learning problems to construct decision policies for an autonomous system to perform a variety of simulated missions."

"At the end of the program, the final delivery will be a toolkit library consisting of machine learning and human-computer interface software modules that could be used to develop future explainable AI systems. After the program is complete, these toolkits would be available for further refinement and transition into defense or commercial applications"

http://www.darpa.mil/program/explainable-artificial-intelligence

Here's a puzzle based on something I used to be confused about:

It is known that utility functions are equivalent (i.e. produce the same preferences over actions) up to a positive affine transformation: u'(x) = au(x) + b where a is positive.

Suppose I have u(vanilla) = 3, u(chocolate) = 8. I prefer an action that yields a 50% chance of chocolate over an action that yields a 100% chance of vanilla, because 0.5(8) > 1.0(3).

Under the positive affine transformation a = 1, b = 4; we get that u'(vanilla) = 7 and u'(chocolate) = 12. Therefore I now prefer the action that yields a 100% chance of vanilla, because 1.0(7) > 0.5(12).

How to resolve the contradiction?

"The evidence of harm would be the evidence that you can hurt some cognitive functions with the same stimulation protocols that help another cognitive function. But they're completely correct that we don't have any evidence saying you're definitely hurting yourself. We do have evidence that you're definitely changing your brain."

interview:

http://www.wbur.org/commonhealth/2016/07/11/caution-brain-hacking

Paper:

http://onlinelibrary.wiley.com/doi/10.1002/ana.24689/references

I was aware of the variability of responses to stim, but not the paper that leveraging one brain function could impair another. This was also written to give the docs some info to help inform their patients.

edit

I'll also tuck this in here, as i posted it to open thread.

Texting changes brain waves to new, previously unknown, pattern.

http://sciencebulletin.org/archives/2623.html

Makes me wonder if they were using spell check, or the new, shortend speak. By using constructed kernels, or images of words and concepts, it looks like machine learning retrieval or construction is already  being practiced here ?

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part Z: The Craft and the Community (pp. 1651-1750). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

Z. The Craft and the Community

312. Raising the Sanity Waterline - Behind every particular failure of social rationality is a larger and more general failure of social rationality; even if all religious content were deleted tomorrow from all human minds, the larger failures that permit religion would still be present. Religion may serve the function of an asphyxiated canary in a coal mine - getting rid of the canary doesn't get rid of the gas. Even a complete social victory for atheism would only be the beginning of the real work of rationalists. What could you teach people without ever explicitly mentioning religion, that would raise their general epistemic waterline to the point that religion went underwater?

313. A Sense That More Is Possible - The art of human rationality may have not been much developed because its practitioners lack a sense that vastly more is possible. The level of expertise that most rationalists strive to develop is not on a par with the skills of a professional mathematician - more like that of a strong casual amateur. Self-proclaimed "rationalists" don't seem to get huge amounts of personal mileage out of their craft, and no one sees a problem with this. Yet rationalists get less systematic training in a less systematic context than a first-dan black belt gets in hitting people.

314. Epistemic Viciousness - An essay by Gillian Russell on "Epistemic Viciousness in the Martial Arts" generalizes amazingly to possible and actual problems with building a community around rationality. Most notably the extreme dangers associated with "data poverty" - the difficulty of testing the skills in the real world. But also such factors as the sacredness of the dojo, the investment in teachings long-practiced, the difficulty of book learning that leads into the need to trust a teacher, deference to historical masters, and above all, living in data poverty while continuing to act as if the luxury of trust is possible.

315. Schools Proliferating Without Evidence - The branching schools of "psychotherapy", another domain in which experimental verification was weak (nonexistent, actually), show that an aspiring craft lives or dies by the degree to which it can be tested in the real world. In the absence of that testing, one becomes prestigious by inventing yet another school and having students, rather than excelling at any visible performance criterion. The field of hedonic psychology (happiness studies) began, to some extent, with the realization that you could measure happiness - that there was a family of measures that by golly did validate well against each other. The act of creating a new measurement creates new science; if it's a good measurement, you get good science.

316. Three Levels of Rationality Verification - How far the craft of rationality can be taken, depends largely on what methods can be invented for verifying it. Tests seem usefully stratifiable into reputational, experimental, andorganizational. A "reputational" test is some real-world problem that tests the ability of a teacher or a school (like running a hedge fund, say) - "keeping it real", but without being able to break down exactly what was responsible for success. An "experimental" test is one that can be run on each of a hundred students (such as a well-validated survey). An "organizational" test is one that can be used to preserve the integrity of organizations by validating individuals or small groups, even in the face of strong incentives to game the test. The strength of solution invented at each level will determine how far the craft of rationality can go in the real world.

317. Why Our Kind Can't Cooperate - The atheist/libertarian/technophile/sf-fan/early-adopter/programmer/etc crowd, aka "the nonconformist cluster", seems to be stunningly bad at coordinating group projects. There are a number of reasons for this, but one of them is that people are as reluctant to speak agreement out loud, as they are eager to voice disagreements - the exact opposite of the situation that obtains in more cohesive and powerful communities. This is not rational either! It is dangerous to be half a rationalist (in general), and this also applies to teaching only disagreement but not agreement, or only lonely defiance but not coordination. The pseudo-rationalist taboo against expressing strong feelings probably doesn't help either.

318. Tolerate Tolerance - One of the likely characteristics of someone who sets out to be a "rationalist" is a lower-than-usual tolerance for flawed thinking. This makes it very important to tolerate other people's tolerance - to avoid rejecting them because they tolerate people you wouldn't - since otherwise we must all have exactly the same standards of tolerance in order to work together, which is unlikely. Even if someone has a nice word to say about complete lunatics and crackpots - so long as they don't literally believe the same ideas themselves - try to be nice to them? Intolerance of tolerance corresponds to punishment of non-punishers, a very dangerous game-theoretic idiom that can lock completely arbitrary systems in place even when they benefit no one at all.

319. Your Price for Joining - The game-theoretical puzzle of the Ultimatum game has its reflection in a real-world dilemma: How much do you demand that an existing group adjust toward you, before you will adjust toward it? Our hunter-gatherer instincts will be tuned to groups of 40 with very minimal administrative demands and equal participation, meaning that we underestimate the inertia of larger and more specialized groups and demand too much before joining them. In other groups this resistance can be overcome by affective death spirals and conformity, but rationalists think themselves too good for this - with the result that people in the nonconformist cluster often set their joining prices way way way too high, like an 50-way split with each player demanding 20% of the money. Nonconformists need to move in the direction of joining groups more easily, even in the face of annoyances and apparent unresponsiveness. If an issue isn't worth personally fixing by however much effort it takes, it's not worth a refusal to contribute.

320. Can Humanism Match Religion's Output? - Anyone with a simple and obvious charitable project - responding with food and shelter to a tidal wave in Thailand, say - would be better off by far pleading with the Pope to mobilize the Catholics, rather than with Richard Dawkins to mobilize the atheists. For so long as this is true, any increase in atheism at the expense of Catholicism will be something of a hollow victory, regardless of all other benefits. Can no rationalist match the motivation that comes from the irrational fear of Hell? Or does the real story have more to do with the motivating power of physically meeting others who share your cause, and group norms of participating?

321. Church vs. Taskforce - Churches serve a role of providing community - but they aren't explicitly optimized for this, because their nominal role is different. If we desire community without church, can we go one better in the course of deleting religion? There's a great deal of work to be done in the world; rationalist communities might potentially organize themselves around good causes, while explicitly optimizing for community.

322. Rationality: Common Interest of Many Causes - Many causes benefit particularly from the spread of rationality - because it takes a little more rationality than usual to see their case, as a supporter, or even just a supportive bystander. Not just the obvious causes like atheism, but things like marijuana legalization. In the case of my own work this effect was strong enough that after years of bogging down I threw up my hands and explicitly recursed on creating rationalists. If such causes can come to terms with not individually capturing all the rationalists they create, then they can mutually benefit from mutual effort on creating rationalists. This cooperation may require learning to shut up about disagreements between such causes, and not fight over priorities, except in specialized venues clearly marked.

323. Helpless Individuals - When you consider that our grouping instincts are optimized for 50-person hunter-gatherer bands where everyone knows everyone else, it begins to seem miraculous that modern-day large institutions survive at all. And in fact, the vast majority of large modern-day institutions simply fail to exist in the first place. This is why funding of Science is largely through money thrown at Science rather than donations from individuals - research isn't a good emotional fit for the rare problems that individuals can manage to coordinate on. In fact very few things are, which is why e.g. 200 million adult Americans have such tremendous trouble supervising the 535 members of Congress. Modern humanity manages to put forth very little in the way of coordinated individual effort to serve our collective individual interests.

324. Money: The Unit of Caring - Omohundro's resource balance principle implies that the inside of any approximately rational system has a common currency of expected utilons. In our world, this common currency is called "money" and it is the unit of how much society cares about something - a brutal yet obvious point. Many people, seeing a good cause, would prefer to help it by donating a few volunteer hours. But this avoids the tremendous gains of comparative advantage, professional specialization, and economies of scale - the reason we're not still in caves, the only way anything ever gets done in this world, the tools grownups use when anyone really cares. Donating hours worked within a professional specialty and paying-customer priority, whether directly, or by donating the money earned to hire other professional specialists, is far more effective than volunteering unskilled hours.

325. Purchase Fuzzies and Utilons Separately - Wealthy philanthropists typically make the mistake of trying to purchase warm fuzzy feelings, status among friends, and actual utilitarian gains, simultaneously; this results in vague pushes along all three dimensions and a mediocre final result. It should be far more effective to spend some money/effort on buying altruistic fuzzies at maximum optimized efficiency (e.g. by helping people in person and seeing the results in person), buying status at maximum efficiency (e.g. by donating to something sexy that you can brag about, regardless of effectiveness), and spending most of your money on expected utilons (chosen through sheer cold-blooded shut-up-and-multiply calculation, without worrying about status or fuzzies).

326. Bystander Apathy - The bystander effect is when groups of people are less likely to take action than an individual. There are a few explanations for why this might be the case.

327. Collective Apathy and the Internet - The causes of bystander apathy are even worse on the Internet. There may be an opportunity here for a startup to deliberately try to avert bystander apathy in online group coordination.

328. Incremental Progress and the Valley - The optimality theorems for probability theory and decision theory, are for perfect probability theory and decision theory. There is no theorem that incremental changes toward the ideal, starting from a flawed initial form, must yield incremental progress at each step along the way. Since perfection is unattainable, why dare to try for improvement? But my limited experience with specialized applications suggests that given enough progress, one can achieve huge improvements over baseline - it just takes a lot of progress to get there.

329. Bayesians vs. Barbarians - Suppose that a country of rationalists is attacked by a country of Evil Barbarians who know nothing of probability theory or decision theory. There's a certain concept of "rationality" which says that the rationalists inevitably lose, because the Barbarians believe in a heavenly afterlife if they die in battle, while the rationalists would all individually prefer to stay out of harm's way. So the rationalist civilization is doomed; it is too elegant and civilized to fight the savage Barbarians... And then there's the idea that rationalists should be able to (a) solve group coordination problems, (b) care a lot about other people and (c) win...

330. Beware of Other-Optimizing - Aspiring rationalists often vastly overestimate their own ability to optimize other people's lives. They read nineteen webpages offering productivity advice that doesn't work for them... and then encounter the twentieth page, or invent a new method themselves, and wow, it really works - they've discovered the true method. Actually, they've just discovered the one method in twenty that works for them, and their confident advice is no better than randomly selecting one of the twenty blog posts. Other-Optimizing is exceptionally dangerous when you have power over the other person - for then you'll just believe that they aren't trying hard enough.

331. Practical Advice Backed by Deep Theories - Practical advice is genuinely much, much more useful when it's backed up by concrete experimental results, causal models that are actually true, or valid math that is validly interpreted. (Listed in increasing order of difficulty.) Stripping out the theories and giving the mere advice alone wouldn't have nearly the same impact or even the same message; and oddly enough, translating experiments and math into practical advice seems to be a rare niche activity relative to academia. If there's a distinctive LW style, this is it.

332. The Sin of Underconfidence - When subjects know about a bias or are warned about a bias, overcorrection is not unheard of as an experimental result. That's what makes a lot of cognitive subtasks so troublesome - you know you're biased but you're not sure how much, and if you keep tweaking you may overcorrect. The danger of underconfidence (overcorrecting for overconfidence) is that you pass up opportunities on which you could have been successful; not challenging difficult enough problems; losing forward momentum and adopting defensive postures; refusing to put the hypothesis of your inability to the test; losing enough hope of triumph to try hard enough to win. You should ask yourself "Does this way of thinking make me stronger, or weaker?"

333. Go Forth and Create the Art! - I've developed primarily the art of epistemic rationality, in particular, the arts required for advanced cognitive reductionism... arts like distinguishing fake explanations from real ones and avoiding affective death spirals. There is much else that needs developing to create a craft of rationality - fighting akrasia; coordinating groups; teaching, training, verification, and becoming a proper experimental science; developing better introductory literature... And yet it seems to me that there is a beginning barrier to surpass before you can start creating high-quality craft of rationality, having to do with virtually everyone who tries to think lofty thoughts going instantly astray, or indeed even realizing that a craft of rationality exists and that you ought to be studying cognitive science literature to create it. It's my hope that my writings, as partial as they are, will serve to surpass this initial barrier. The rest I leave to you.

This has been a collection of notes on the assigned sequence for this fortnight. The most important part of the reading group though is discussion, which is in the comments section. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

This is the end, beautiful friend!

on the 22 August 2015, I wrote an apps list of useful apps, in the comments were a number of suggestions which I immediately tried.  This is an update.  Original can be found at this link:

http://lesswrong.com/r/discussion/lw/mnm/a_list_of_apps_that_are_useful_to_me_and_other/

I rewrite the whole list below.  

But first - my recommended list in short:

• Get an external battery block (and own more than enough spare power cables)
• Wunderlist
• Ingress
• How are you feeling?
• Alarm clock plus
• Twilight
• Business calendar
• Clipper
• Rain alarm
• Data monitor
• Rescuetime
• Powercalc
• Es File viewer
• WheresmyDroid?
• Google Docs/sheets etc.
• (possibly pushbullet and DTG GTD but I have not had them for long enough)

New:

Timestamp Widget. - on clicking to open it - it logs a timestamp.  Can include notes too.

Wunderlist - Recommend it - for shared shopping lists, or any kind of list of things to do.  It's not perfect but it works.

T2 mood tracker - as a second backup to my other mood tracker.  This one takes more effort to do so I only enter the data every few days.  YMMV it might be useful to you.

HOVBX - an overlay for google hangouts that sits on top of the call buttons so you don't accidentally call people (useful for groups who butt-dial each other)

Fleksy - A different keyboard - it seems faster but I am used to swiftkey so I don't use this one.

Tagtime - useful to try.  reminds you hourly or so to tag what you are currently working on.  I used it for a while to help keep me on track.  I noticed I was significantly off track and eventually stopped using it because I felt bad about it.  I feel like I spend more time on-task now but because I want to.  This was a step in the journey of deciding to do that.

Alarm clock plus - it's the best alarm clock app.  I don't use alarms often but this one does everything.

Squats/Push ups/sit ups/pull ups - Rittr labs - good at a simple exercise routine.  Just tells you what to do.  designed to get you from zero to "up to N" of an exercise (250 or 100) so gives you instruction on how many to do each day.  Worth trying.  Didn't work for me, but for other reasons about my lifestyle.

Twilight - mentioned above, replaces night mode and does what f.lux with a PC (filters to be less blue at night)

World clock - started talking to people in different time zones and this was handy.

CPU-Z - lists out all the phone's sensors and tells you their outputs.  cool for looking at gyroscopes/accelerometers.

Coffee meets bagel - dating app.  One profile per day, accept/reject.  Has a different feel to tinder

Bumble - US only; Like Tinder but the girl has to message you first or the connection disappears.

Business Calendar - Best calendar I have found so far

Clipper - Clipboard app for holding the last 20 or so things you have copied.  Also for showing you what's currently on the "copy" 

Pixlr - photo editor.  It's a good one, don't use it often

Rain Alarm - Very good app.  Tells you if it's raining anywhere nearby.  Can be enough to tell you "I should walk home sooner" but also just interesting to have a bit more awareness of your environment.

Audio Scope - Cool science app for viewing the audio scope

Spectrum analyze - Cool science app for viewing the audio spectrum

Frequensee - Fun science app for viewing audio spectrum data

PitchLab lite - Neat for understanding pitch when singing or listening to musical notes.  Another science-visualisation app

Spectralview analyser - another spectrum analyser

Pulsepoint AED - Initiative to gather a public map of all AED's worldwide.  To help; get the app and check the details of nearby AED's

FBreader - Ebook reader.  Pretty good, can control brightness and font size.

KIK - Social app like whatsapp/viber etc.  Don't use it yet, got it on a recommendation.

Wildwilderness - Reporting app for if you see suspicious wildlife trade going on anywhere in the world.  Can report anonymously, any details help.  

DGT GTD - Newly suggested by LW, have not tried to use it yet

Pushbullet - Syncs phone notifications with your PC so you can access things via PC.

I have noticed I often wish "Damn I wish someone had made an app for that" and when I search for it I can't find it.  Then I outsource the search to facebook or other people; and they can usually say - yes, its called X.  Which I can put down to an inability to know how to search for an app on my part; more than anything else.

With that in mind; I wanted to solve the problem of finding apps for other people.

The following is a list of apps that I find useful (and use often) for productive reasons:

The environment

This list is long.  The most valuable ones are the top section that I use regularly.  

Other things to mention:

Internal storage - I have a large internal memory card because I knew I would need lots of space.  So I played the "out of sight out of mind game" and tried to give myself as much space as possible by buying a large internal card.  The future of phones is to not use a microSD card and just use internal storage.  I was taking 1000 photos a month, and since having storage troubles and my phone slowing down I don't take nearly even 1 photo a day.  I would like to change that and will probably make it a future bug of mine to solve.

Battery - I use anker external battery blocks to save myself the trouble of worrying about batteries.  If prepared I leave my house with 2 days of phone charge (of 100% use).  I used to count "wins" of days I beat my phone battery (stay awake longer than it) but they are few and far between.  Also I doubled my external battery power and it sits at two days not one (28000mA + 2*460ma spare phone batteries) This is still true but those batteries don't do what they used to.  Anker have excellent service and refunded the battery that did not stay strong.  I would recommend to all phone users to have a power block.  Phones just are not made with enough battery.

Phone - I have a Samsung S4 (android Running KitKat) because it has a few features I found useful that were not found in many other phones - Cheap, Removable battery, external storage card, replaceable case. I am now on lolipop, and have made use of the external antenna port for a particularly bad low-signal location.

Screen cover - I am using the one that came with the phone still Still

I carry a spare phone case, in the beginning I used to go through one each month; now I have a harder case than before it hasn't broken. I change phone case colours for aesthetics every few months.

I also have swapped out the plastic frame that holds the phone case on as these broke, it was a few dollars on ebay and I needed a teeny screwdriver but other than that it works great now!

MicroUSB cables - I went through a lot of effort to sort this out, it's still not sorted, but its "okay for now".  The advice I have - buy several good cables (read online reviews about it), test them wherever possible, and realise that they die.  Also carry a spare or two.  I have now spent far too much time on this problem.  I am at the end of my phone's life and the MicroUSB port is dying, I have replaced it with a new one which is also not great, and I now leave my phone plugged into it's microUSB cable.  I now use Anker brand cabled which are excellent, but my phone still kills one every few weeks.  The whole idea of the MicroUSB plug is awful.  They don't work very well at all.

Restart - I restart my phone probably most days when it gets slow.  It's got programming bugs, but this solution works for now.

The overlays

These sit on my screen all the time.

Data monitor - Gives an overview of bits per second upload or download. updated every second. ✓

CpuTemp - Gives an overlay of the current core temperature.  My phone is always hot, I run it hard with bluetooth, GPS and wifi blaring all the time.  I also have a lot of active apps. ✓

M̶i̶n̶d̶f̶u̶l̶n̶e̶s̶s̶ ̶b̶e̶l̶l̶ ̶-̶ ̶M̶y̶ ̶p̶h̶o̶n̶e̶ ̶m̶a̶k̶e̶s̶ ̶a̶ ̶c̶h̶i̶m̶e̶ ̶e̶v̶e̶r̶y̶ ̶h̶a̶l̶f̶ ̶h̶o̶u̶r̶ ̶t̶o̶ ̶r̶e̶m̶i̶n̶d̶ ̶m̶e̶ ̶t̶o̶ ̶c̶h̶e̶c̶k̶,̶ ̶"̶A̶m̶ ̶I̶ ̶d̶o̶i̶n̶g̶ ̶s̶o̶m̶e̶t̶h̶i̶n̶g̶ ̶o̶f̶ ̶h̶i̶g̶h̶-̶v̶a̶l̶u̶e̶ ̶r̶i̶g̶h̶t̶ ̶n̶o̶w̶?̶"̶ ̶i̶t̶ ̶s̶o̶m̶e̶t̶i̶m̶e̶s̶ ̶s̶t̶o̶p̶s̶ ̶m̶e̶ ̶f̶r̶o̶m̶ ̶d̶o̶i̶n̶g̶ ̶c̶r̶a̶p̶ ̶t̶h̶i̶n̶g̶s̶.̶ Wow that didn't last.  It was so annoying that I stopped using it.

Facebook chat heads - I often have them open, they have memory leaks and start slowing down my phone after a while, I close and reopen them when I care enough.✓ memory leaks improved but are still there.

The normals:

Facebook - communicate with people.  I do this a lot.✓

Inkpad - its a note-taking app, but not an exceptionally great one; open to a better suggestion.✓

Ingress - it makes me walk; it gave me friends; it put me in a community.  Downside is that it takes up more time than you want to give it.  It's a mobile GPS game.  Join the Resistance. Highly recommend

Maps (google maps) - I use this most days; mostly for traffic assistance to places that I know how to get to.✓

Camera - I take about 1000 photos a month.  Generic phone-app one. I take significantly less photos now, my phone slowed down so the activation energy for *open the camera* is higher.  I plan to try to fix this soon

Assistive light - Generic torch app (widget) I use this daily.✓

Hello - SMS app.  I don't like it but its marginally better than the native one.✓

S̶u̶n̶r̶i̶s̶e̶ ̶c̶a̶l̶e̶n̶d̶a̶r̶ ̶-̶ ̶I̶ ̶d̶o̶n̶'̶t̶ ̶l̶i̶k̶e̶ ̶t̶h̶e̶ ̶n̶a̶t̶i̶v̶e̶ ̶c̶a̶l̶e̶n̶d̶a̶r̶;̶ ̶I̶ ̶d̶o̶n̶'̶t̶ ̶l̶i̶k̶e̶ ̶t̶h̶i̶s̶ ̶o̶r̶ ̶a̶n̶y̶ ̶o̶t̶h̶e̶r̶ ̶c̶a̶l̶e̶n̶d̶a̶r̶.̶ ̶ ̶T̶h̶i̶s̶ ̶i̶s̶ ̶t̶h̶e̶ ̶l̶e̶a̶s̶t̶ ̶b̶a̶d̶ ̶o̶n̶e̶ ̶I̶ ̶h̶a̶v̶e̶ ̶f̶o̶u̶n̶d̶.̶ ̶ ̶I̶ ̶h̶a̶v̶e̶ ̶a̶n̶ ̶a̶p̶p̶ ̶c̶a̶l̶l̶e̶d̶ ̶"̶f̶a̶c̶e̶b̶o̶o̶k̶ ̶s̶y̶n̶c̶"̶ ̶w̶h̶i̶c̶h̶ ̶h̶e̶l̶p̶s̶ ̶w̶i̶t̶h̶ ̶e̶n̶t̶e̶r̶i̶n̶g̶ ̶i̶n̶ ̶a̶ ̶f̶r̶a̶c̶t̶i̶o̶n̶ ̶o̶f̶ ̶t̶h̶e̶ ̶e̶v̶e̶n̶t̶s̶ ̶i̶n̶ ̶m̶y̶ ̶l̶i̶f̶e̶.̶

Business Calendar - works better, has a better interface than Sunrise.

Phone, address book, chrome browser.✓  I use tab sync, and recommend it for all your chrome-enabled devices.

GPS logger - I have a log of my current gps location every 5 minutes.  If google tracks me I might as well track myself.  I don't use this data yet but its free for me to track; so if I can find a use for the historic data that will be a win. I don't make use of this data and can access my google data just fine so I might stop tracking this.

Quantified apps:

Fit - google fit; here for multiple redundancy✓

S Health - Samsung health - here for multiple redundancy✓

Fitbit - I wear a flex step tracker every day, and input my weight daily manually through this app✓

Basis - I wear a B1 watch, and track my sleep like a hawk.✓

Rescuetime - I track my hours on technology and wish it would give a better breakdown. (I also paid for their premium service)✓

Voice recorder - generic phone app; I record around 1-2 hours of things I do per week.  Would like to increase that. I now use this for one hour a month or less.

Narrative - I recently acquired a life-logging device called a narrative, and don't really know how to best use the data it gives.  But its a start. I tried using the device but it has poor battery life.  I also received negative feedback when wearing it in casual settings.  This increases the activation energy to using it.  I also can't seem to wear it at the right height and would regularly take photos of the tops of people's heads.  I would come home with a photo a minute for a day (and have the battery die on it a few times) and have one use-able photo in the lot.  significantly lower than I was expecting.

How are you feeling? - Mood tracking app - this one is broken but the best one I have found, it doesn't seem to open itself after a phone restart; so it won't remind you to enter in a current mood.  I use a widget so that I can enter in the mood quickly.  The best parts of this app are the way it lets you zoom out, and having a 10 point scale.  I used to write a quick sentence about what I was feeling, but that took too much time so I stopped doing it. Highly recommend I use this every day.

Stopwatch - "hybrid stopwatch" - about once a week I time something and my phone didn't have a native one.  This app is good at being a stopwatch.✓

Callinspector - tracks ingoing or outgoing calls and gives summaries of things like, who you most frequently call, how much data you use, etc.  can also set data limits. I dont do anything with this data so I think I will stop using it and save my phone's battery life.

Misc

Powercalc - the best calculator app I could find ✓

N̶i̶g̶h̶t̶ ̶m̶o̶d̶e̶ ̶-̶ ̶f̶o̶r̶ ̶s̶a̶v̶i̶n̶g̶ ̶b̶a̶t̶t̶e̶r̶ ̶(̶i̶t̶ ̶d̶i̶m̶s̶ ̶y̶o̶u̶r̶ ̶s̶c̶r̶e̶e̶n̶)̶,̶ ̶I̶ ̶d̶o̶n̶'̶t̶ ̶u̶s̶e̶ ̶t̶h̶i̶s̶ ̶o̶f̶t̶e̶n̶ ̶b̶u̶t̶ ̶i̶t̶ ̶i̶s̶ ̶g̶o̶o̶d̶ ̶a̶t̶ ̶w̶h̶a̶t̶ ̶i̶t̶ ̶d̶o̶e̶s̶.̶ ̶ ̶I̶ ̶w̶o̶u̶l̶d̶ ̶c̶o̶n̶s̶i̶d̶e̶r̶ ̶a̶n̶ ̶a̶p̶p̶ ̶t̶h̶a̶t̶ ̶d̶i̶m̶s̶ ̶t̶h̶e̶ ̶b̶l̶u̶e̶ ̶l̶i̶g̶h̶t̶ ̶e̶m̶i̶t̶t̶e̶d̶ ̶f̶r̶o̶m̶ ̶m̶y̶ ̶s̶c̶r̶e̶e̶n̶;̶ ̶h̶o̶w̶e̶v̶e̶r̶ ̶I̶ ̶d̶o̶n̶'̶t̶ ̶n̶o̶t̶i̶c̶e̶ ̶a̶n̶y̶ ̶n̶e̶g̶a̶t̶i̶v̶e̶ ̶s̶l̶e̶e̶p̶ ̶e̶f̶f̶e̶c̶t̶s̶ ̶s̶o̶ ̶I̶ ̶h̶a̶v̶e̶ ̶b̶e̶e̶n̶ ̶p̶u̶t̶t̶i̶n̶g̶ ̶o̶f̶f̶ ̶g̶e̶t̶t̶i̶n̶g̶ ̶a̶r̶o̶u̶n̶d̶ ̶t̶o̶ ̶i̶t̶.̶ ̶

Advanced signal status - about once a month I am in a place with low phone signal - this one makes me feel better about knowing more details of what that means.✓

Ebay - To be able to buy those $5 solutions to problems on the spot is probably worth more than $5 of "impulse purchases" that they might be classified as.✓

C̶a̶l̶ ̶-̶ ̶a̶n̶o̶t̶h̶e̶r̶ ̶c̶a̶l̶e̶n̶d̶a̶r̶ ̶a̶p̶p̶ ̶t̶h̶a̶t̶ ̶s̶o̶m̶e̶t̶i̶m̶e̶s̶ ̶c̶a̶t̶c̶h̶e̶s̶ ̶e̶v̶e̶n̶t̶s̶ ̶t̶h̶a̶t̶ ̶t̶h̶e̶ ̶f̶i̶r̶s̶t̶ ̶o̶n̶e̶ ̶m̶i̶s̶s̶e̶s̶.̶ Nope just using business calendar now.

ES file explorer - for searching the guts of my phone for files that are annoying to find.  Not as used or as useful as I thought it would be but still useful.✓

Maps.Me - I went on an exploring adventure to places without signal; so I needed an offline mapping system.  This map saved my life.✓ Have not used this since then, but I will not delete it.

Wikipedia - information lookup✓

Youtube - don't use it often, but its there.✓

How are you feeling? (again) - I have this in multiple places to make it as easy as possible for me to enter in this data✓

Play store - Makes it easy to find.✓

Gallery - I take a lot of photos, but this is the native gallery and I could use a better app.✓

Social

In no particular order;

F̶a̶c̶e̶b̶o̶o̶k̶ ̶g̶r̶o̶u̶p̶s̶ was so annoying I got rid of it, Yahoo Mail, Skype, Facebook Messenger chat heads, Whatsapp, meetup, google+, Hangouts, Slack, Viber, OKcupid, Gmail, Tinder, Chatango, CoffeeMeetsBagel, Signal.  Of which I use very little.

They do social things.  

I don't really use:  Viber, OKC, Gmail, Tinder, Chatango, CMB, Signal, whatsapp, G+.

I use: Slack, Facebook messenger, yahoo mail  every day.

Not used:

(ticks here mean they are still in this category and are not used)

Trello✓

Workflowy✓

pocketbook✓

snapchat Deleted.

AnkiDroid - Anki memoriser app for a phone. ✓

MyFitnessPal - looks like a really good app, have not used it ✓

Fitocracy - looked good✓

I got these apps for a reason; but don't use them.

Not on my front pages:

These I don't use as often; or have not moved to my front pages (skipping the ones I didn't install or don't use)

S memo - samsung note taking thing, I rarely use, but do use once a month or so.✓

Drive, Docs, Sheets - The google package.  Its terrible to interact with documents on your phone, but I still sometimes access things from my phone.✓Useful for viewing, not effective for editing.

bubble - I don't think I have ever used this Deleted

Compass pro - gives extra details about direction. I never use it.Deleted

(ingress apps) Glypher, Agentstats, integrated timer, cram, notify Don't use them, but still there

TripView (public transport app for my city) Deleted

Convertpad - converts numbers to other numbers. Sometimes quicker than a google search.✓

ABC Iview - National TV broadcasting channel app.  Every program on this channel is uploaded to this app, I have used it once to watch a documentary since I got the app. Deleted

AnkiDroid - I don't need to memorise information in the way it is intended to be used; so I don't use it. Cram is also a flashcard app but I don't use it. Not used

First aid - I know my first aid but I have it anyway for the marginal loss of 50mb of space. Still haven't used it once.

Triangle scanner - I can scan details from NFC chips sometimes. Still haven't used it once.

MX player - does videos better than native apps. Rarely used

Zarchiver - Iunno.  Does something.  Rarely used

Pandora - Never used Deleted

Soundcloud - used once every two months, some of my friends post music online.  Deleted - They have a web interface.

Barcode scanner - never used

Diskusage - Very useful.  Visualises where data is being taken up on your phone, helps when trying to free up space.✓

Swiftkey - Better than native keyboards.  Gives more freedom, I wanted a keyboard with black background and pale keys, swiftkey has it.✓

Google calendar - don't use it, but its there to try to use.✓

Sleepbot - doesn't seem to work with my phone, also I track with other methods, and I forget to turn it on; so its entirely not useful in my life for sleep tracking. Deleted

My service provider's app.

AdobeAcrobat - use often; not via the icon though. ✓

Wheresmydroid? - seems good to have; never used.  My phone is attached to me too well for me to lose it often.  I have it open most of the waking day maybe. ✓ I actually set this up and tested if it worked.  It doesn't work from install, needs an account (which I now have) make sure you actually have an account

Uber - I don't use ubers. Deleted

Terminal emulator, AIDE, PdDroid party, Processing Android, An editor for processing, processing reference, learn C++ - programming apps for my phone, I don't use them, and I don't program much. Deleted some to make space on my phone.

Airbnb - Have not used yet, done a few searches for estimating prices of things. Deleted - Web interface better.

Heart rate - measures your heart rate using the camera/flash.  Neat, not useful other than showing off to people how its possible to do. ✓

Basis - (B1 app), - has less info available than their new app. ✓

BPM counter - Neat if you care about what a "BPM" is for music.  Don't use often. ✓

Sketch guru - fun to play with, draws things. ✓

DJ studio 5 - I did a dj thing for a friend once, used my phone.  was good. ✓

Facebook calendar Sync - as the name says. ✓

Dual N-back - I Don't use it.  I don't think it has value giving properties. Deleted

Awesome calendar - I don't use but it comes with good reccomendations. Deleted Use Business Calendar now.

Battery monitor 3 - Makes a graph of temperature and frequency of the cores.  Useful to see a few times.  Eventually its a bell curve. ✓

urbanspoon - local food places app. ✓use google mostly now.

Gumtree - Australian Ebay (also ebay owns it now) ✓

Printer app to go with my printer ✓

Car Roadside assistance app to go with my insurance ✓

Virgin air entertainment app - you can use your phone while on the plane and download entertainment from their in-flight system. ✓

Two things now;

What am I missing? Was this useful?  Ask me to elaborate on any app and why I used it.  If I get time I will do that anyway. 

P.S. this took 1.5 hours to review and rewrite.

P.P.S - I was intending to make, keep and maintain a list of useful apps, that is not what this document is.  If there are enough suggestions that it's time to make and keep a list; I will do that.

My table of contents links to my other writings

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part V: Value Theory (pp. 1359-1450). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

V. Value Theory

264. Where Recursive Justification Hits Bottom - Ultimately, when you reflect on how your mind operates, and consider questions like "why does Occam's Razor work?" and "why do I expect the future to be like the past?", you have no other option but to use your own mind. There is no way to jump to an ideal state of pure emptiness and evaluate these claims without using your existing mind.

265. My Kind of Reflection - A few key differences between Eliezer Yudkowsky's ideas on reflection and the ideas of other philosophers.

266. No Universally Compelling Arguments - Because minds are physical processes, it is theoretically possible to specify a mind which draws any conclusion in response to any argument. There is no argument that will convince every possible mind.

267. Created Already in Motion - There is no computer program so persuasive that you can run it on a rock. A mind, in order to be a mind, needs some sort of dynamic rules of inference or action. A mind has to be created already in motion.

268. Sorting Pebbles into Correct Heaps - A parable about an imaginary society that has arbitrary, alien values.

269. 2-Place and 1-Place Words - It is possible to talk about "sexiness" as a property of an observer and a subject. It is also equally possible to talk about "sexiness" as a property of a subject, as long as each observer can have a different process to determine how sexy someone is. Failing to do either of these will cause you trouble.

270. What Would You Do Without Morality? - If your own theory of morality was disproved, and you were persuaded that there was no morality, that everything was permissible and nothing was forbidden, what would you do? Would you still tip cabdrivers?

271. Changing Your Metaethics - Discusses the various lines of retreat that have been set up in the discussion on metaethics.

272. Could Anything Be Right? - You do know quite a bit about morality. It's not perfect information, surely, or absolutely reliable, but you have someplace to start. If you didn't, you'd have a much harder time thinking about morality than you do.

273. Morality as Fixed Computation - A clarification about Yudkowsky's metaethics.

274. Magical Categories - We underestimate the complexity of our own unnatural categories. This doesn't work when you're trying to build a FAI.

275. The True Prisoner's Dilemma - The standard visualization for the Prisoner's Dilemma doesn't really work on humans. We can't pretend we're completely selfish.

276. Sympathetic Minds - Mirror neurons are neurons that fire both when performing an action oneself, and watching someone else perform the same action - for example, a neuron that fires when you raise your hand or watch someone else raise theirs. We predictively model other minds by putting ourselves in their shoes, which is empathy. But some of our desire to help relatives and friends, or be concerned with the feelings of allies, is expressed as sympathy, feeling what (we believe) they feel. Like "boredom", the human form of sympathy would not be expected to arise in an arbitrary expected-utility-maximizing AI. Most such agents would regard any agents in its environment as a special case of complex systems to be modeled or optimized; it would not feel what they feel.

277. High Challenge - Life should not always be made easier for the same reason that video games should not always be made easier. Think in terms of eliminating low-quality work to make way for high-quality work, rather than eliminating all challenge. One needs games that are fun to play and not just fun to win. Life's utility function is over 4D trajectories, not just 3D outcomes. Values can legitimately be over the subjective experience, the objective result, and the challenging process by which it is achieved - the traveller, the destination and the journey.

278. Serious Stories - Stories and lives are optimized according to rather different criteria. Advice on how to write fiction will tell you that "stories are about people's pain" and "every scene must end in disaster". I once assumed that it was not possible to write any story about a successful Singularity because the inhabitants would not be in any pain; but something about the final conclusion that the post-Singularity world would contain no stories worth telling seemed alarming. Stories in which nothing ever goes wrong, are painful to read; would a life of endless success have the same painful quality? If so, should we simply eliminate that revulsion via neural rewiring? Pleasure probably does retain its meaning in the absence of pain to contrast it; they are different neural systems. The present world has an imbalance between pain and pleasure; it is much easier to produce severe pain than correspondingly intense pleasure. One path would be to address the imbalance and create a world with more pleasures, and free of the more grindingly destructive and pointless sorts of pain. Another approach would be to eliminate pain entirely. I feel like I prefer the former approach, but I don't know if it can last in the long run.

279. Value is Fragile - An interesting universe, that would be incomprehensible to the universe today, is what the future looks like if things go right. There are a lot of things that humans value that if you did everything else right, when building an AI, but left out that one thing, the future would wind up looking dull, flat, pointless, or empty. Any Future not shaped by a goal system with detailed reliable inheritance from human morals and metamorals, will contain almost nothing of worth.

280. The Gift We Give to Tomorrow - How did love ever come into the universe? How did that happen, and how special was it, really?

This has been a collection of notes on the assigned sequence for this fortnight. The most important part of the reading group though is discussion, which is in the comments section. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

The next reading will cover Part W: Quantified Humanism (pp. 1453-1514) and Interlude: The Twelve Virtues of Rationality (pp. 1516-1521). The discussion will go live on Wednesday, 23 March 2016, right here on the discussion forum of LessWrong.

So I'm going through the sequences (in AI to Zombies) and I get to the bit about Asch's Conformity Experiment.

It's a good bit of writing, but I mostly pass by without thinking about it too much.  I've been taught about the experiment before, and while Eliezer's point of whether or not the subjects were behaving rationally is interesting, it kind of got swallowed up by his discussion of lonely dissent, which I thought was more engaging.

Later, after I'd passed the section on cult attractors and got into the section on letting go, a thought occurred to me, something I'd never actually thought before.

Eliezer notes:

Three-quarters of the subjects in Asch's experiment gave a "conforming" answer at least once.  A third of the subjects conformed more than half the time.

That answer is surprising.  It was surprising to me the first time I learned about the experiment, and I think it's surprising to just about everyone the first time they hear it.  Same thing with a lot of the psychology surrounding heuristics and biases, actually.  Forget the Inquisition - no one saw the Stanford Prison Experiment coming.

Here's the thought I had:  Why was that result so surprising to me?

I'm not an expert in history, but I know plenty of religious people.  I've learned about the USSR and China, about Nazi Germany and Jonestown.  I have plenty of available evidence of times where people went along with things they wouldn't have on their own.  And not all of them are negative.  I've gone to blood drives I probably wouldn't have if my friends weren't going as well.

When I thought about what my prediction would be, had I been asked what percentage of people I thought would dissent before being told, I think I would have guessed that more than 80% of subject would consistently dissent.  If not higher.

And yet that isn't what the experiment shows, and it isn't even what history shows.  For every dissenter in history, there have to be at least a few thousand conformers.  At least.  So why did I think dissent was the norm?

I notice that I am confused.

So I decide to think about it, and my brain immediately spits out: you're an American in an individualistic culture.  Hypothesis: you expect people to conform less because of the culture you live in/were raised in.  This begs the question: have their been cross-cultural studies done on Asch's Conformity Experiment?  Because if people in China conform more than people in America, then how much people conform probably has something to do with culture.

A little googling brings up a 1996 paper that does a meta-analysis on studies that repeated Asch's experiments, either with a different culture, or at a later date in time.  Their findings:

The results of this review can be summarized in three parts.

First, we investigated the impact of a number of potential moderator variables, focusing just on those studies conducted in the United States where we were able to investigate their relationship with conformity, free of any potential interactions with cultural variables. Consistent with previous research, conformity was significantly higher, (a) the larger the size of the majority, (b) the greater the proportion of female respondents, (c) when the majority did not consist of out-group members, and (d) the more ambiguous the stimulus. There was a nonsignificant tendency for conformity to be higher, the more consistent the majority. There was also an unexpected interaction effect: Conformity was higher in the Asch (1952b, 1956) paradigm (as was expected), but only for studies using Asch's (1956) stimulus materials; where other stimulus materials were used (but where the task was also judging which of the three comparison lines was equal to a standard), conformity was higher in the Crutchfield (1955) paradigm. Finally, although we had expected conformity to be lower when the participant's response was not made available to the majority, this variable did not have a significant effect.

The second area of interest was on changes in the level of conformity over time. Again the main focus was on the analysis just using studies conducted in the United States because it is the changing cultural climate of Western societies which has been thought by some to relate to changes in conformity. We found a negative relationship. Levels of conformity in general had steadily declined since Asch's studies in the early 1950s. We did not find any evidence for a curvilinear trend (as, e.g., Larsen, 1982, had hypothesized), and the direction was opposite to that predicted by Lamb and Alsifaki (1980).

The third and major area of interest was in the impact of cultural values on conformity, and specifically differences in individualism-collectivism. Analyses using measures of cultural values derived from Hofstede (1980, 1983), Schwartz (1994), and Trompenaars (1993) revealed significant relationships confirming the general hypothesis that conformity would be higher in collectivist cultures than in individualist cultures. That all three sets of measures gave similar results, despite the differences in the samples and instruments used, provides strong support for the hypothesis. Moreover, the impact of the cultural variables was greater than any other, including those moderator variables such as majority size typically identified as being important factors.

Cultural values, it would seem, are significant mediators of response in group pressure experiments.

So, while the paper isn't definitive, it (and the papers it draws from) show reasonable evidence that there is a cultural impact on how much people conform.

I thought about that for a little while, and then I realized that I hadn't actually answered my own question.

My confusion stems from the disparity between my prediction and reality.  I'm not wondering about the effect culture has on conformity (the territory), I'm wondering about the effect culture has on my prediction of conformity (the map).

In other words, do people born and raised in a culture with collectivist values (China, for example) or who actually do conform beyond the norm (people who are in a flying-saucer cult, or the people actually living in a compound) expect people to conform more than I did?  Is their map any different from mine?

Think about it - with all the different cult attractors, it probably never feels as though you are vastly conforming, even if you are in a cult.  The same can probably be said for any collectivist society.  Imagine growing up in the USSR - would you predict that people would conform with any higher percentage than someone born in 21st century America?  If you were raised in an extremely religious household, would you predict that people would conform as much as they do?  Less?  More?

How many times have I agreed with a majority even when I knew they probably weren't right, and never thought of it as "conformity"?  It took a long time for my belief in god to finally die, even when I could admit that I just believed that I believed.  And why did I keep believing (or keep trying to/saying that I believed)?

Because it's really hard to actually dissent.  And I wasn't even lonely.

So why was my map that wrong?

What background process or motivated reasoning or...whatever caused that disparity?

One thing that, I think, contributes, is that I was generalizing from fictional evidence.  Batman comes far more readily to my mind than Jonestown.  For that matter, Batman comes more readily to my mind than the millions of not-Batmans in Gotham city.  I was also probably not being moved by history enough.  For every Spartacus, there are at minimum hundreds of not-Spartuses, no matter what the not-Spartacuses say when asked.

But to predict that three-quarters of subjects would conform at least once seems to require a level of pessimism beyond even that.  After all, there were no secret police in Asch's experiment; no one had emptied their bank accounts because they thought the world was ending.

Perhaps I'm making a mistake by putting myself into the place of the subject of the experiment.  I think I'd dissent, but I would predict that most people think that, and most people conformed at least once.  I'm also a reasonably well-educated person, but that didn't seem to help the college students in the experiment.

Has any research been done on people's prediction of their own and other's conformity, particularly across cultures or in groups that are "known" for their conformity (communism, the very religious, etc.)?  Do people who are genuine dissenters predict that more people will dissent than people who genuinely conform?

I don't think this is a useless question.  If you're starting a business that offers a new solution to a problem where solutions already exist, are you overestimating how many people will dissent and buy your product?

Based on an earlier suggestion, here's an outreach thread where you can leave comments about any recent outreach that you have done to convey rationality-style ideas broadly. The goal of having this thread is to organize information about outreach and provide community support and recognition for raising the sanity waterline. Likewise, doing so can help inspire others to emulate some aspects of these good deeds through social proof and network effects.

For a moment lets assume there is some alien intelligent life on our galaxy which is older than us and that it have succeeded in creating super-intelligent self-modifying AI.

Then what set of values and/or goals it is plausible for it to have, given our current observations (I.e. that there is no evidence of it`s existence)?

Some examples:

It values non-interference with nature (some kind of hippie AI)

It values camouflage/stealth for it own defense/security purposes.

It just cares about exterminating their creators and nothing else.

Other thoughts?

TL;DR: Biorisks could result in extinction because of multipandemic in near future and their risks is the same order magnitude as risks of UFAI. A lot of biorisks exist, they are cheap and could happen soon.

It may be surprising that number of published research about risks of biological global catastrophe is much less than number of papers about risks of self-improving AI. (One of exception here is "Strategic terrorism” research parer by former chief technology officer of Microsoft.)

It can’t be explain by the fact that biorisks have smaller probability (it will not be known until Bostrom will write the book “Supervirus”). I mean we don’t know it until a lot of research will be done.

Also biorisks are closer in time than AI risks and because of it they shadow AI risks, lowering the probability that extinction will happen by means of UFAI, because it could happen before it by means of bioweapons (e.g. if UFAI risk is 0.9, but chances that we will die before its creation from bioweapons is 0.8, than actual AI risk is 0.18). So studying biorisks may be more urgent than AI risks. 

There is no technical problem to create new flu virus that could kill large part of human population. And the idea of multi pandemic - that it the possibility to release 100 different agents simultaneously - tells us that biorisk could have arbitrary high global lethality. Most of bad things from this map may be created in next 5-10 years, and no improbable insights are needed. Biorisks are also very cheap in production and small civic or personal biolab could  be used to create them.

May be research in estimation probability of human extinction by biorisks had been done secretly? I am sure that a lot of analysis of biorisks exist in secret. But this means that they do not exist in public and scientists from other domains of knowledge can’t independently verify them and incorporate into broader picture of risks. The secrecy here may be useful if it concerns concrete facts about how to crete a dangerous virus. (I was surprised by effectiveness with which Ebola epidemic was stopped after the decision to do so was made, so maybe I should not underestimate government knowledge on the topic).

I had concerns if I should publish this map. I am not a biologist and chances that I will find really dangerous information are small. But what if I inspire bioterrorists to create bioweapons? Anyway we have a lot of movies with such inspiration. 

So I self-censored one idea that may be too dangerous to publish and put black box instead. I also have a section of prevention methods in the lower part of the map. All ideas in the map may be found in wikipedia or other open sources.

The goal of this map is to show importance of risks connected with new kinds of biological weapons which could be created if all recent advances in bioscience will be used for bad. The map shows what we should be afraid off and try to control. So it is map of possible future development of the field of biorisks.

Not any biocatastrophe will result in extinction, it is in the fat tail of the distribution. But smaller catastrophes may delay other good things and wider our window of vulnerability. If protecting measures will be developed on the same speed as possible risks we are mostly safe. If total morality of bioscientists is high we are most likely safe too - no one will make dangerous experiments.

Timeline: Biorisks are growing at least exponentially with the speed of Moore law in biology. After AI will be created and used to for global government and control, biorisks will probably ended. This means that last years before AI creation will be most dangerous from the point of biorisks. 

The first part of the map presents biological organisms that could be genetically edited for global lethality and each box presents one scenario of a global catastrophe. While many boxes are similar to existing bioweapons, they are not the same as not much known bioweapons could result in large scale pandemic (except smallpox and flu). Most probable biorisks are outlined in red in the map. And the real one will be probably not from the map as the world bio is very large and I can’t cover it all.

The map is provided with links which are clickable in the pdf, which is here: http://immortality-roadmap.com/biorisk.pdf 

I have grown up in a family of meat-eaters and therefore have been eating meat all my life. I until recently I have never spent much time thinking about it. I justified my behaviour by saying that animal lives do not matter, because they are not self-conscious and animal pain does not matter, because they have no memory of pain and therefore, as soon as the actual pain is over it is like it has never happened.

In the recent weeks I have spent some time to properly think this through and form an informed believe about whether I can justify eating meat. I would like to hear your thoughts about my thought process and results, because this is a decision that I really don’t want to get wrong.

I have Identified 5 possible problems with meat consumption.

1. Meat requires us to kill animals. 
2.  Factory farmed animals are in a considerable amount of pain for most of their life.
3.  Meat productions requires much more space than producing plants, and therefore might contribute to the world hunger
4.  Some Studies claim that meat, especially if factory farmed, is unhealthy.
5. Meat production is bad for the environment (partly because of point 4, but also for other reasons)

I have decided to ignore problems 4-5 at the beginning, because admitting that they are true would impose weaker restrictions on me. If I come to the conclusion, that I don’t want to eat meat for reason 1, I could no longer eat any meat and reason 2 would forbid me to eat factory farmed meat, which would essentially bring my meat consumption down to something close to zero. 

Reasons 4 and 5 would limit my meat consumption far less, since I do lots of other things that are unhealthy (like eating candy and snacks) or harmful to the environment (like traveling by plane) and while I might come to the conclusion that I want to reduce my meat consumption for reasons 4-5, I expect to have many situations left, where eating meat gives me enough utility to still do it in spite of that reasons.

Reason 3 would also be important, but I am fairly sure, that the problem mostly lies with the lack of spending power in poorer countries, and that it will not lead to more food in Africa if I stop eating meat. For that reason I did not do further research on this.

So what I did was to think about problems 1 and 2 and decide to revisit 4 and 5 if I come to the conclusion that 1 and 2 still allow me to continue eating meat like I do now. 

Is it justifiable to kill animals?

It is clear to me that it is wrong to kill a Human being with a not significantly damaged brain. It is also clear that I have absolutely no problems with killing bacteria or other very simple living beings. Therefore there must exist some features besides the fact that they live that a human has and a bacterium has not, that divides living beings into things that I am willing to kill and things that I am not willing to kill.

The criterion that I used up to know was self-consciousness, which is very convenient because it puts the line between humans (and likely great apes as well) on one side, and basically everything I want to eat on the other side.

There are quite a few things that justify this criterion such as:

1. From a preference utilitarian Perspective, only a self-conscious being can have preferences for the future, therefore you can only violate the preferences of a self-conscious being by killing it. This would be a knock down argument under the premise that preference utilitarism (and not for example normal utilitarism) is the ethical principle to go with 
2. Although I am no expert in this field I believe that it is relatively easy to build a virtual being (for example in a computer game) or with a bit more effort even a robot, that behaves in the way that leads current researchers come to the conclusion that animals have some kind Of Utility. I count the fact that it is easy to build such a thing as evidence, that animals might function in a similar way and I would not have a problem with “hurting” this virtual thing. Therefor if Animals work this way I have no problem with hurting them.
3. This explanation from Eliezer: https://m.facebook.com/yudkowsky/posts/10152588738904228  which I will come back to when I talk about pain, but which is relevant here as well. (Might to some degree be similar to my  point 2)  

There are however other Arguments against it. 

1. Some animals do things that are far more complex than reacting to pain and simple pleasures such as forming relationships for life or mourning if a group member dies. Those things require a more developed brain and are features that most people would see as characteristic for Humans. Since the fact that we kill animals but not humans must come from differences between them, the similar both are, the less likely it is that treating them differently is justified. 
2. From a certain utilitarian perspective (Namely the one that cares about utility of existing beings but not about none existing beings it would be wrong to kill animals with positive utility. And since if animals can have utility it would obviously be wrong to breed them and make their life miserable so that they have negative utility, this would mean that we could not kill animals

I find the arguments against killing animals to be far weaker, since I do not follow the particular form of utilitarism that supports them and since I cannot really explain why the features I named under 1 should forbid me to kill animals. In addition to that I count the fact that Peter Singer, who is against all killing of animals and is arguably a pretty clever person has found no better way to justify his statement, that one should not kill animals at all, than the idea that this will lead us to continue to objectify them and ignore their pain. Since Singer has found no better reason and he probably spent a lot of time doing it, it is likely that there is none.

Although I am fairly confident, that killing Animals is in line with my ethical believes I still see some trouble. If I am wrong on this this might be an incredible harmful decision, since it will lead to the death of many animals (probably hundreds of them, if I don’t reduce my meat consumption for other reasons). Therefore I have to be incredibly confident that I have not overlooked something in order to continue to eat meat. And I have limited time and probably a strong motivation to come to the conclusion that meat eating is okay, which clouds my judgement. I feel that I need more evidence. As far as I know there are lots of meat eaters here and some of them will have thought about this. Why are you so confident that animal life’s do not matter? Is it that I overlooked major arguments or is the self-consciousness just a more of a knock down argument than I think?

Animals and Pain

It is relatively well established that animals show reactions that one could associate with pain and they have a nerve system that allows pain. Singer has proclaimed that in his 1975 book Animal Liberation for mammals and birds and cited research on it, and as far as I know no one has really corrected him on that. I also found papers that claim the same for fish and lobsters and I have not found any counterevidence. So the question that remains is, do animals get negative utility from pain, and do they have utility functions at all.

Eliezer Argues in this post https://m.facebook.com/yudkowsky/posts/10152588738904228 that they don’t have utility. I can understand his model, but I could also imagine that an animal mind works in other ways. I am no expert in evolutionary biology, but as far as I know, the mainstream opinion among scientists right now is that animals have pain.

There is for Example the Cambridge declaration of conciousness (http://fcmconference.org/img/CambridgeDeclarationOnConsciousness.pdf). It might have a different understanding of the word consciousness compared to the one which I think is most popular among the lesswrong community (Consciousness as being aware of its own existence), but it clearly states that animals have affective states and therefor utility. If animals can suffer pain, than factory farming is incredibly wrong. I would therefore have to be very certain (surely above 99% confidence) of the fact that they don’t or I cannot justify to eat factory farmed meat. The question is: How can I be so sure if a significant amount of experts are of a different opinion. Does anyone have any actual research on the topic that explains the reasons why animals do not have utility in more detail than Eliezer did? Basically I would need something that not only explains why this is a plausible hypothesis but something that explain why they could not possibly have evolved in a way that they feel pain. So basically, why a pig that feels pain makes no sense from an evolutionary perspective.

If my current believes don’t shift anymore I will stop eating factory farmed meat, but not stop to eat any meat at all. I would be happy about any additional evidence, or about oppinions on the conclusions I draw from my evidence.

[Cross-posted from FB]

I've got an economic question that I'm not sure how to answer.

I've been thinking about trends in AI development, and trying to get a better idea of what we should expect progress to look like going forward.

One important question is: how much do existing AI systems help with research and the development of new, more capable AI systems?

The obvious answer is, "not much." But I think of AI systems as being on a continuum from calculators on up. Surely AI researchers sometimes have to do arithmetic and other tasks that they already outsource to computers. I expect that going forward, the share of tasks that AI researchers outsource to computers will (gradually) increase. And I'd like to be able to draw a trend line. (If there's some point in the future when we can expect most of the work of AI R&D to be automated, that would be very interesting to know about!)

So I'd like to be able to measure the share of AI R&D done by computers vs humans. I'm not sure of the best way to measure this. You could try to come up with a list of tasks that AI researchers perform and just count, but you might run into trouble as the list of tasks to changes over time (e.g. suppose at some point designing an AI system requires solving a bunch of integrals, and that with some later AI architecture this is no longer necessary).

What seems more promising is to abstract over the specific tasks that computers vs human researchers perform and use some aggregate measure, such as the total amount of energy consumed by the computers or the human brains, or the share of an R&D budget spent on computing infrastructure and operation vs human labor. Intuitively, if most of the resources are going towards computation, one might conclude that computers are doing most of the work.

Unfortunately I don't think that intuition is correct. Suppose AI researchers use computers to perform task X at cost C_x1, and some technological improvement enables X to be performed more cheaply at cost C_x2. Then, all else equal, the share of resources going towards computers will decrease, even though their share of tasks has stayed the same.

On the other hand, suppose there's some task Y that the researchers themselves perform at cost H_y, and some technological improvement enables task Y to be performed more cheaply at cost C_y. After the team outsources Y to computers the share of resources going towards computers has gone up. So it seems like it could go either way -- in some cases technological improvements will lead to the share of resources spent on computers going down and in some cases it will lead to the share of resources spent on computers going up.

So here's the econ part -- is there some standard economic analysis I can use here? If both machines and human labor are used in some process, and the machines are becoming both more cost effective and more capable, is there anything I can say about how the expected share of resources going to pay for the machines changes over time?

This will be of interest mainly to EA-friendly LWs, and is cross-posted on the EA Forum, The Life You Can Save, and Intentional Insights

The Life You Can Save has an excellent tool to help people easily visualize and quantify the impact of their giving: the Impact Calculator. It enables people to put in any amount of money they want, then click on a charity, and see how much of an impact their money can have. It's a really easy way to promote effective giving to non-EAs, but even EAs who didn't see it before can benefit. I certainly did, when I first played around with it. So I wrote a blog post, copy-pasted below, for The Life You Can Save and for Intentional Insights, to help people learn about the Impact Calculator. If you like the blog, please share this link to The Life You Can Save blog, as opposed to this post. Any feedback on the blog post itself is welcomed!

__________________________________________________________________________

How a Calculator Helped Me Multiply My Giving

It feels great to see hope light up in the eyes of a beggar in the street as you stop to look at them when others pass them by without a glance. Their faces widen in a smile as you reach into your pocket and take out your wallet. "Thank you so much" is such a heartwarming phrase to hear from them as you pull out five bucks and put the money in the hat in front of them. You walk away with your heart beaming as you imagine them getting a nice warm meal at McDonalds due to your generosity.

In risk modeling, there is a well-known distinction between aleatory and epistemic uncertainty, which is sometimes referred to, or thought of, as irreducible versus reducible uncertainty. Epistemic uncertainty exists in our map; as Eliezer put it, “The Bayesian says, ‘Uncertainty exists in the map, not in the territory.’” Aleatory uncertainty, however, exists in the territory. (Well, at least according to our map that uses quantum mechanics, according to Bells Theorem – like, say, the time at which a radioactive atom decays.) This is what people call quantum uncertainty, indeterminism, true randomness, or recently (and somewhat confusingly to myself) ontological randomness – referring to the fact that our ontology allows randomness, not that the ontology itself is in any way random. It may be better, in Lesswrong terms, to think of uncertainty versus randomness – while being aware that the wider world refers to both as uncertainty. But does the distinction matter?

To clarify a key point, many facts are treated as random, such as dice rolls, are actually mostly uncertain – in that with enough physics modeling and inputs, we could predict them. On the other hand, in chaotic systems, there is the possibility that the “true” quantum randomness can propagate upwards into macro-level uncertainty. For example, a sphere of highly refined and shaped uranium that is *exactly* at the critical mass will set off a nuclear chain reaction, or not, based on the quantum physics of whether the neutrons from one of the first set of decays sets off a chain reaction – after enough of them decay, it will be reduced beyond the critical mass, and become increasingly unlikely to set off a nuclear chain reaction. Of course, the question of whether the nuclear sphere is above or below the critical mass (given its geometry, etc.) can be a difficult to measure uncertainty, but it’s not aleatory – though some part of the question of whether it kills the guy trying to measure whether it’s just above or just below the critical mass will be random – so maybe it’s not worth finding out. And that brings me to the key point.

In a large class of risk problems, there are factors treated as aleatory – but they may be epistemic, just at a level where finding the “true” factors and outcomes is prohibitively expensive. Potentially, the timing of an earthquake that would happen at some point in the future could be determined exactly via a simulation of the relevant data. Why is it considered aleatory by most risk analysts? Well, doing it might require a destructive, currently technologically impossible deconstruction of the entire earth – making the earthquake irrelevant. We would start with measurement of the position, density, and stress of each relatively macroscopic structure, and the perform a very large physics simulation of the earth as it had existed beforehand. (We have lots of silicon from deconstructing the earth, so I’ll just assume we can now build a big enough computer to simulate this.) Of course, this is not worthwhile – but doing so would potentially show that the actual aleatory uncertainty involved is negligible. Or it could show that we need to model the macroscopically chaotic system to such a high fidelity that microscopic, fundamentally indeterminate factors actually matter – and it was truly aleatory uncertainty. (So we have epistemic uncertainty about whether it’s aleatory; if our map was of high enough fidelity, and was computable, we would know.)

It turns out that most of the time, for the types of problems being discussed, this distinction is irrelevant. If we know that the value of information to determine whether something is aleatory or epistemic is negative, we can treat the uncertainty as randomness. (And usually, we can figure this out via a quick order of magnitude calculation; Value of Perfect information is estimated to be worth $100 to figure out which side the dice lands on in this game, and building and testing / validating any model for predicting it would take me at least 10 hours, my time is worth at least $25/hour, it’s negative.) But sometimes, slightly improved models, and slightly better data, are feasible – and then worth checking whether there is some epistemic uncertainty that we can pay to reduce. In fact, for earthquakes, we’re doing that – we have monitoring systems that can give several minutes of warning, and geological models that can predict to some degree of accuracy the relative likelihood of different sized quakes.

So, in conclusion; most uncertainty is lack of resolution in our map, which we can call epistemic uncertainty. This is true even if lots of people call it “truly random” or irreducibly uncertain – or if they are fancy, aleatory uncertainty. Some of what we assume is uncertainty is really randomness. But lots of the epistemic uncertainty can be safely treated as aleatory randomness, and value of information is what actually makes a difference. And knowing the terminology used elsewhere can be helpful.

I was just reading an article online, and one of the comments mentioned a political issue (the legality of corporate contributions to political campaigns). One of the responses what a comment saying "Not until we abandon this mentality, we the victims are the majority, we can take back this country, all we need to do is open our eyes and stand up." When I saw this comment, I agreed with the sentiment - but nevertheless, I shrugged and moved on. Sure, it is an issue that I strongly believe in, and an issue on which I thought most people would agree with me - but nevertheless, there was nothing I could do about it. Sure, if everyone who agreed on this took a stand (or at least wrote a letter to their congressional representative) we could probably do something about it together - but I could only control my own actions, and in acting alone I'd only be wasting my time.

That got me thinking. This isn't the first time I've come across these sorts of issues. At its heart, this is a coordination problem - lots of people want to do something, but it doesn't make sense for any individual to act unless many others do as well. We don't have a way to solve these sorts of problems, which is quite unfortunate. Except... why can't we have such a system?

Right now, I'm imagining a website where you get to create "causes" and also add your name to them along with a number specifying how many other supporters you'd need to see before you would be willing to take (a pre-specified) action towards the cause. What are the reasons that something like this wouldn't work?

I fact, we do have several websites that work sort-of like this already. Kickstarter is one. The White House Petitions system is another. The first of these has been a wild success; the second, less so (as far as I understand it). So there is clearly some merit to the idea, but also some major setbacks. 

What do people think of this?

If you’ve been planning to get around to maybe thinking about Effective Altruism, now is a great time to get consider making a commitment. As part of Giving What We Can's pledge campaign, people are signing the Giving What We Can pledge - to donate 10% of their future income to the charities they believe will do the most good in the world. It is based on the belief that we can make a real difference by thoroughly assessing evidence and contributing some of our resources to address the most pressing global concerns. The pledge is not legally binding, but is a public declaration of a lasting commitment to the cause. For anyone not ready to make the full commitment to taking the pledge, people are also signing up to 'Try Giving' as part of the campaign - where you commit to donate an amount for a finite time.

Last year in a similar event over 80 people took the pledge, which resulted in almost $19,000,000 being pledged to effective charities. To give you an idea of what this could achieve, a recent GiveWell estimate suggests that, if donated today to the Against Malaria Foundation, this amount could be expected to buy and distribute about 3.5 million bednets and avert the loss of almost 6700 lives (though there is much uncertainty around these figures).

If you think the campaign is a good idea and you'd like more people to hear about it, it would be a great help if you invited anyone you think would be interested in the event; also if you supported the campaign on Thunderclap. If you'd like to help out even more, then join our pledge event organisation Facebook group.

Any questions about the pledge, the campaign, or anything related are more than welcome.
 
About Giving What We Can: GWWC is a meta-charity which researches and evaluates charities on the basis of the impact they have, and also a community with GWWC chapters across the world. It is part of the Centre for Effective Altruism and was co-founded by a LessWronger.

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss The World: An Introduction (pp. 834-839) and Part O: Lawful Truth (pp. 843-883). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

O. Lawful Truth

The World: An Introduction

181. Universal Fire - You can't change just one thing in the world and expect the rest to continue working as before.

182. Universal Law - In our everyday lives, we are accustomed to rules with exceptions, but the basic laws of the universe apply everywhere without exception. Apparent violations exist only in our models, not in reality.

183. Is Reality Ugly? - There are three reasons why a world governed by math can still seem messy. First, we may not actually know the math. Secondly, even if we do know all of the math, we may not have enough computing power to do the full calculation. And finally, even if we did know all the math, and we could compute it, we still don't know where in the mathematical system we are living.

184. Beautiful Probability - Bayesians expect probability theory, and rationality itself, to be math. Self-consistent, neat, even beautiful. This is why Bayesians think that Cox's theorems are so important.

185. Outside the Laboratory - Those who understand the map/territory distinction will integrate their knowledge, as they see the evidence that reality is a single unified process.

186. The Second Law of Thermodynamics, and Engines of Cognition - To form accurate beliefs about something, you really do have to observe it. It's a very physical, very real process: any rational mind does "work" in the thermodynamic sense, not just the sense of mental effort. Engines of cognition are not so different from heat engines, though they manipulate entropy in a more subtle form than burning gasoline. So unless you can tell me which specific step in your argument violates the laws of physics by giving you true knowledge of the unseen, don't expect me to believe that a big, elaborate clever argument can do it either.

187. Perpetual Motion Beliefs - People learn under the traditional school regimen that the teacher tells you certain things, and you must believe them and recite them back; but if a mere student suggests a belief, you do not have to obey it. They map the domain of belief onto the domain of authority, and think that a certain belief is like an order that must be obeyed, but a probabilistic belief is like a mere suggestion. And when half-trained or tenth-trained rationalists abandon their art and try to believe without evidence just this once, they often build vast edifices of justification, confusing themselves just enough to conceal the magical steps. It can be quite a pain to nail down where the magic occurs - their structure of argument tends to morph and squirm away as you interrogate them. But there's always some step where a tiny probability turns into a large one - where they try to believe without evidence - where they step into the unknown, thinking, "No one can prove me wrong".

188. Searching for Bayes-Structure - If a mind is arriving at true beliefs, and we assume that the second law of thermodynamics has not been violated, that mind must be doing something at least vaguely Bayesian - at least one process with a sort-of Bayesian structure somewhere - or it couldn't possibly work.

This has been a collection of notes on the assigned sequence for this fortnight. The most important part of the reading group though is discussion, which is in the comments section. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

The next reading will cover Part P: Reductionism 101 (pp. 887-935). The discussion will go live on Wednesday, 16 December 2015, right here on the discussion forum of LessWrong.

Most discussion of uploading talks of uploading from the inside out: simply, a biological person undergoes a disruptive procedure which digitises their mind. The digital mind then continues the person’s timeline as a digital existence, with all that entails.

The thing that stands out here is the disruptive nature of the process from biological to digital being. It is not only a huge step to undergo such a transformation, but few things in reality operate in such binary terms. More commonly, things happen gradually.

Being an entrepreneur and also having a keen interest in the future, I both respect audacious visions, and study how they come to be realised. Very rarely does progress come from someone investing a bunch of resources in a black-box process that ends in a world-changing breakthrough. Much more commonly, massive innovations are realised through a process of iteration and exploration, fueled by a need that motivates people to solve thousands of problems, big and small. Massive trends interact with other innovations to open up opportunities that when exploited cause a further acceleration of innovation. Every successful startup and technology, from Facebook to Tesla and from mobile phones to modern medicine can be understood in these terms.

With this lens in mind, how might uploading be realised? This is one potential timeline, barring AI explosion or existential catastrophy.

It is perhaps useful to explore the notion of “above/below the API”. A slew of companies have formed, often called “Uber for X” or “AirBnB for Y”, solving needs we have, through a computer system, such as a laptop or a mobile phone app. The app might issue a call to a server via an API, and that server may delegate the task to some other system, often powered by other humans. The original issuer of the command then gets their need covered, minimising direct contact with other humans, the traditional way of having our needs covered. It is crucial to understand that API-mediated interactions win because they are superior to their traditional alternative. Once they were possible, it was only natural for them to proliferate. As an example, compare the experience of taking a taxi with using Uber.

And so computer systems are inserted between human-to-human interactions. This post is composed on a computer, through which I will publish it in a digital location, where it might be seen by others. If I am to hear their response to it, it will also be mediated by APIs. Whenever a successful new API is launched, fortunes are made and lost. An entire industry, venture capital, exists to fund efforts to bring new APIs into existence, each new API making life easier for its users than what came before, and adding additional API layers.

As APIs flood interpersonal space, humans gain superpowers. Presence is less and less important, and a person anywhere in the connected world can communicate and effect change anywhere else. And with APIs comes control of personal space and time. Personal safety increases both by decreasing random physical contact and by always being connected to others who can send help if something goes wrong. The demand for connectivity and computation is driving networking everywhere, and the cost of hardware to fall through the floor.

Given the trends that are in motion, what’s next? Well, if computer-mediated experience is increasing, it might grow to the point where every interaction a human has with the world around them will be mediated by computers. If this sounds absurd, think of noise-cancelling headphones. Many of us now use them not to listen to music, but to block the sound from our environment. Or consider augmented reality. If the visual field, the data pipeline of the brain, can be used to provide critical, or entertaining, context about the physical environment, who would want to forego it? Consider biofeedback: if it’s easy to know at all times what is happening within our bodies and prevent things from going wrong, who wouldn’t want to? It’s not a question of whether these needs exist, but of when technology will be able to cover them.

Once most interaction is API-mediated, the digital world switches from opt-in to opt-out. It’s not a matter of turning the laptop on, but of turning it off for a while, perhaps to enjoy a walk in nature, or for a repair. But wouldn’t you want to bring your augmented reality goggles that can tell you the story of each tree, and ensure you’re not exposed to any pathogens as you wander in the biological jungle? As new generations grow up in a computer-mediated world, fewer and fewer excursions into the offline will happen. Technology, after all, is what was invented after you were born. Few of us consider hunting and gathering our food or living in caves to be a romantic return to the past. When we take a step backward, perhaps to signal virtue, like foregoing vaccination or buying locally grown food, we make sure our move will not deprive us of the benefits of the modern world.

Somewhere around the time when APIs close the loop around us or even before then, the human body will begin to be modified. Artificial limbs that are either plainly superior to their biological counterparts, or better adapted to that world will make sense, and brain-computer interfaces (whether direct or via the existing senses) will become ever more permanent. As our bodies are replaced with mechanical parts, the brain will come next. Perhaps certain simple parts will be easy to replace with more durable, better performing ones. Intelligence enhancement will finally be possible by adding processing power natural selection alone could never have evolved. Gradually, step by small step, the last critical biological components will be removed, as a final cutting of the cord with the physical world.

Humans will have digitised themselves, not by inventing a machine that takes flesh as input and outputs ones and zeroes, not by cyberpunk pioneers jumping into an empty digital world to populate it. We will have done it by making incremental choices, each one a sound rational decision that was in hindsight inevitable, incorporating inventions that made sense, and in the end it will be unclear when the critical step was made. We will have uploaded ourselves simply in the course of everyday life.

The First Step

The first step on the path to truth is superstition.  We all start there, and should acknowledge that we start there.

Superstition is, contrary to our immediate feelings about the word, the first stage of understanding.  Superstition is the attribution of unrelated events to a common (generally unknown or unspecified) cause - it could be called pattern recognition. The "supernatural" component generally included in the definition is superfluous, because supernatural merely refers to that which isn't part of nature - which means reality -, which is an elaborate way of saying something whose relationship to nature is not yet understood, or else nonexistent.  If we discovered that ghosts are real, and identified an explanation - overlapping entities in a many-worlds universe, say - they'd cease to be supernatural and merely be natural.

Just as the supernatural refers to unexplained or imaginary phenomena, superstition refers to unexplained or imaginary relationships, without the necessity of cause.  If you designed an AI in a game which, after five rounds of being killed whenever it went into rooms with green-colored walls, started avoiding rooms with green-colored walls, you've developed a good AI.  It is engaging in superstition, it has developed an incorrect understanding of the issue.  But it hasn't gone down the wrong path - there is no wrong path in understanding, there is only the mistake of stopping.  Superstition, like all belief, is only useful if you're willing to discard it.

The Next Step

Incorrect understanding is the first - and necessary - step to correct understanding.  It is, indeed, every step towards correct understanding.  Correct understanding is a path, not an achievement, and it is pursued, not by arriving at the correct conclusion in the first place, but by testing your ideas and discarding those which are incorrect.

No matter how much intelligent you are, you cannot skip the "incorrect understanding" step of knowledge, because that is every step of knowledge.  You must come up with wrong ideas in order to get at the right ones - which will always be one step further.  You must test your ideas.  And again, the only mistake is stopping, in assuming that you have it right now.

Intelligence is never your bottleneck.  The ability to think faster isn't necessarily the ability to arrive at the right answer faster, because the right answer requires many wrong ones, and more importantly, identifying which answers are indeed wrong, which is the slow part of the process.

Better answers are arrived at by the process of invalidating wrong answers.

The Winding Path

The process of becoming Less Wrong is the process of being, in the first place, wrong.  It is the state of realizing that you're almost certainly incorrect about everything - but working on getting incrementally closer to an unachievable "correct".  It is a state of anti-hubris, and requires a delicate balance between the idea that one can be closer to the truth, and the idea that one cannot actually achieve it.

The art of rationality is the art of walking this narrow path.  If ever you think you have the truth - discard that hubris, for three steps from here you'll see it for superstition, and if you cannot see that, you cannot progress, and there your search for truth will end.  That is the path of the faithful.

But worse, the path is not merely narrow, but winding, with frequent dead ends requiring frequent backtracking.  If ever you think you're closer to the truth - discard that hubris, for it may inhibit you from leaving a dead end, and there your search for truth will end.  That is the path of the crank.

The path of rationality is winding and directionless.  It may head towards beauty, then towards ugliness; towards simplicity, then complexity.  The correct direction isn't the aesthetic one; those who head towards beauty may create great art, but do not find truth.  Those who head towards simplicity might open new mathematical doors and find great and useful things inside - but they don't find truth, either.  Truth is its own path, found only by discarding what is wrong.  It passes through simplicity, it passes through ugliness; it passes through complexity, and also beauty.  It doesn't belong to any one of these things.

The path of rationality is a path without destination.

Written as an experiment in the aesthetic of Less Wrong.  I'd appreciate feedback into the aesthetic interpretation of Less Wrong, rather than the sense of deep wisdom emanating from it (unless the deep wisdom damages the aesthetic).

I've made very extensive notes, along with my assessment, of Daniel Kahneman's Thinking, Fast and Slow, and have passed it around to aspiring rationalist friends who found my notes very useful. So I though I would share these with the Less Wrong community by creating a Less Wrong Wiki article with these notes. Feel free to optimize the article based on your own notes as well. Hope this proves as helpful to you as it did to those others whom I shared my notes with.

A putative new idea for AI control; index here.

EDIT: the mathematics of this approach can be found here.

In a previous post, I described a procedure for querying an AI about whether a certain outcome was desirable for it. When I did so, I didn't realise the power of the idea.

In brief, that idea allows us to query an AI with utility v as to whether an outcome X or an outcome ¬X was the one it most desired in a particular situation. By judicious use of 'false beliefs', the AI would react as if it had miraculous powers to determine X vs ¬X in ways that no-one could ever know about.

Many (including Toby Ord) has suggested variants on the 'honey pot' idea: leave something available to the AI that looks like an oversight, but is actually a trap, to see if the AI goes beyond the parameters of what we'd expect. Maybe a wire to the internet that the AI is not supposed to use, but seems to be unobserved.

The problem with these ideas is that the honey pots had to be physically constructed, physically accessible to the AI, and the AI had to be wrong about whether it was a trap or not.

But using the method above, we can go much further. We can test the AI in any scenario which we can imagine, as long as we can precisely physically describe it. We can check what would happen if the AI was given the ability to launch the world's nuclear arsenals, or divert an asteroid towards Earth. We can check what it would design if it was given unrestricted access to a DNA synthesiser. We could see what it would do if it could rewrite the text of a speech an important world leader was due to give. We could see whether it would sabotage the creation of other AIs - and if so, which ones. We could see what it did to a probe humanity planned to launch to the stars, what numbers it would call if it had access to a phone, what it would purchase if it had capital.

And, in all cases, the AI would be behaving as if its actions were undetectable and certain to succeed, while we ensured that they were (almost certainly) detectable and certain to fail. There is no "almost" in that certain to fail condition, because the AI is acting as if it had miraculous powers that it doesn't have.

Now, we can't use testing as our sole method of AI control (for reasons similar to here), but it does allow us some pretty effective ways of catching bad AIs.

Some of us enjoy being sidekicks. 

Some of us would like to meet sidekicks in potential, see how the interaction goes, and have sidekicks. 

Last time I tried posting about sidekick matchmaking here, it turned out to be very valuable for me, but not for many people (I think only two pairs of sidekick were created as a result). Now, once again I'd like to find someone who enjoys that role to help me out with many projects. 

I'm looking for suggestions on how to get people together to do that. For the time being, if someone needs a sidekick or wants to be one, post about it in the comment section. I'd love to see a permanent solution for this information spreading problem. 

My experience with Sidekicks

I'm not sure what Anna and Nick thought of their sidekicks, but my experience was undeniably positive. Having a sidekick was motivating, saved me great time, and, most importantly, felt like I got a surge of muscle strength specifically in the types of tasks I'm particularly inept at. 

By contrast, my experience with people hired to help was mixed (virtual assistants) or negative (personal assistant). 

Use the comment section to either offer or request sidekicks, explaining a little more about you and what you'd like this partnership to mean

There seems to be a growing consensus among the community that while Less Wrong is great at improving epistemic rationality, it is rather lacking when it comes to resources for instrumental rationality. I've been thinking about how to address this. This can be very hard because many of the questions most important to instrumental rationality lack an objective answer and depend heavily on individual circumstance. Consider for example the question, "How do I become a more interesting person?", that is the first survey article I've published. One person might easily have the resources to go travelling and gain new experiences, while another person might be prevented by their financial situation. One person may enjoy the process of broadening their experience by reading, while another may simply detest books. Ignoring these individual circumstances will lead to much of the advice being unsuitable

It therefore seems that in a general resource, that is forced by its very nature to ignore individual circumstances, that the best response would be to gather together as many ideas as possible. It is hoped that each rationalist has the capacity to critically examine each suggestion that is proposed and reject those that would be counterproductive. This differs from a standard list article as, instead of limiting itself to an arbitrary number of ideas, or only using ideas thought of by the author, I have made a comprehensive list and taken ideas from different sources. Taking ideas from different sources is extremely important - a single person can only possess so much creativity. It also decreases the influence of the author's subjective point of view - I might never have said something myself, but I might be willing to include it in a list of ideas. Another problem with lists is that if they are wordy, they take a long time to read through, while if they are concise, they may be misunderstood. Summarising whilst linking to a source means that extra detail is available for those who need it.

One flaw is that the production of this lists will always be greatly subjective. I really like Mark Manson and am probably going to quote him a lot in these lists, but another person might love The Secret and quote it everywhere instead. Regardless of this subjectivity, if you think that a particular source lacks value, then you can choose to just ignore that source and just read the rest of the article. If there is a noticeable omission, that can be addressed in the comments, or, in extreme cases, by producing a rival list. So I think that these articles can work well regardless of subjectivity.

What problem is this designed to solve?

This has already been discussed above, but I want to go into more detail about the current process when someone has one of these subjective questions. The current process probably looks like Googling the question or searching the question on a trusted source (ie. Quora or Reddit). There are many good answers and good ideas, but they are spread out all over the Internet. It is very possible for someone to fail to find a suggestion that would have helped them. Gathering together a large number of different resources helps to minimise this. It also helps people to discover new sources that they might not have thought to look at.

What feedback am I after?

As well as general support or criticisms of the idea, I'd also like to see some suggestions on which questions you'd love to see a survey for.

Just published an article in the The Huffington Post about using rationality-informed strategies to manage my mental illness. Hope this helps people think more rationally about this topic.

The prediction expert Philip Tetlock writes in New York Times on the power of precise predictions to counter political polarization. Note the similarity to Robin Hanson's futarchy idea.

IS there a solution to this country’s polarized politics?

Consider the debate over the nuclear deal with Iran, which was one of the nastiest foreign policy fights in recent memory. There was apocalyptic rhetoric, multimillion-dollar lobbying on both sides and a near-party-line Senate vote. But in another respect, the dispute was hardly unique: Like all policy debates, it was, at its core, a contest between competing predictions.

Opponents of the deal predicted that the agreement would not prevent Iran from getting the bomb, would put Israel at greater risk and would further destabilize the region. The deal’s supporters forecast that it would stop (or at least delay) Iran from fielding a nuclear weapon, would increase security for the United States and Israel and would underscore American leadership.

The problem with such predictions is that it is difficult to square them with objective reality. Why? Because few of them are specific enough to be testable. Key terms are left vague and undefined. (What exactly does “underscore leadership” mean?) Hedge words like “might” or “could” are deployed freely. And forecasts frequently fail to include precise dates or time frames. Even the most emphatic declarations — like former Vice President Dick Cheney’s prediction that the deal “will lead to a nuclear-armed Iran” — can be too open-ended to disconfirm.

//

Non-falsifiable predictions thus undermine the quality of our discourse. They also impede our ability to improve policy, for if we can never judge whether a prediction is good or bad, we can never discern which ways of thinking about a problem are best.

The solution is straightforward: Replace vague forecasts with testable predictions. Will the International Atomic Energy Agency report in December that Iran has adequately resolved concerns about the potential military dimensions of its nuclear program? Will Iran export or dilute its quantities of low-enriched uranium in excess of 300 kilograms by the deal’s “implementation day” early next year? Within the next six months, will any disputes over I.A.E.A. access to Iranian sites be referred to the Joint Commission for resolution?

Such questions don’t precisely get at what we want to know — namely, will the deal make the United States and its allies safer? — but they are testable and relevant to the question of the Iranian threat. Most important, they introduce accountability into forecasting. And that, it turns out, can depolarize debate.

In recent years, Professor Tetlock and collaborators have observed this depolarizing effect when conducting forecasting “tournaments” designed to identify what separates good forecasters from the rest of us. In these tournaments, run at the behest of the Intelligence Advanced Research Projects Activity (which supports research relevant to intelligence agencies), thousands of forecasters competed to answer roughly 500 questions on various national security topics, from the movement of Syrian refugees to the stability of the eurozone.

The tournaments identified a small group of people, the top 2 percent, who generated forecasts that, when averaged, beat the average of the crowd by well over 50 percent in each of the tournament’s four years. How did they do it? Like the rest of us, these “superforecasters” have political views, often strong ones. But they learned to seriously consider the possibility that they might be wrong.

What made such learning possible was the presence of accountability in the tournament: Forecasters were able see their competitors’ predictions, and that transparency reduced overconfidence and the instinct to make bold, ideologically driven predictions. If you can’t hide behind weasel words like “could” or “might,” you start constructing your predictions carefully. This makes sense: Modest forecasts are more likely to be correct than bold ones — and no one wants to look stupid.

This suggests a way to improve real-world discussion. Suppose, during the next ideologically charged policy debate, that we held a public forecasting tournament in which representatives from both sides had to make concrete predictions. (We are currently sponsoring such a tournament on the Iran deal.) Based on what we have seen in previous tournaments, this exercise would decrease the distance between the two camps. And because it would be possible to determine a “winner,” it would help us learn whether the conservative or liberal assessment of the issue was more accurate.

 

Either way, we would begin to emerge from our dark age of political polarization.

If someone has died it doesn’t mean that you should stop trying to return him to life. There is one clear thing that you should do (after cryonics): collect as much information about the person as possible, as well as store his DNA sample, and hope that future AI will return him to life based on this information.

Two meanings of “Digital immortality”

The term “Digital immortality” is often confused with the notion of mind uploading, as the end result is almost the same: a simulated brain in a computer. https://en.wikipedia.org/wiki/Digital_immortality

But here, by the term “Digital immortality” I mean reconstruction of the person based on his digital footprint and other traces by future AI after this person death.

Mind uploading in the future will happen while the original is still alive (or while the brain exists in a frozen state) and will be connected to a computer by some kind of sophisticated interface, or the brain will be scanned. It cannot be done currently. 

On the other hand, reconstruction based on traces will be done by future AI. So we just need to leave enough traces and we could do it now.

But we don’t know how much traces are enough, so basically we should try to produce and preserve as many traces as possible. However, not all traces are equal in their predictive value. Some are almost random, and others are so common that they do not provide any new information about the person.

Cheapest way to immortality

Creating traces is an affordable way of reaching immortality. It could even be done for another person after his death, if we start to collect all possible information about him. 

Basically I am surprised that people don’t do it all the time. It could be done in a simple form almost for free and in the background – just start a video recording app on your notebook, and record everything into shared folder connected with a free cloud. (Evocam program for Mac is excellent, and mail.ru provides up 100gb free).

But really good digital immortality require 2-3 month commitment for self-description with regular every year updates. It may also require maximum several thousand dollars investment in durable disks, DNA testing, videorecorders, and free time to do it.

I understand how to set up this process and could help anyone interested.

Identity

The idea of personal identity is outside the scope of this map. I have another map on this topic (now in draft), I assume that the problem of personal identity will be solved in the future. Perhaps we will prove that information only is enough to solve the problem, or we will find that continuity of consciousness, but we will be able to construct mechanisms to transfer this identity independently of information. 

Digital immortality requires a very weak notion of identity. i.e. a model of behavior and thought processes is enough for an identity. This model may have some differences from the original, which I call “one night difference”, that is the typical difference between me-yesterday and me-today after one night's sleep. The meaningful part of this information has size from several megabytes to gigabits, but we may need to collect much more information as we can’t now extract meaningful part from random.

DI may also be based on even weaker notion of identity, that anyone who thinks that he is me, is me. Weaker notions of identity require less information to be preserved, and in last case it may be around 10K bytes (including name, indexical information and basic traits description)

But the question about the number of traces needed to create an almost exact model of a personality is still open. It also depends on predictive power of future AI: the stronger is AI, the less traces are enough.

Digital immortality is plan C in my Immortality Roadmap, where Plan A is life extension and Plan B is cryonics; it is not plan A, because it requires solving the identity problem plus the existence of powerful future AI.

Self-description

I created my first version of it in the year 1990 when I was 16, immediately after I had finished school. It included association tables, drawings and lists of all people known to me, as well as some art, memoires, audiorecordings and encyclopedia od everyday objects around me.

There are several approaches to achieving digital immortality. The most popular one is passive that is simply videorecording of everything you do.

My idea was that a person can actively describe himself from inside. He may find and declare the most important facts about himself. He may run specific tests that will reveal hidden levels of his mind and sub consciousness. He can write a diary and memoirs. That is why I called my digital immortality project “self-description”.

Structure of the map

This map consists of two parts: theoretical and practical. The theoretical part lists basic assumptions and several possible approaches to reconstructing an individual, in which he is considered as a black box. If real neuron actions will become observable, the "box" will become transparent and real uploading will be possible.

There are several steps in the practical part:

- The first step includes all the methods of fixing information while the person of interest is alive.

- The second step is about preservation of the information.

- The third step is about what should be done to improve and promote the process.

- The final fourth step is about the reconstruction of the individual, which will be performed by AI after his death. In fact it may happen soon, may be in next 20-50 years.

There are several unknowns in DI, including the identity problem, the size and type of information required to create an exact model of the person, and the required power of future AI to operate the process. These and other problems are listed in the box on the right corner of the map.

The pdf of the map is here, and jpg is below.

Previous posts with maps:

Doomsday Argument Map

AGI Safety Solutions Map

A map: AI failures modes and levels

A Roadmap: How to Survive the End of the Universe

A map: Typology of human extinction risks

Roadmap: Plan of Action to Prevent Human Extinction Risks

Immortality Roadmap

Previous thread: http://lesswrong.com/lw/mnq/instrumental_rationality_questions_thread/

This thread is for asking the rationalist community for practical advice.  It's inspired by the stupid questions series, but with an explicit focus on instrumental rationality.

Questions ranging from easy ("this is probably trivial for half the people on this site") to hard ("maybe someone here has a good answer, but probably not") are welcome.  However, please stick to problems that you actually face or anticipate facing soon, not hypotheticals.

As with the stupid questions thread, don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better, and please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

(See also the Boring Advice Repository)

As part of my broader project of promoting rationality widely, I'm going to work on making rationality-themed merchandise with slogans. I'd appreciate any ideas on what slogans would be short (5 words or less), engaging, and accessible, and appealing for both aspiring rationalists and smart youth and young adults who are just starting to learn about rationality. As an example, slogans like "Growing Mentally Stronger" or "Updating My Beliefs" are good, but "Tsuyoku Naritai!" is not, however much I personally like that slogan.

Eliezer commented on FB about a post Announcing GoodAI (by Marek Rosa GoodAIs CEO). I think this deserves some discussion as it has a quite effective approach to harness the crowd to improve the AI:

As part of GoodAI’s development, our team created a visual tool called Brain Simulator where users can design their own artificial brain architectures. We released Brain Simulator to the public today for free under and open-source, non-commercial license– anyone who’s interested can access Brain Simulator and start building their own artificial brain. [...]

By integrating Brain Simulator into Space Engineers and Medieval Engineers [a game], players will have the option to design their own AI brains for the games and implement it, for example, as a peasant character. Players will also be able to share these brains with each other or take an AI brain designed by us and train it to do things they want it to do (work, obey its master, and so on). The game AIs will learn from the player who trains them (by receiving reward/punishment signals; or by imitating player's behavior), and will have the ability to compete with each other. The AI will be also able to learn by imitating other AIs.

This integration will make playing Space Engineers and Medieval Engineers more fun, and at the same time our AI technology will gain access to millions of new teachers and a new environment. This integration into our games will be done by GoodAI developers. We are giving AI to players, and we are bringing players to our AI researchers.

The Doomsday argument (DA) is controversial idea that humanity has a higher probability of extinction based purely on probabilistic arguments. The DA is based on the proposition that I will most likely find myself somewhere in the middle of humanity's time in existence (but not in its early time based on the expectation that humanity may exist a very long time on Earth.)

­

There were many different definitions of the DA and methods of calculating it, as well as rebuttals. As a result we have developed a complex group of ideas, and the goal of the map is to try to bring some order to it. The map consists of various authors' ideas. I think that I haven't caught all existing ideas, and the map could be improved significantly – but some feedback is needed on this stage.

The map has the following structure: the horizontal axis consists of various sampling methods (notably SIA and SSA), and the vertical axis has various approaches to the DA, mostly Gott's (unconditional) and Carters’s (update of probability of existing risk). But many important ideas can’t fit in this scheme precisely, and these have been added on the right hand side. 

In the lower rows the link between the DA and similar arguments is shown, namely the Fermi paradox, Simulation argument and Anthropic shadow, which is a change in the probability assessment of natural catastrophes based on observer selection effects. 

On the right part of the map different ways of DA rebuttal are listed and also a vertical raw of possible positive solutions.

I think that the DA is mostly true but may not mean inevitable extinction.

Several interesting ideas may need additional clarification and they will also put light on the basis of my position on DA.

Meta-DA

The first of these ideas is that the most reasonable version of the DA at our current stage of knowledge is something that may be called the meta-DA, which presents our uncertainty about the correctness of any DA-style theories and our worry that the DA may indeed be true.

The meta-DA is a Bayesian superstructure built upon the field of DA theories. The meta-DA tells us that we should attribute non-zero Bayesian probability to one or several DA-theories (at least until they are disproved in a generally accepted way) and since the DA itself is a probabilistic argument, then these probabilities should be combined.

As a result the Meta-DA means an increase of total existential risks until we disprove (or prove) all versions of the DA, which may be not easy. We should anticipate such an increase in risk as a demand to be more precautious but not in a fatalist “doom imminent” way.

Reference class

The second idea concerns the so-called problem of reference class that is the problem of which class of observer I belong to in the light of question of the DA. Am I randomly chosen from all animals, humans, scientists or observer-moments?

The proposed solution is that the DA is true for any referent class from which I am randomly chosen, but the mere definition of the referent class is defining the type it will end as; it should not be global catastrophe. In short, any referent class has its own end. For example, if I am randomly chosen from the class of all humans, than the end of the class may mean not an extinction but a creation of the beginning of the class of superhumans.

But any suitable candidate for the DA-logic referent class must provide the randomness of my position in it. In that case I can’t be a random example of the class of mammals, because I am able to think about the DA and a zebra can’t.

As a result the most natural (i.e. providing a truly random distribution of observers) referent class is a class of observers who know about and can think about DA. The ability to understand the DA is the real difference between conscious and unconscious observers.

But this class is small and young. It started in 1983 with the works of Carter and now includes perhaps several thousand observers. If I am in the middle of it, there will be just several thousand more DA-aware observers and there will only be several decades more before the class ends (which unpleasantly will coincide with the expected “Singularity” and other x-risks). (This idea was clear to Carter and also is used in so called in so-called Self-referencing doomsday argument rebuttal https://en.wikipedia.org/wiki/Self-referencing_doomsday_argument_rebuttal)

This may not necessarily mean the end of the global catastrophe, but it may mean that there will soon be a DA rebuttal. (And we could probably choose how to fulfill the DA prophecy by manipulating of the number of observers in the referent class.)

DA and medium life expectancy

DA is not unnatural way to see in the future as it seems to be. The more natural way to understand the DA is to see it as an instrument to estimate medium life expectancy in the certain group.

For example, I think that I can estimate medium human life expectancy based on your age. If you are X years old, human medium life expectancy is around 2X. “Around” here is very vague term as it more like order of magnitude. For example if you are 25 years old, I could think that medium human life expectancy is several decades years and independently I know its true (but not 10 millisecond or 1 million years). And as medium life expectancy is also may be applied to the person in question it may mean that he will also most probably live the same time (if we will not do something serious about life extension). So there is no magic or inevitable fate in DA.

But if we apply the same logic to civilization existence, and will count only a civilization capable to self-destruction, e.g. roughly after 1945, or 70 years old, it would provide medium life expectancy of technological civilizations around 140 years, which extremely short compare to our estimation that we may exist millions of years and colonize the Galaxy.

Anthropic shadow and fragility of our environment

|t its core is the idea that as a result of natural selection we have more chances to find ourselves in the world, which is in the meta-stable condition on the border of existential catastrophe, because some catastrophe may be long overdue. (Also because universal human minds may require constantly changing natural conditions in order to make useful adaptations, which implies an unstable climate – and we live in period of ice ages)

In such a world, even small human actions could result in global catastrophe. For example if we pierce a overpressured ball with a needle.

The most plausible candidates for such metastable conditions are processes that must have happened a long time ago in most worlds, but we can only find ourselves in the world where they are not. For the Earth it may be sudden change of the atmosphere to a Venusian subtype (runaway global warming). This means that small human actions could have a much stronger result for atmospheric stability (probably because the largest accumulation of methane hydrates in earth's history resides on the Arctic Ocean floor, which is capable of a sudden release: see https://en.wikipedia.org/wiki/Clathrate_gun_hypothesis). Another option for meta-stability is provoking a strong supervolcane eruption via some kind of earth crust penetration (see “Geoingineering gone awry” http://arxiv.org/abs/physics/0308058)

Thermodynamic version of the DA 

Also for the western reader is probably unknown thermodynamic version of DA suggested in Strugatsky’s novel “Definitely maybe” (Originally named “A Billion years before the end of the world”). It suggests that we live in thermodynamic fluctuation and as smaller and simpler fluctuations are more probable, there should be a force against complexity, AI development or our existence in general. Plot of the novel is circled around pseudo magical force, which distract best scientists from work using girls, money or crime. After long investigation they found that it is impersonal force against complexity.

This map is a sub-map for the planned map “Probability of global catastrophe” and its parallel maps are a “Simulation argument map” and a “Fermi paradox map” (both are in early drafts).

PDF of the map: http://immortality-roadmap.com/DAmap.pdf 

Previous posts with maps:

AGI Safety Solutions Map

A map: AI failures modes and levels

A Roadmap: How to Survive the End of the Universe

A map: Typology of human extinction risks

Roadmap: Plan of Action to Prevent Human Extinction Risks

Immortality Roadmap

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part I: Seeing with Fresh Eyes (pp. 365-406). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

I. Seeing with Fresh Eyes

87. Anchoring and Adjustment - Exposure to numbers affects guesses on estimation problems by anchoring your mind to an given estimate, even if it's wildly off base. Be aware of the effect random numbers have on your estimation ability.

88. Priming and Contamination - Contamination by Priming is a problem that relates to the process of implicitly introducing the facts in the attended data set. When you are primed with a concept, the facts related to that concept come to mind easier. As a result, the data set selected by your mind becomes tilted towards the elements related to that concept, even if it has no relation to the question you are trying to answer. Your thinking becomes contaminated, shifted in a particular direction. The data set in your focus of attention becomes less representative of the phenomenon you are trying to model, and more representative of the concepts you were primed with.

89. Do We Believe Everything We're Told - Some experiments on priming suggest that mere exposure to a view is enough to get one to passively accept it, at least until it is specifically rejected.

90. Cached Thoughts - Brains are slow. They need to cache as much as they can. They store answers to questions, so that no new thought is required to answer. Answers copied from others can end up in your head without you ever examining them closely. This makes you say things that you'd never believe if you thought them through. So examine your cached thoughts! Are they true?

91. The "Outside the Box" Box - When asked to think creatively there's always a cached thought that you can fall into. To be truly creative you must avoid the cached thought. Think something actually new, not something that you heard was the latest innovation. Striving for novelty for novelty's sake is futile, instead you must aim to be optimal. People who strive to discover truth or to invent good designs, may in the course of time attain creativity.

92. Original Seeing - One way to fight cached patterns of thought is to focus on precise concepts.

93. Stranger Than History - Imagine trying to explain quantum physics, the internet, or any other aspect of modern society to people from 1900. Technology and culture change so quickly that our civilization would be unrecognizable to people 100 years ago; what will the world look like 100 years from now?

94. The Logical Fallacy of Generalization from Fictional Evidence - The Logical Fallacy of Generalization from Fictional Evidence consists in drawing the real-world conclusions based on statements invented and selected for the purpose of writing fiction. The data set is not at all representative of the real world, and in particular of whatever real-world phenomenon you need to understand to answer your real-world question. Considering this data set leads to an inadequate model, and inadequate answers.

95. The Virtue of Narrowness - One way to fight cached patterns of thought is to focus on precise concepts.

96. How to Seem (and be) Deep - To seem deep, find coherent but unusual beliefs, and concentrate on explaining them well. To be deep, you actually have to think for yourself.

97. We Change Our Minds Less Often Than We Think - We all change our minds occasionally, but we don't constantly, honestly reevaluate every decision and course of action. Once you think you believe something, the chances are good that you already do, for better or worse.

98. Hold Off On Proposing Solutions - Proposing solutions prematurely is dangerous, because it introduces weak conclusions in the pool of the facts you are considering, and as a result the data set you think about becomes weaker, overly tilted towards premature conclusions that are likely to be wrong, that are less representative of the phenomenon you are trying to model than the initial facts you started from, before coming up with the premature conclusions.

99. The Genetic Fallacy - The genetic fallacy seems like a strange kind of fallacy. The problem is that the original justification for a belief does not always equal the sum of all the evidence that we currently have available. But, on the other hand, it is very easy for people to still believe untruths from a source that they have since rejected.

This has been a collection of notes on the assigned sequence for this fortnight. The most important part of the reading group though is discussion, which is in the comments section. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

The next reading will cover Part J: Death Spirals (pp. 409-494). The discussion will go live on Wednesday, 23 September 2015, right here on the discussion forum of LessWrong.

While thinking about my own next career steps, I've been writing down some of my thoughts about what's in an impactful career.

In the process, I wrote an introductory report on what seem to me to be practical approaches to problems in catastrophic risks. It's intended to complement the analysis that 80,000 Hours provides by thinking about what general roles we ought to perform, rather than analysing specific careers and jobs, and by focusing specifically on existential risks.

I'm happy to receive feedback on it, positive and negative. 

Here it is: Reducing Catastrophic Risks, A Practical Introduction.

tl;dr A median maximiser will expect to win. A mean maximiser will win in expectation. As we face repeated problems of similar magnitude, both types take on the advantage of the other. However, the median maximiser will turn down Pascal's muggings, and can say sensible things about distributions without means.

Prompted by some questions from Kaj Sotala, I've been thinking about whether we should use the median rather than the mean when comparing the utility of actions and policies. To justify this, see the next two sections: why the median is like the mean, and why the median is not like the mean.

Why the median is like the mean

The main theoretic justifications for the use of expected utility - hence of means - are the von Neumann Morgenstern axioms. Using the median obeys the completeness and transitivity axioms, but not the continuity and independence ones.

It does obey weaker forms of continuity; but in a sense, this doesn't matter. You can avoid all these issues by making a single 'ultra-choice'. Simply list all the possible policies you could follow, compute their median return, and choose the one with the best median return. Since you're making a single choice, independence doesn't apply.

So you've picked the policy π_m with the highest median value - note that to do this, you need only know an ordinal ranking of worlds, not their cardinal values. In what way is this like maximising expected utility? Essentially, the more options and choices you have - or could hypothetically have - the closer this policy must be to expected utility maximalisation.

Assume u is a utility function compatible with your ordinal ranking of the worlds. Then π_u = 'maximise the expectation of u' is also a policy choice. If we choose π_m, we get a distribution d_mu of possible values of u. Then E(u|π_m) is within the absolute deviation (using d_mu) of the median value of d_mu. This absolute deviation always exists for any distribution with an expectation, and is itself bounded by the standard deviation, if it exists.

Thus maximising the median is like maximising the mean, with an error depending on the standard deviation. You can see it as a risk averse utility maximising policy (I know, I know - risk aversion is supposed to go in defining the utility, not in maximising it. Read on!). And as we face more and more choices, the standard deviation will tend to fall relative to the mean, and the median will cluster closer and closer to the mean.

For instance, suppose we consider the choice of whether to buckle our seatbelt or not. Assume we don't want to die in a car accident that a seatbelt could prevent; assume further that the cost of buckling a seatbelt is trivial but real. To simplify, suppose we have an independent 1/Ω chance of death every time we're in a car, and that a seatbelt could prevent this, for some large Ω. Furthermore, we will be in a car a total of ρΩ, for ρ < 0.5. Now, it seems, the median recommends a ridiculous policy: never wear seatbelts. Then you pay no cost ever, and your chance of dying is less than 50%, so this has the top median.

And that is indeed a ridiculous result. But it's only possible because we look at seatbelts in isolation. Every day, we face choices that have small chances of killing us. We could look when crossing the street; smoke or not smoke cigarettes; choose not to walk close to the edge of tall buildings; choose not to provoke co-workers to fights; not run around blindfolded. I'm deliberately including 'stupid things no-one sensible would ever do', because they are choices, even if they are obvious ones. Let's gratuitously assume that all these choices also have a 1/Ω chance of killing you. When you collect together all the possible choices (obvious or not) that you make in your life, this will be ρ'Ω choice, for ρ' likely quite a lot bigger than 1.

Assume that avoiding these choices has a trivial cost, incommensurable with dying (ie no matter how many times you have to buckle your seatbelt, it still better than a fatal accident). Now median-maximisation will recommend taking safety precautions for roughly (ρ'-0.5)Ω of these choices. This means that the decision of a median maximiser will be close to those of a utility maximiser - they take almost the same precautions - though the outcomes are still pretty far apart: the median maximiser accepts a 49.99999...% chance of death.

But now add serious injury to the mix (still assume the costs are incommensurable). This has a rather larger probability, and the median maximiser will now only accept a 49.99999...% chance of serious injury. Or add light injury - now they only accept a 49.99999...% chance of light injury. If light injuries are additive - two injuries are worse than one - then the median maximiser becomes even more reluctant to take risks. We can now relax the assumption of incommensurablility as well; the set of policies and assessments becomes even more complicated, and the median maximiser moves closer to the mean maximiser.

The same phenomena tends to happen when we add lotteries of decisions, chained decisions (decisions that depend on other decisions), and so on. Existential risks are interesting examples: from the selfish point of view, existential risks are just other things that can kills us - and not the most unlikely ones, either. So the median maximiser will be willing to pay a trivial cost to avoid an xrisk. Will a large group of median maximisers be willing to collectively pay a large cost to avoid an xrisk? That gets into superrationality, which I haven't considered yet in this context.

But let's turn back to the mystical utility function that we are trying to maximise. It's obvious that humans don't actually maximise a utility function; but according to the axioms, we should do so. Since we should, people on this list tend to often assume that we actually have one, skipping over the process of constructing it. But how would that process go? Let's assume we've managed to make our preferences transitive, already a major good achievement. How should we go about making them independent as well? We can do so as we go along. But if we do it ahead of time, chances are that we will be comparing hypothetical situations ("Do I like chocolate twice as much as sex? What would I think of a 50% chance of chocolate vs guaranteed sex? Well, it depends on the situation...") and thus construct a utility function. This is where we have to make decisions about very obscure and unintuitive hypothetical tradeoffs, and find a way to fold all our risk aversion/risk love into the utility.

When median maximising, we do exactly the same thing, except we constrain ourselves to choices that are actually likely to happen to us. We don't need a full ranking of all possible lotteries and choices; we just need enough to decide in the situations we are likely to face. You could consider this a form of moral learning (or preference learning). From our choices in different situations (real or possible), we decide what our preferences are in these situations, and this determines our preferences overall.

Why the median is not like the mean

Ok, so the previous paragraph argues that median maximising, if you have enough choices, functions like a clunky version of expected utility maximising. So what's the point?

The point is those situations that are not faced sufficiently often, or that have extreme characteristics. A median maximiser will reject Pascal's mugging, for instance, without any need for extra machinery (though they will accept Pascal's muggings if they face enough independent muggings, which is what we want - for stupidly large values of "enough"). They cope fine with distributions that have no means - such as the Cauchy distribution or a utility version of the St Petersburg paradox. They don't fall into paradox when facing choices with infinite (but ordered) rewards.

In a sense, median maximalisation is like expected utility maximalisation for common choices, but is different for exceptionally unlikely or high impact choices. Or, from the opposite perspective, expected utility maximising gives high probability of good outcomes for common choices, but not for exceptionally unlikely or high impact choices.

Another feature of the general idea (which might be seen as either a plus or a minus) is that it can get around some issues with total utilitarianism and similar ethical systems (such as the repugnant conclusion). What do I mean by this? Well, because the idea is that only choices that we actually expect to make matter, we can say, for instance, that we'd prefer a small ultra happy population to a huge barely-happy one. And if this is the only choice we make, we need not fear any paradoxes: we might get hypothetical paradoxes, just not actual ones. I won't put too much insistence on this point, I just thought it was an interesting observation.

For lack of a Cardinal...

Now, the main issue is that we might feel that there are certain rare choices that are just really bad or really good. And we might come to this conclusion by rational reasoning, rather than by experience, so this will not show up in the median. In these cases, it feels like we might want to force some kind of artificial cardinal order on the worlds, to make the median maximiser realise that certain rare events must be considered beyond their simple ordinal ranking.

In this case, maybe we could artificially add some hypothetical choices to our system, making us address these questions more than we actually would, and thus drawing them closer to the mean maximising situation. But there may be other, better ways of doing this.

Anyway, that's my first pass at constructing a median maximising system. Comments and critics welcome!

EDIT: We can use the absolute deviation (technically, the mean absolute deviation around the mean) to bound the distance between median and mean. This itself is bounded by the standard deviation, if it exists.

In HPMOR Eliezer makes "Something to Protect" Harry's power that the Dark Lord doesn't have. In Posture for Mental Arts Valentine from CFAR argues that it's likely a key part of having proper mental posture.

Did any of you make a conscious attempt to develop this sense of having something to protect? If so what worked for you? What didn't work?

Is there relevant academic research on the topic that's useful to know?