You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Irrationality Quotes August 2016

5 PhilGoetz 01 August 2016 07:12PM

Rationality quotes are self-explanatory.  Irrationality quotes often need some context and explication, so they would break the flow in Rationality Quotes.

Rationality Quotes August 2016

2 bbleeker 01 August 2016 09:32AM

Another month, another rationality quotes thread. The rules are:

  • Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
  • Post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
  • Do not quote yourself.
  • Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
  • No more than 5 quotes per person per monthly thread, please.

August 2016 Media Thread

1 ArisKatsaris 01 August 2016 07:00AM

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

  • Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.
  • If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.
  • Please post only under one of the already created subthreads, and never directly under the parent media thread.
  • Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.
  • Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.

Open Thread, Aug. 1 - Aug 7. 2016

4 Elo 01 August 2016 12:12AM

If it's worth saying, but not worth its own post, then it goes here.


Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "

Clean work gets dirty with time

2 Romashka 31 July 2016 07:59PM
Edited for clarity (hopefully) with thanks to Squirrell_in_Hell.

Lately, I find myself more and more interested in how the concept of "systematized winning" can be applied to large groups of people who have one thing in common, and that not even time, but a hobby or a general interest in a specific discipline. It doesn't seem (to me) to much trouble people working on their own individual qualities - performers, martial artists, managers (who would self-identify as belonging to these sets), but I am basing this on "general impressions" and will be glad to be corrected. It does seem to be a norm for some other sets, like sailors, who keep correcting maps every voyage.

The field in which I have been for some years (botany) does have something similar to what sailors do, which lets us to see how floras change over time, etc. However, different questions arise when novel sub-disciplines branch off the main trunk, and naturally, the people asking these new questions keep reaching back for some kind of pre-existing observations. And often they don't check how much weight can be assigned to these observations, which, I think, is a bad habit that won't lead to "winning".

It is not "industrial rationality" per se, but a distantly related thing, and I think we might have to recognize it somehow. Or at least, recognize that it requires different assumptions... No set victory, for example... Still, it probably matters to more living people than pure "industrial rationality" does, & ignoring it won't make it go away.

continue reading »

Should we enable public binding precommitments?

0 capybaralet 31 July 2016 07:47PM

The ability to make arbitrary public binding precommitments seems like a powerful tool for solving coordination problems.

We'd like to be able to commit to cooperating with anyone who will cooperate with us, as in the open-source prisoner's dilemma (although this simple case is still an open problem, AFAIK).  But we should be able to do this piece-meal.

It seems like we are moving in this direction, with things like Etherium that enable smart contracts.  Technology should enable us to enforce more real-world precommitments, since we'll be able to more easily monitor and make public our private data.

Optimistically, I think this could allow us to solve coordination issues robustly enough to have a very low probability of any individual actor making an unsafe AI.  This would require a lot of people to make the right kind of precommitments.

I'm guesing there are a lot of potential downsides and ways it could go wrong, which y'all might want to point out.

[Effective Altruism] Promoting Effective Giving at Conferences via Speed Giving Games

3 Gleb_Tsipursky 30 July 2016 03:16PM

Conferences provide a high-impact opportunity to promote effective giving. This is the broad take-away from an experiment in promoting effective giving at two conferences in recent months: the Unitarian Universalist (UU) General Assembly and the Secular Student Alliance (SSA) National Convention. This was an experiment run by Intentional Insights (InIn), an EA meta-charity devoted to promoting effective giving and rational thinking to a broad audience, with financial sponsorship from The Life You Can Save.

 

The outcomes, as detailed below, suggest that conferences can offer cost-effective opportunities to communicate effective giving messages to important stakeholders. An especially promising way to do so is to use Speed Giving Games (SGG) as a low-threshold strategy since recent findings show GGs are an excellent means of promoting effective giving. This encourages participants to self-organize full-length Giving Games (GG) when they return back to their homes.

 

This article aims both to describe our experiences at UU and SSA and to serve as a guide to others who want to adopt these approaches to promote effective giving via conferences. The article is thus divided into several parts:

  • Evaluating the demographic group you want to target;

  • Evaluating the potential impact and cost of the conference;

  • Steps to prepare for the conference;

  • Outcomes of the conference;

  • Assessment of the experiment and conclusions;


Picking the Right Conference: Consider Demographics

 

Before deciding on a conference, make sure you target the right demographic. We at InIn, in agreement with The Life You Can Save, picked the two conferences mentioned above for a couple of reasons.

 

First, the UU and SSA both unite people who we thought were well-suited for promoting effective giving. Members of these organizations already put a considerable value both on improving the world, and on using reason and evidence to inform their actions in doing so.

 

Our work at SSA is part of our broader effort, in collaboration with The Life You Can Save and the Local Effective Altruist Network, to promote effective giving to secular, humanist, and skeptic groups. We do so by holding GGs targeted to their needs: appearing on podcasts, writing articles in secular venues about effective giving, and collaborating with a number of national and international common-interest organizations. Besides the SSA, this includes the Foundation Beyond Belief, United Coalition of Reason, American Humanist Association, International Humanist Ethical Union, and others.

 

The UU religious denomination is a more experimental focus group. It builds upon the success of the above-mentioned project, and expands to promote effective giving to people who are still somewhat reason-oriented, even if reason is less central for them. Yet UU members are strongly committed to action to improve the world, and generally show more active efforts on the social justice and civic engagement front than members of the secular, humanist, and skeptic movement. Thus, we at InIn and The Life You Can Save decided to target them as well.

 

Second, picking the right demographic also means having at least some people who are familiar with the language, needs, desires, and passions of the niche group you are targeting, and have some connections within it. Knowing the interests and language of the demographics is really valuable for understanding how to frame the concept of effective giving to those demographics. Having people with pre-existing connections and networks within that demographic allows you to approach them as an insider, giving you instant credibility and much more leverage when introducing the audience to an unfamiliar concept.

 

For the SSA, we had it easy, due to our extensive connections in the secular/skeptic/humanist movement. The SSA Executive Director is on the Intentional Insights Advisory Board,  our members regularly appear on podcasts and write for venues within that movement, and many of our members attend local humanist/secular/skeptic groups.

 

We had fewer connections in UU, but the ones that we did have were sufficient. Our two co-founders and some of our members attend UU churches. Intentional insights creates curriculum content for the UU movement, appears on relevant podcasts and writes for major venues. This proved to be more than enough familiarity from the perspective of knowing the language and interests.

 

Picking the Right Conference: Consider Impact and Costs

 

After choosing the right demographic, consider and balance the potential impact and effectiveness of each conference.

 

Number and influence of attendees:

 

Both the UU and the secular/skeptic/humanist movements hold a number of conferences. Fortunately, a single annual conference unites the whole UU movement, with over 3,500 UU leaders from around the world coming. Moreover, the people who come to the UU General Assembly constitute the most active members of the movement – Ministers, Religious Education Directors, church staff, lay leaders and prominent writers – in other words, those stakeholders most capable of spreading effective giving ideas into the UU community.

 

The SSA event had far fewer people, with just over 200 attendees. However, many movers and shakers from the secular/skeptic/humanist movement attend the conference. This makes it attractive from the perspective of spreading effective giving ideas in the movement.

 

Impact of your role at conference:

 

First, most conferences have tabling opportunities for exhibitors, and as an exhibitor, you can hold SGGs at your table. We did that both at the SSA and UU, and I doubt we would have gone to either without that opportunity, since we found it to be very effective at promoting effective giving.

 

Caption: Intentional Insights table at the Secular Student Alliance conference (courtesy of InIn)

 

Second, if you have an opportunity to be a speaker and can promote effective giving at your talk, this raises the impact you can make at a conference. That said, unless you can focus your talk on effective giving or at least give out relevant materials and sign-up sheets, simply mentioning effective giving may not be that impactful. It all depends on how you go about it, and whether the concept is relevant to your talk and memorable to the audience. I was a speaker at the SSA, and worked effective giving into my talk without focusing on it, as well as distributed relevant materials about effective giving.

 

Third, consider whether you have specific networking opportunities at a conference that are  helpful for promoting effective giving. For instance, this might involve having small-group or one-on-one meetings with influencers where you can safely promote effective giving without seeming pushy. At both the SSA and UU, we had both pre-scheduled and spontaneous meetings with notable people, which allowed us to promote effective giving concepts.

 

Costs: One of the fundamental aspects of effective giving is cost-effectiveness, and it is important to apply this metric to marketing effective giving, as well.

 

For the experiment with promoting effective giving at conferences, we at InIn decided to collaborate with The Life You Can Save on the most low-cost opportunities. Thus, one of the reasons we chose the UU and SSA conventions is that they both happened in Columbus, where InIn is based. InIn provided the people who ran the table and did the networking, and The Life You Can Save covered fees for conference registration, tabling, and other miscellaneous fees.

 

The UUA conference registration is around $450 per participant, and $800 for a table. Fortunately, as InIn is a member of a UU organization through which we promote Giving Games and other InIn materials, we were able to use a table at a discount, for $200. Miscellaneous fees included parking and food, for around $20 per participant per day. We had 2 people at the conference each day, so for the 5-day conference, that was $200. We also had about $175 in marketing costs to design and print flyers. We registered only one person, as we got one free participant with a table, so the total cost came down to $1025.

 

The SSA conference registration fee is around $135 per participant, and $150 for a table. As a speaker, I got a free registration, and another free registration accompanied the table. Parking and food cost $140 for the 3-day conference, and marketing costs came out to $150, for a total of $340.

 

Prepare Well

 

To prepare for the conferences, we at InIn brainstormed about the appropriate ways to present effective giving at both conferences. We then prepared talking points relevant to each audience, and coordinated with all people who would table at both conferences to ensure they knew how to present effective giving to the two audiences well.

 

As an example, you can see the GGs packet adapted to the language and interests of the SSA here and UU here. The main modifications are in the “Activity Overview” section, and these changes represent the broad difference in the kind of language we used.

 

Besides the language, we put a lot of effort into designing attractive marketing materials for our table. We created a large sign, visible from a long distance, with “Free Money” in red. People are attracted both to the color red and to the phrase “Free Money,” and it is highly important to draw attention in the context of a busy conference.

 

Caption: SGG activity overview for both UU and SSA conferences (courtesy of InIn)

 

We hired a professional designer to compose an attractive layout for the SGG activity at our table. SGGs involve having people make a decision between two charities. Their vote results in a dollar each going to either charity, sponsored by an outside party, usually The Life You Can Save. It was important to create a nice layout that people could engage with quickly and easily, again due to distractions in the conference setting. We chose GiveDirectly as the effective charity, and the Mid-Ohio Food Bank as a local and not so effective charity.

 

For those who participated in SGGs, then aimed at getting them to sign up for the InIn newsletter and The Life You Can Save newsletter, and engaging with them in conversations about effective giving. We also printed out shorter versions of the UU and SSA Giving Games packets. These had brief descriptions of the full Giving Games, with links to the longer versions they could host back in their SSA student clubs or UU congregations.

 

Another thing we did is schedule meetings in advance with some influencers to discuss effective giving opportunities. We also made sure to schedule meetings spontaneously during the conference with notables who seemed interested in effective giving. For those who expressed an interest but did not have time to meet, we made sure to exchange contact information and follow up afterwards.

 

Finally, we applied to be speakers at both conferences. We succeeded with the SSA, but not with UU. Still, we decided to attend the UU conference, because the costs were low enough since we did not have to travel and The Life You Can Save judged the potential impact worthwhile.

 

Conference Outcomes

 

At the UU conference, we had around 75 people play the SGG, so around 2% of attendees. Of those, about 65% (just under 50 people) signed up for the newsletter. We had 50 packets with GG descriptions printed, and we ran out by the end of the conference. Additionally, about 70% of the people who played there voted for GiveDirectly.

 

We also had meetings with some notable parties interested in effective giving. Especially promising was a meeting with the Executive Director of the Unitarian Universalist Humanist Association (UUHA), who expressed a strong interest in bringing GGs to her constituents. There are hundreds of UU Humanist groups within congregations around the world. We are currently working on testing a GG at a local UU Humanist group, and we will then write up the results for the UUHA blog. We had some other promising meetings as well, but no one was as interested as the UUHA.

 

At the SSA conference, we had 15 people play the SGG, so around 7.5% of attendees. Of those, 80% signed up for the newsletter, so about 12 people. The same proportion, 80%, voted for GiveDirectly.

 

We gave away around 35 GG packets with descriptions, as some people did not want to play the SGG, but were interested in having their clubs host it. Distributing packets was especially helped by the fact that I was a speaker at the SSA, and promoted and handed out packets at my presentation.

 

The meetings with notable parties proved more promising at the SSA. We met with staff from two national secular organizations, the American Ethical Union and the Center for Inquiry, who expressed an interest in promoting GGs to their members. A number of influencers expressed enthusiasm over the concept of effective giving, and wanted to promote it broadly in the secular/skeptic/humanist movement.

 

Assessment and Conclusion

 

We would have been satisfied at both conferences to have at least half of the people who played the SGG vote for GiveDirectly and have half the people sign up. We ended up with 70% voting for GiveDirectly at UU and 80% at SSA, and 65% signing up for the newsletter at UU and 80% at the SSA. So, these conferences strongly exceeded our baseline expectations. We did not have specific expectations for giving away packets or meetings with notables. Yet looking back, we certainly did not expect the level of interest we got for conference participants holding Giving Games back home - we would have printed more packets for the UU had we thought they might run out.

 

The evidence from GGs shows they are a great method to promote effective giving. Getting influencers from target demographics engaged with GGs not only gets the activists to give more effectively, but also encourages the activists to hold GGs back at their groups.

 

After all, holding GGs is a win-win for secular/skeptic/humanist groups and UU congregations alike. They get to engage in an activity that embodies their values of using reason and evidence. At the same time, they get to improve the world and build a sense of community without spending a penny.

 

For those of us promoting effective giving, it presents these ideas to a new audience, and enables the audience to continue engaging if they wish. The newsletter sign-ups are especially indicative of people’s interests. So are the numbers of people who took packets to host GGs back at their groups. We at InIn already heard from several people who are arranging Giving Games after being exposed to the adapted GG packets, including a UU church that is arranging to have a GG for all 500 members of the church. Based on these outcomes, we at InIn and The Life You Can Save decided it would be even worthwhile to invest into traveling to distant conferences given the right conditions - having a table,  speaking role, potential influencers, etc.

 

So, consider promoting effective giving at conferences to audiences not directly related to existing effective altruism communities. Hopefully, the steps I outlined above will help you decide on the best opportunities to do so. I would be glad to chat with you about specifics and share more details; email me at gleb@intentionalinsights.org.

 

Acknowledgments: For feedback on earlier stages of this draft, my gratitude to Jon Behar, Laura Gamse, Ryan Carey, Malcolm Ocean, Matthijs Maas, Yaacov Tarko, Dony Christie, Jake Krycia, Remmelt Ellen, Alexander Semenychev, Ian Pritchford, Ed Chen, Lune Nekesa, Jo Duyvestyn, and others who wished to remain anonymous.

The map of p-zombies

6 turchin 30 July 2016 09:12AM
No real p-zombies exist in any probable way, but a lot of ideas about them have been suggested. This map is the map of ideas. It may be fun or may be useful.

The most useful application of p-zombies research is to determine whether we could loose something important during uploading.

We have to solve the problem of consciousness before we will be uploaded. It will be the most stupid end of the world: everybody is alive and happy but everybody is p-zombie. 

Most ideas here are from Stanford Encyclopedia of Philosophy, Lesswrong wiki, Rational wiki, recent post of EY and from works of Chalmers and Dennett. Some ideas are mine. 

The pdf is here.


Weekly LW Meetups

1 FrankAdamek 29 July 2016 03:45PM

This summary was posted to LW Main on July 29th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, New Hampshire, New York, Philadelphia, Research Triangle NC, San Francisco Bay Area, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

Open thread, Jul. 25 - Jul. 31, 2016

3 MrMind 25 July 2016 07:07AM

If it's worth saying, but not worth its own post, then it goes here.


Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "

A rational unfalsifyable believe

1 Arielgenesis 25 July 2016 02:15AM
A rational unfalsifyable believe

I'm trying to argue that it is possible for someone rational to hold on to a believe that is unfalsifyable and remain rational.

There are three people in a room. Adam, Cain, and Able. Able was murdered. Adam and Cain was taken into police custody. The investigation was thorough but it remains inconclusive. The technology was not advanced enough to produce conclusive evidence. The arguments are basically you did it, no, you did it.

Adam has a wife, her name is Eve. Eve believed that Adam is innocent. She believed so because she has known Adam very well and the Adam that she knew, wouldn't commit murder. She uses Adam's character and her personal relationship with him as evidence.

Cain, trying to defend himself, asked Eve. "What does it take for her to change her believe". She replied, "show me the video recording, then I would believe". But there was no video recording. Then she said, "show me any other evidence that is as strong as a video recording". But there was no such evidence as well.

Cain pointed out, "the evidence that you use for your believe is personal relationship and his character. Then if there are evidence against his character, would you change your mind?"

After some thinking and reflection, she finally said. "Yes, if it could be proven that I have been deceived all these years, then I will believe otherwise."

All of Adam's artifact were gathered, collected and analysed. The search was so thorough, there could never be any new evidence about what Adam had did before the custody that could be presented in the future. All points to Adam good character.

Eve was happy. Cain was not. Then he took one step further. He proposed, "Eve, people could change. If Adam change in the future into man of bad character, would you be convinced that he could have been the murderer?"

"Yes, if Adam changed, then I would believe that it is possible for Adam to be the murderer." Eve said. 

Unfortunately, Adam died the next day. Cain said to Eve, "how do you propose that your belief about Adam's innocence be falsified now?"

"It cannot be falsified now." Eve replied. 

"Then you must be irrational."

  • Is Eve irrational?
  • Can believing an unfalsifyable believe be rational?
  • Can this argument be extended to believe in God?
 

Street Epistemology - letting people name their thinking errors

4 Bound_up 24 July 2016 07:43PM

https://www.youtube.com/watch?v=Exmjlc4PfEQ

 

Anthony Magnabosco does what he calls Street Epistemology, usually applying it to supernatural (usually religious) beliefs.

 

The great thing about his method (and his manner, guy's super personable) is that he avoids the social structure of a debate, of two people arguing, of a zero-sum one game where person wins at the other's loss.

 

I've struggled with trying to figure out how to let people save face in disputes (when they're making big, awful mistakes), even considering including minor errors (that don't affect the main point) in my arguments so that they could point them out and we could both admit we were wrong (in their case, about things which do affect the main point) and move on.

 

But this guy's technique manages to invite people to correct their own errors (people are SOOOO much more rational when they're not defensive) and they DO it. No awkwardness, no discomfort, and people pointing out the flaws in their own arguments, and then THANKING him for the talk afterwards and referring him to their friends to talk. Even though they just admitted that their cherished beliefs might not deserve the certainty they've been giving them.

 

This is applied to religion in this video, but this seems to me to be a generally useful method when you confront someone making an error in their thinking. Are you forcing people to swallow their pride a little (over and over) when they talk with you? Get that out, and watch how much more open people can be.

Weekly LW Meetups

1 FrankAdamek 22 July 2016 04:00PM

This summary was posted to LW Main on July 22nd. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, New Hampshire, New York, Philadelphia, Research Triangle NC, San Francisco Bay Area, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

[Link] Suffering-focused AI safety: Why “fail-safe” measures might be particularly promising

8 wallowinmaya 21 July 2016 08:22PM

The Foundational Research Institute just published a new paper: "Suffering-focused AI safety: Why “fail-safe” measures might be our top intervention". 

It is important to consider that [AI outcomes] can go wrong to very different degrees. For value systems that place primary importance on the prevention of suffering, this aspect is crucial: the best way to avoid bad-case scenarios specifically may not be to try and get everything right. Instead, it makes sense to focus on the worst outcomes (in terms of the suffering they would contain) and on tractable methods to avert them. As others are trying to shoot for a best-case outcome (and hopefully they will succeed!), it is important that some people also take care of addressing the biggest risks. This perspective to AI safety is especially promising both because it is currently neglected and because it is easier to avoid a subset of outcomes rather than to shoot for one highly specific outcome. Finally, it is something that people with many different value systems could get behind.

Fallacymania: party game where you notice fallacies in arguments

4 Alexander230 21 July 2016 09:34AM

Fallacymania is a game developed by Moscow LessWrong community. Main goals of this game is to help people notice fallacies in arguments, and of course to have fun. The game requires 3-20 players (recommended 4-12), and some materials: printed A3 sheets with fallacies (5-10 sheets), card deck with fallacies (you can cut one A3 sheet into cards, or print stickers and put them to common playing cards), pens and empty sheets, and 1 card deck of any type with at least 50 cards (optional, for counting guessing attempts). Rules of the game are explained here:

https://drive.google.com/open?id=0BzyKVqP6n3hKQWNzV3lWRTYtRzg

This is the sheet of fallacies, you can download it and print on A3 or A2 sheet of paper:

https://drive.google.com/open?id=0BzyKVqP6n3hKVEZSUjJFajZ2OTA

Also you can use this sheet to create playing cards for debaters.

When we created this game, we used these online articles and artwork about fallacies:

http://obraz.io/ru/posters/poster_view/1/?back_link=%2Fru%2F&lang=en&arrow=right
http://www.informationisbeautiful.net/visualizations/rhetological-fallacies/
http://lesswrong.com/lw/e95/the_noncentral_fallacy_the_worst_argument_in_the/

Also I've made electronic version of Fallacymania for Tabletop Simulator (in Steam Workshop):

http://steamcommunity.com/sharedfiles/filedetails/?id=723941480

 

In partially observable environments, stochastic policies can be optimal

5 Stuart_Armstrong 19 July 2016 10:42AM

I always had the informal impression that the optimal policies were deterministic (choosing the best option, rather than some mix of options). Of course, this is not the case when facing other agents, but I had the impression this would hold when facing the environment rather that other players.

But stochastic policies can also be needed if the environment is partially observable, at least if the policy is Markov (memoryless). Consider the following POMDP (partially observable Markov decision process):

There are two states, 1a and 1b, and the agent cannot tell which one they're in. Action A in state 1a and B in state 1b, gives a reward of -R and keeps the agent in the same place. Action B in state 1a and A in state 1b, gives a reward of R and moves the agent to the other state.

The returns for the two deterministic policies - A and B - are -R every turn except maybe for the first. While the return for the stochastic policy of 0.5A + 0.5B is 0 per turn.

Of course, if the agent can observe the reward, the environment is no longer partially observable (though we can imagine the reward is delayed until later). And the general policy of "alternate A and B" is more effective that the 0.5A + 0.5B policy. Still, that stochastic policy is the best of the memoryless policies available in this POMDP.

Open thread, Jul. 18 - Jul. 24, 2016

3 MrMind 18 July 2016 07:17AM

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.


Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

Earning money with/for work in AI safety

7 rmoehn 18 July 2016 05:37AM

(I'm re-posting my question from the Welcome thread, because nobody answered there.)

I care about the current and future state of humanity, so I think it's good to work on existential or global catastrophic risk. Since I've studied computer science at a university until last year, I decided to work on AI safety. Currently I'm a research student at Kagoshima University doing exactly that. Before April this year I had only little experience with AI or ML. Therefore, I'm slowly digging through books and articles in order to be able to do research.

I'm living off my savings. My research student time will end in March 2017 and my savings will run out some time after that. Nevertheless, I want to continue AI safety research, or at least work on X or GC risk.

I see three ways of doing this:

  • Continue full-time research and get paid/funded by someone.
  • Continue research part-time and work the other part of the time in order to get money. This work would most likely be programming (since I like it and am good at it). I would prefer work that helps humanity effectively.
  • Work full-time on something that helps humanity effectively.


Oh, and I need to be location-independent or based in Kagoshima.

I know http://futureoflife.org/job-postings/, but all of the job postings fail me in two ways: not location-independent and requiring more/different experience than I have.

Can anyone here help me? If yes, I would be happy to provide more information about myself.

(Note that I think I'm not in a precarious situation, because I would be able to get a remote software development job fairly easily. Just not in AI safety or X or GC risk.)

[Link] NYU conference: Ethics of Artificial Intelligence (October 14-15)

4 ignoranceprior 16 July 2016 09:07PM

FYI: https://wp.nyu.edu/consciousness/ethics-of-artificial-intelligence/

This conference will explore these questions about the ethics of artificial intelligence and a number of other questions, including:

What ethical principles should AI researchers follow?
Are there restrictions on the ethical use of AI?
What is the best way to design morally beneficial AI?
Is it possible or desirable to build moral principles into AI systems?
When AI systems cause benefits or harm, who is morally responsible?
Are AI systems themselves potential objects of moral concern?
What moral framework is best used to assess questions about the ethics of AI?

Speakers and panelists will include:

Nick Bostrom (Future of Humanity Institute), Meia Chita-Tegmark (Future of Life Institute), Mara Garza (UC Riverside, Philosophy), Sam Harris (Project Reason), Demis Hassabis (DeepMind/Google), Yann LeCun (Facebook, NYU Data Science), Peter Railton (University of Michigan, Philosophy), Francesca Rossi (University of Padova, Computer Science), Stuart Russell (UC Berkeley, Computer Science), Susan Schneider (University of Connecticut, Philosophy), Eric Schwitzgebel (UC Riverside, Philosophy), Max Tegmark (Future of Life Institute), Wendell Wallach (Yale, Bioethics), Eliezer Yudkowsky (Machine Intelligence Research Institute), and others.

Organizers: Ned Block (NYU, Philosophy), David Chalmers (NYU, Philosophy), S. Matthew Liao (NYU, Bioethics)

A full schedule will be circulated closer to the conference date.

Registration is free but required. REGISTER HERE. Please note that admission is limited, and is first-come first-served: it is not guaranteed by registration.

Two forms of procrastination

10 Viliam 16 July 2016 08:30PM

I noticed something about myself when comparing two forms of procrastination:

a) reading online discussions,
b) watching movies online.

Reading online discussions (LessWrong, SSC, Reddit, Facebook) and sometimes writing a comment there, is a huge sink of time for me. On the other hand, watching movies online is almost harmless, at least compared with the former option. The difference is obvious when I compare my productivity at the end of the day when I did only the former, or only the latter. The interesting thing is that at the moment it feels the other way round.

When I start watching a movie that is 1:30:00 long, or start watching a series where each part is 40:00 long but I know I will probably watch more than one part a day, I am aware from the beginning that I am going to lose more than one hour of time; possibly several hours. On the other hand, when I open the "Discussion" tab on LessWrong, the latest "Open Thread" on SSC, my few favorite subreddits, and/or my Facebook "Home" page, it feels like it will only take a few minutes -- I will click on the few interesting links, quickly skim through the text, and maybe write a comment or two -- it certainly feels like much less than an hour.

But the fact is, when I start reading the discussions, I will probably click on at least hundred links. Most of the pages I will read just as quickly as I imagined, but there will be a few that will take disproportionally more time; either because they are interesting and long, or because they contain further interesting links. And writing a comment sometimes takes more time than it seems; it can easily be a half an hour for a three-paragraphs-long comment. (Ironically, this specific article gets written rather quickly, because I know what I want to write, and I write it directly. But there are comments where I think a lot, and keep correcting my text, to avoid misunderstanding when debating a sensitive topic, etc.) And when I stop doing it, because I want to make something productive for a change, I will feel tired. Reading many different things, trying to read quickly, and formulating my answers, that all makes me mentally exhausted. So after I close the browser, I just wish I could take a nap.

On the other hand, watching a movie does not make me tired in that specific way. The movies runs at its own speed and doesn't require me to do anything actively. Also, there is no sense of urgency; none of the "if I reply to this now, people will notice and respond, but if I do it a week later, no one will care anymore". So I feel perfectly comfortable pausing the movie at any moment, doing something productive for a while, then unpausing the movie and watching more. I know I won't miss anything.

I think it's the mental activity during my procrastination that both makes me tired and creates the illusion that it will take less time than it actually does. When the movie says 1:30:00, I know it will be 1:30:00 (or maybe a little less because of the final credits). With a web page, I can always tell myself "don't worry, I will read this one really fast", so there is the illusion that I have it under control, and can reduce the time waste. The fact that I am reading an individual page really fast makes me underestimate how much time it took to read all those pages.

On the other hand, sometimes I do inverse procrastination -- I start watching a movie, pause it a dozen times and do some useful work during the breaks -- and at the end of the day I spent maybe 90% of the time working productively, while my brain tells me I just spent the whole day watching a movie, so I almost feel like I had a free day.

Okay, so how could I use this knowledge to improve my productivity?

1) Knowing the difference between the two forms of procrastination, whenever I feel a desire to escape to the online world, I should start watching a movie instead of reading some debate, because thus I will waste less time, even if it feels the other way round.

2) Integrate it with pomodoro? 10 minutes movie, 50 minutes work, then again, and at the end of the day my lying brain will tell me "dude, you didn't work at all today, you were just watching movies, of course you should feel awesome!".

Do you have a similar experience? No idea how typical is this. No need to hurry with responding, I am going to watch a movie now. ;-)

Quick puzzle about utility functions under affine transformations

6 Liron 16 July 2016 05:11PM

Here's a puzzle based on something I used to be confused about:

It is known that utility functions are equivalent (i.e. produce the same preferences over actions) up to a positive affine transformation: u'(x) = au(x) + b where a is positive.

Suppose I have u(vanilla) = 3, u(chocolate) = 8. I prefer an action that yields a 50% chance of chocolate over an action that yields a 100% chance of vanilla, because 0.5(8) > 1.0(3).

Under the positive affine transformation a = 1, b = 4; we get that u'(vanilla) = 7 and u'(chocolate) = 12. Therefore I now prefer the action that yields a 100% chance of vanilla, because 1.0(7) > 0.5(12).

How to resolve the contradiction?

 

The map of cognitive biases, errors and obstacles affecting judgment and management of global catastrophic risks

3 turchin 16 July 2016 12:11PM
“We had two bags of grass, seventy-five pellets of mescaline, five sheets of high powered blotter acid, a salt shaker half full of cocaine, and a whole galaxy of multi-colored uppers, downers, screamers, laughers... and also a quart of tequila, a quart of rum, a case of Budweiser, a pint of raw ether and two dozen amyls. Not that we needed all that for the trip, but once you get locked into a serious drug collection, the tendency is to push it as far as you can.”

The map is based on 100+ page text which lists a 200+ biases and other cognitive problems which may arise during identification and prevention of x-risks. 

The map is very preliminary, after all. Much more biases could be identified. The main idea is to show that there are many biases, so any our judgement about x-risks will be distorted. Larger safety margins though required.

The map also intended to show that we could classify biases and other obstacles, and it could help to navigate us to truth and safety. The goal of the map is to show complexity of prediction and management of future x-risks, as well as help to identify sources of mistakes.

Each box has link on full explanation of the bias.

Boxes which were covered in Eliezer Yudkowsky about cognitive biases article are red.

Numbers in the boxes arrow on paragraphs in the full text.

The Y axis of the map shows stages of x-risk prevention in time, on which certain biases may work. Some biases could influence assessment of risks and other will show themselves in management of dangerous situations.

The X axis gives very rough typology of biases based of their sources.
See Bostrom's article for the similar typology:  

He suggests 3 types of biases: believes, shortcomings of our mind and ignorance.

Sorry that it all looks like tetris. ))

Read pdf with clickable links.


[CORE] Concepts for Understanding the World

2 SquirrelInHell 16 July 2016 10:53AM

Background:

I'm recently doing a big project to increase my scholarship and modeling power for both rationality and traditional "serious" topics. One thing I found very useful is taking notes with a clear structure.

The structure I'm using currently is as follows:

- write down useful concepts,

- write down (as a separate category) useful heuristics & things to do in various situations,

- do not write facts, opinions or anything else (I rely on unaided memory to get more filtering).

Heuristic: learn concepts before facts!

Note that you can be mistaken about facts, but you can't harm your epistemology by learning concepts. Even if a concept turns out to be useless or misleading, you are better off knowing about it, understanding how it's misleading, and being able to avoid the trap when you see it.

Let's share concepts!

Please give (at a minimum) a name and a reference (link). A short description in plain language is also welcome.

 



I started to write these "concept" notes electronically only 2 days ago, so the list below is VERY random and sparse. In any case, let's make this a collective effort :)

[ Added: up-to-date version of the list below is here: http://squirrelinhell.github.io/concepts/ ]



adaptation-executer
➡ organism acts only to execute evolved adaptations
➡ it does not depend on awareness of the goal (maximizing inclusive fitness)

adaptive capacity
➡ ability of a system to adapt to changing environment

akrasia
➡ acting against your own better judgement

Aristotelian epistemology
➡ statements are true or false
➡ truth can be shown by deductive reasoning

base rate neglect
➡ using specific evidence as if it replaced priors

Bayesian epistemology
➡ beliefs have degrees of certainty from 0 to 1
➡ updated upon seeing evidence

behaviorism
➡ an approach that explains behavior by conditioning (rather than thoughts or emotions)

brotherhood
➡ all-male social group with rules that prevent girl-related drama within the group

CDT Stage 1
➡ subject to reflexes
➡ early childhood

CDT Stage 2
➡ reflexes are objects
➡ subject to personal interests (desires, needs, wishes)
➡ childhood to early adolescence

CDT Stage 3
➡ personal interests are objects
➡ subject to relationships/social pressure

CDT Stage 4
➡ relationships are objects
➡ subject to systems (principles, rules, structure, commitments)

CDT Stage 5
➡ systems are objects
➡ can juggle around multiple systems
➡ can handle errors or incompleteness of a system

Chesterton Fence
➡ rule not to do reforms without understanding reasoning behind current state

cognitive ease
➡ how hard it is to think something gets used as a proxy for more complicated judgements

Conservation of Expected Evidence
➡ expectation of posterior probability is equal to prior probability
➡ unlikely strong evidence is balanced by likely weak evidence in the opposite direction
➡ absence of evidence is (weak) evidence of absence

Counterfactual Mugging
➡ Omega says it had tossed a fair coin to determine if you win $10000 or lose $100
➡ you only get $10000 if you would accept loss of $100
➡ it is already known that you lost
➡ do you pay $100?

countersignaling
➡ not signaling X to show that you are above people who signal X
➡ doesn't work when confused with lower level

Curse of Development
➡ when making progress, effectiveness will often decrease before it becomes higher than ever

denominator neglect
➡ in "X out of Y", absolute value of X influences intuitive judgement

dimimishing returns
➡ decreasing marginal output, as input is incrementally increased

Dunnig-Kruger effect
➡ bias in which relatively unskilled persons overestimate their skill a lot

ending on a high note
➡ the last part leaves the strongest impression
➡ either (1) do something positive at the end
➡ or (2) end the interaction after something positive happens

exposure therapy
➡ treatment for anxiety or phobias by forced prolonged exposure while safety is guaranteed

forced legibility
➡ reorganising a complex system which seems irrational, but it was only a failure to understand it

halo effect
➡ seeing something positive or negative influences judgement of all other aspects

homeostasis
➡ property of a system that regulates a variable to keep it at a constant level

honing mode
➡ conversation converges on an idea
➡ focused on corrections and critique

hyperbolic discounting
➡ value(t) = 1/(1 + C * t)
➡ intuition discounts approximately like this
➡ any discounting except exponential is inconsistent under passage of time

inclusive fitness
➡ ability to pass on genes (including genes passed on by relatives)

iterated hurt
➡ someone is hurt by the knowledge that you were willing to hurt them
➡ this works even if there is no actual hurt

jamming mode
➡ conversation is a divergent exploration
➡ based on remixing and building on ideas

Laws of Authority
➡ enforced by a selected powerful entity

Laws of Reality
➡ enforced automatically by the environment

Laws of Society
➡ enforced by group consensus
➡ large majority finds the terms beneficial or acceptable
➡ defectors are punished by volunteers

learned blank
➡ taking for granted lack of skill/knowledge in some area

level reversal
➡ level N+1 is sometimes superficially similar to level N-1
➡ that the next level is counterintuitive is what makes levels recognizable in the first place

loss aversion
➡ intuitive judgement is based on losses and gains relative to a "reference point"
➡ losses weigh around 2 times more than gains

lost purpose
➡ pursuing an instrumental goal that no longer has value

marginal cost
➡ change of cost per one additional unit of produced resource

marginal utility
➡ change of utility per one additional unit of consumed resource

marginalism
➡ a theory that explains prices of products in terms of their marginal utility

meta-contrarianism
➡ being contrarian to a contrarian position
➡ might be more concerned with signaling than accuracy

motivation system
➡ outputs "wanting" and "not wanting"
➡ implemented entirely in S1

mutual knowledge
➡ everyone knows, and they know that everyone knows, and they know that others know that everyone knows, etc.

mystic epistemology
➡ everything is possible
➡ let's believe nothing

Newcomb's problem
➡ Omega prepared a trasparent box and an opaque box
➡ transparent box contains $1000
➡ opaque box contains $1,000,000 iff Omega predicted you will take only the opaque box
➡ do you take both boxes, or only the opaque box?

Nyquist frequency
➡ half of the sampling rate
➡ for every sinusoid with frequency above, there is a sinusoid (alias) that has the same samples and frequency below

Parfit's Hitchhiker
➡ you will be saved from death on a desert by a car driver iff you promise to pay $100 later
➡ the driver can detect lying perfectly
➡ does your decision theory allow you to commit to paying with certainty you won't change it later?

Peter principle
➡ if promotion is based on performance in the current role, then it stabilizes on reaching incompetence

planning system
➡ consciously building chains of actions
➡ can imagine intermediate states

regression to the mean
➡ prediction from a correlated variable is attenuated by amount of correlation
➡ more noise or smaller sample requires bigger adjustment

reinforcement learning
➡ agent interacts with the environment
➡ correct actions are learned from when it gets rewards

Schelling fence
➡ "arbitrary boundary used to prevent a ""slippery slope"" situation"

Schelling point
➡ prediction of what others expect you to think you are expected to do

self-reinforcement
➡ using conditioning to influence your own patterns of behaviour
➡ doesn't work (learning is based on surprise)
➡ some techniques that work might look very similar (they actually manipulate emotions)

Smoking Lesion
➡ alternative world in which smoking is correlated with cancer but does not cause it
➡ there is a genetic lesion that increases chances of both
➡ you don't know if you have it
➡ you want to smoke, but you dislike cancer much more
➡ do you smoke?

societal collapse
➡ disintegration of a human society
➡ often together with most civilization advances

status illegibility
➡ social groups are held together by unclear relative statuses in the middle section

subject-object model
➡ a view of development as a shift from being subject to X, to manipulating X as an object in context
➡ intermediate stages: becoming aware of broader view, having 2 conflicting views, adopting the broader view with occasional lapses

sunk cost fallacy
➡ including resources that have already been spent in a decision about the future

S1 storytelling
➡ System 1 interprets data by constructing plausible stories
➡ it does it more easily with *less* data

System 1
➡ unconscious, effortless thinking
➡ fast, parallel and runs all the time

System 2
➡ conscious, effortful thinking
➡ slow, serial and runs on demand
➡ it can install new patterns of behaviour, but sucks at controlling anything directly
➡ uses working memory

ugh field
➡ mental flinch from thinking about something (or even admitting there is a problem)

vivid probability
➡ unlikely outcomes/events are overweighted when they evoke vivid mental imagery, and neglected otherwise
➡ e.g. explicitly mentioning, adding details, presenting probability as frequency all contribute to overweighing

New LW Meetup: Greenville NC

0 FrankAdamek 15 July 2016 03:45PM

This summary was posted to LW Main on July 15th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, New Hampshire, New York, Philadelphia, Research Triangle NC, San Francisco Bay Area, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

confirmation bias, thought experiment

1 Douglas_Reay 15 July 2016 12:19PM

Why do people end up with differing conclusions, given the same data?

 

Model

The information we get from others can not always be 100% relied upon.  Some of the people telling you stuff are liars, some are stupid, and some are incorrectly or insufficiently informed.  Even in the case where the person giving you an opinion is honest, smart and well informed, they are still unlikely to be able to tell you accurately how reliable their own opinion is.

So our brains use an 'unreliability' factor.  Automatically we take what others tell us, and discount it by a certain amount, depending on how 'unreliable' we estimate the source to be.

We also compare what people tell us about 'known reference points' in order to update our estimates of their unreliability.

If Sally tells me that vaccines cause AIDS and I am very much more certain that this is not the case, than I am of Sally's reliability, then instead of modifying my opinion about what causes AIDS, I modify my opinion of how reliable Sally is.

If I'm only slightly more certain, then I might take the step of asking Sally her reason for thinking that, and looking at her data.

If I have a higher opinion of Sally than my own knowledge of science, and I don't much care or am unaware of what other people think about the relationship between vaccines and AIDS, then I might just accept what she says, provisionally, without checking her data.

If I have a very much higher opinion of Sally, then not only will I believe her, but my opinion of her reliability will actually increase as I assess her as some mould-breaking genius who knows things that others do not.

 

Importantly, once we have altered our opinion, based upon input that we originally considered to be fairly reliable, we are very bad at reversing that alteration, if the input later turns out to be less reliable than we originally thought.  This is called the "continued influence effect", and we can use it to explain a number of things...

 

Experiment

Let us consider a thought experiment where two subjects, Peter and Paul, are exposed to input about a particular topic (such as "Which clothes washing powder is it best to use?") from multiple sources.   Both will be exposed to the same sources, 100 in favour of using the Persil brand of washing powder, and 100 in favour of using the Bold brand of washing powder, but in a different order.

If they both start off with no strong opinion in either direction, would we expect them to end the experiment with roughly the same opinion as each other, or can we manipulate their opinions into differing, just by changing the order in which the sources are presented?

Suppose, with Peter, we start him off with 10 of the Persil side's most reputable and well argued sources, to raise Peter's confidence in sources that support Persil.

We can then run another 30 much weaker pro-Persil sources past him, and he is likely to just nods and accept, without bothering to examine the validity of the arguments too closely, because he's already convinced.

At this point, when he'll consider a source to be a bit suspect, straight away, just because they don't support Persil, we introduce him to the pro-Bold side, starting with the least reliable - the ones that are obviously stupid or manipulative.   Further more, we don't let the pro-Bold side build up momentum.   For every three poor pro-Bold sources, we interrupt with a medium reliability pro-Persil source that's rehashing pro-Persil points that Peter is by now familiar with and agrees with.

After seeing the worst 30 pro-Bold sources, Peter now don't just consider them to be a bit suspect - he considers them to be down right deceptive and mentally categorises all such sources as not worth paying attention to.   Any further pro-Bold sources, even ones that seem to be impartial and well reasoned, he's going to put down as being fakes created by malicious researchers in the pay of an evil company.

We can now, safely, expose Peter to the medium-reliability pro-Bold sources and even the good ones, and will need less and less to refute them, just a reminder to Peter of 'which side he is on', because it is less about the data now, and more about identity - he doesn't see himself as the sort of person who'd support Bold.   He's not a sheep.  He's not taken in by the hoax.

Finally, after 80 pro-Persil sources and 90 pro-Bold sources, we have 10 excellent pro-Bold sources whose independence and science can't fairly be questioned.   But it is too late for them to have much effect, and there are 20 good pro-Persil sources to balance them.

For Paul we do the reverse, starting with pro-Bold sources and only later introducing the pro-Persil side once a known reference point has been established as an anchor.

 

Simulation

Obviously, things are rarely that clear cut in real life.   But people also don't often get data from both sides of an argument at a precisely equal rate.   They bump around randomly, and once one side accumulates some headway, it is unlikely to be reversed.

We could add a third subject, Mary, and consider what is likely to happen if she is exposed to a random succession of sources, each with a 50% chance of supporting one side or the other, and each with a random value on a scale of 1(poor) to 3 (good) for honesty, validity and strength of conclusion supported by the claimed data.

If we use mathematics to make some actual models of the points at which a source agreeing or disagreeing with you affects your estimate of their reliability, we can use a computer simulation of the above thought experiment to predict how different orders of presentation will affect people's final opinion, under each model.   Then we could compare that against real-world data, to see which model best matches reality.

 

Prediction

I think, if this experiment were carried out, one of the properties that would emerge naturally from it is the backfire effect:

" The backfire effect occurs when, in the face of contradictory evidence, established beliefs do not change but actually get stronger. The effect has been demonstrated experimentally in psychological tests, where subjects are given data that either reinforces or goes against their existing biases - and in most cases people can be shown to increase their confidence in their prior position regardless of the evidence they were faced with. "

 

Further Reading

https://en.wikipedia.org/wiki/Confirmation_bias
https://en.wikipedia.org/wiki/Attitude_polarization
http://www.dartmouth.edu/~nyhan/nyhan-reifler.pdf
http://www.tandfonline.com/doi/abs/10.1080/17470216008416717
http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/
http://www.tandfonline.com/doi/abs/10.1080/14640749508401422
http://rationalwiki.org/wiki/Backfire_effect

Wikipedia usage survey results

7 riceissa 15 July 2016 12:49AM

Contents

Summary

The summary is not intended to be comprehensive. It highlights the most important takeaways you should get from this post.

  • Vipul Naik and I are interested in understanding how people use Wikipedia. One reason is that we are getting more people to work on editing and adding content to Wikipedia. We want to understand the impact of these edits, so that we can direct efforts more strategically. We are also curious!

  • From May to July 2016, we conducted two surveys of people’s Wikipedia usage. We collected survey responses from audience segments include Slate Star Codex readers, Vipul’s Facebook friends, and a few audiences through SurveyMonkey Audience and Google Consumer Surveys. Our survey questions measured how heavily people use Wikipedia, what sort of pages they read or expected to find, the relation between their search habits and Wikipedia, and other actions they took within Wikipedia.

  • Different audience segments responded very differently to the survey. Notably, the SurveyMonkey audience (which is closer to being representative of the general population) appears to use Wikipedia a lot less than Vipul’s Facebook friends and Slate Star Codex readers. Their consumption of Wikipedia is also more passive: they are less likely to explicitly seek Wikipedia pages when searching for a topic, and less likely to engage in additional actions on Wikipedia pages. Even the college-educated SurveyMonkey audience used Wikipedia very little.

  • This is tentative evidence that Wikipedia consumption is skewed towards a certain profile of people (and Vipul’s Facebook friends and Slate Star Codex readers sample much more heavily from that profile). Even more tentatively, these heavy users tend to be more “elite” and influential. This tentatively led us to revise upward our estimates of the social value of a Wikipedia pageview.

  • This was my first exercise in survey construction. I learned a number of lessons about survey design in the process.

  • All the survey questions, as well as the breakdown of responses for each of the audience segments, are described in this post. Links to PDF exports of response summaries are at the end of the post.

Background

At the end of May 2016, Vipul Naik and I created a Wikipedia usage survey to gauge the usage habits of Wikipedia readers and editors. SurveyMonkey allows the use of different “collectors” (i.e. survey URLs that keep results separate), so we circulated several different URLs among four locations to see how different audiences would respond. The audiences were as follows:

  • SurveyMonkey’s United States audience with no demographic filters (62 responses, 54 of which are full responses)
  • Vipul Naik’s timeline (post asking people to take the survey; 70 responses, 69 of which are full responses). For background on Vipul’s timeline audience, see his page on how he uses Facebook.
  • The Wikipedia Analytics mailing list (email linking to the survey; 7 responses, 6 of which are full responses). Note that due to the small size of this group, the results below should not be trusted, unless possibly when the votes are decisive.
  • Slate Star Codex (post that links to the survey; 618 responses, 596 of which are full responses). While Slate Star Codex isn’t the same as LessWrong, we think there is significant overlap in the two sites’ audiences (see e.g. the recent LessWrong diaspora survey results).
  • In addition, although not an actual audience with a separate URL, several of the tables we present below will include an “H” group; this is the heavy users group of people who responded by saying they read 26 or more articles per week on Wikipedia. This group has 179 people: 164 from Slate Star Codex, 11 from Vipul’s timeline, and 4 from the Analytics mailing list.

We ran the survey from May 30 to July 9, 2016 (although only the Slate Star Codex survey had a response past June 1).

After we looked at the survey responses on the first day, Vipul and I decided to create a second survey to focus on the parts from the first survey that interested us the most. The second survey was only circulated among SurveyMonkey’s audiences: we used SurveyMonkey’s US audience with no demographic filters (54 responses), as well as a US audience of ages 18–29 with a college or graduate degree (50 responses). We first ran the survey on the unfiltered audience again because the wording of our first question was changed and we wanted to have the new baseline. We then chose to filter for young college-educated people because our prediction was that more educated people would be more likely to read Wikipedia (the SurveyMonkey demographic data does not include education, and we hadn’t seen the Pew Internet Research surveys in the next section, so we were relying on our intuition and some demographic data from past surveys) and because young people in our first survey gave more informative free-form responses in survey 2 (SurveyMonkey’s demographic data does include age).

We ran a third survey on Google Consumer Surveys with a single question that was a word-to-word replica of the first question from the second survey. The main motivation here was that on Google Consumer Surveys, a single-question survey costs only 10 cents per response, so it was possible to get to a large number of responses at relatively low cost, and achieve more confidence in the tentative conclusions we had drawn from the SurveyMonkey surveys.

Previous surveys

Several demographic surveys regarding Wikipedia have been conducted, targeting both editors and users. The surveys we found most helpful were the following:

  • The 2010 Wikipedia survey by the Collaborative Creativity Group and the Wikimedia Foundation. The explanation before the bottom table on page 7 of the overview PDF has “Contributors show slightly but significantly higher education levels than readers”, which provides weak evidence that more educated people are more likely to engage with Wikipedia.
  • The Global South User Survey 2014 by the Wikimedia Foundation
  • Pew Internet Research’s 2011 survey: “Education level continues to be the strongest predictor of Wikipedia use. The collaborative encyclopedia is most popular among internet users with at least a college degree, 69% of whom use the site.” (page 3)
  • Pew Internet Research’s 2007 survey

Note that we found the Pew Internet Research surveys after conducting our own two surveys (and during the write-up of this document).

Motivation

Vipul and I ultimately want to get a better sense of the value of a Wikipedia pageview (one way to measure the impact of content creation), and one way to do this is to understand how people are using Wikipedia. As we focus on getting more people to work on editing Wikipedia – thus causing more people to read the content we pay and help to create – it becomes more important to understand what people are doing on the site.

For some previous discussion, see also Vipul’s answers to the following Quora questions:

Wikipedia allows relatively easy access to pageview data (especially by using tools developed for this purpose, including one that Vipul made), and there are some surveys that provide demographic data (see “Previous surveys” above). However, after looking around, it was apparent that the kind of information our survey was designed to find was not available.

I should also note that we were driven by our curiosity of how people use Wikipedia.

Survey questions for the first survey

For reference, here are the survey questions for the first survey. A dummy/mock-up version of the survey can be found here: https://www.surveymonkey.com/r/PDTTBM8.

The survey introduction said the following:

This survey is intended to gauge Wikipedia use habits. This survey has 3 pages with 5 questions total (3 on the first page, 1 on the second page, 1 on the third page). Please try your best to answer all of the questions, and make a guess if you’re not sure.

And the actual questions:

  1. How many distinct Wikipedia pages do you read per week on average?

    • less than 1
    • 1 to 10
    • 11 to 25
    • 26 or more
  2. On a search engine (e.g. Google) results page, do you explicitly seek Wikipedia pages, or do you passively click on Wikipedia pages only if they show up at the top of the results?

    • I explicitly seek Wikipedia pages
    • I have a slight preference for Wikipedia pages
    • I just click on what is at the top of the results
  3. Do you usually read a particular section of a page or the whole article?

    • Particular section
    • Whole page
  4. How often do you do the following? (Choices: Several times per week, About once per week, About once per month, About once per several months, Never/almost never.)

    • Use the search functionality on Wikipedia
    • Be surprised that there is no Wikipedia page on a topic
  5. For what fraction of pages you read do you do the following? (Choices: For every page, For most pages, For some pages, For very few pages, Never. These were displayed in a random order for each respondent, but displayed in alphabetical order here.)

    • Check (click or hover over) at least one citation to see where the information comes from on a page you are reading
    • Check how many pageviews a page is getting (on an external site or through the Pageview API)
    • Click through/look for at least one cited source to verify the information on a page you are reading
    • Edit a page you are reading because of grammatical/typographical errors on the page
    • Edit a page you are reading to add new information
    • Look at the “See also” section for additional articles to read
    • Look at the editing history of a page you are reading
    • Look at the editing history solely to see if a particular user wrote the page
    • Look at the talk page of a page you are reading
    • Read a page mostly for the “Criticisms” or “Reception” (or similar) section, to understand different views on the subject
    • Share the page with a friend/acquaintance/coworker

For the SurveyMonkey audience, there were also some demographic questions (age, gender, household income, US region, and device type).

Survey questions for the second survey

For reference, here are the survey questions for the second survey. A dummy/mock-up version of the survey can be found here: https://www.surveymonkey.com/r/28BW78V.

The survey introduction said the following:

This survey is intended to gauge Wikipedia use habits. Please try your best to answer all of the questions, and make a guess if you’re not sure.

This survey has 4 questions across 3 pages.

In this survey, “Wikipedia page” refers to a Wikipedia page in any language (not just the English Wikipedia).

And the actual questions:

  1. How many distinct Wikipedia pages do you read (at least one sentence of) per week on average?

    • Fewer than 1
    • 1 to 10
    • 11 to 25
    • 26 or more
  2. Which of these articles have you read (at least one sentence of) on Wikipedia (select all that apply)? (These were displayed in a random order except the last option for each respondent, but displayed in alphabetical order except the last option here.)

    • Adele
    • Barack Obama
    • Bernie Sanders
    • China
    • Donald Trump
    • Google
    • Hillary Clinton
    • India
    • Japan
    • Justin Bieber
    • Justin Trudeau
    • Katy Perry
    • Taylor Swift
    • The Beatles
    • United States
    • World War II
    • None of the above
  3. What are some of the Wikipedia articles you have most recently read (at least one sentence of)? Feel free to consult your browser’s history.

  4. Recall a time when you were surprised that a topic did not have a Wikipedia page. What were some of these topics?

Survey questions for the third survey (Google Consumer Surveys)

This survey had exactly one question. The wording of the question was exactly the same as that of the first question of the second survey.

  1. How many distinct Wikipedia pages do you read (at least one sentence of) per week on average?

    • Fewer than 1
    • 1 to 10
    • 11 to 25
    • 26 or more

One slight difference was that whereas in the second survey, the order of the options was fixed, the third survey did a 50/50 split between that order and the exact reverse order. Such splitting is a best practice to deal with any order-related biases, while still preserving the logical order of the options. You can read more on the questionnaire design page of the Pew Research Center.

Results

In this section we present the highlights from each of the survey questions. If you prefer to dig into the data yourself, there are also some exported PDFs below provided by SurveyMonkey. Most of the inferences can be made using these PDFs, but there are some cases where additional filters are needed to deduce certain percentages.

We use the notation “SnQm” to mean “survey n question m”.

S1Q1: number of Wikipedia pages read per week

Here is a table that summarizes the data for Q1:

How many distinct Wikipedia pages do you read per week on average? SM = SurveyMonkey audience, V = Vipul Naik’s timeline, SSC = Slate Star Codex audience, AM = Wikipedia Analytics mailing list.
Response SM V SSC AM
less than 1 42% 1% 1% 0%
1 to 10 45% 40% 37% 29%
11 to 25 13% 43% 36% 14%
26 or more 0% 16% 27% 57%

Here are some highlights from the first question that aren’t apparent from the table:

  • Of the people who read fewer than 1 distinct Wikipedia page per week (26 people), 68% were female even though females were only 48% of the respondents. (Note that gender data is only available for the SurveyMonkey audience.)

  • Filtering for high household income ($150k or more; 11 people) in the SurveyMonkey audience, only 2 read fewer than 1 page per week, although most (7) of the responses still fall in the “1 to 10” category.

The comments indicated that this question was flawed in several ways: we didn’t specify which language Wikipedias count nor what it meant to “read” an article (the whole page, a section, or just a sentence?). One comment questioned the “low” ceiling of 26; in fact, I had initially made the cutoffs 1, 10, 100, 500, and 1000, but Vipul suggested the final cutoffs because he argued they would make it easier for people to answer (without having to look it up in their browser history). It turned out this modification was reasonable because the “26 or more” group was a minority.

S1Q2: affinity for Wikipedia in search results

We asked Q2, “On a search engine (e.g. Google) results page, do you explicitly seek Wikipedia pages, or do you passively click on Wikipedia pages only if they show up at the top of the results?”, to see to what extent people preferred Wikipedia in search results. The main implication to this for people who do content creation on Wikipedia is that if people do explicitly seek Wikipedia pages (for whatever reason), it makes sense to give them more of what they want. On the other hand, if people don’t prefer Wikipedia, it makes sense to update in favor of diversifying one’s content creation efforts while still keeping in mind that raw pageviews indicate that content will be read more if placed on Wikipedia (see for instance Brian Tomasik’s experience, which is similar to my own, or gwern’s page comparing Wikipedia with other wikis).

The following table summarizes our results.

On a search engine (e.g. Google) results page, do you explicitly seek Wikipedia pages, or do you passively click on Wikipedia pages only if they show up at the top of the results? SM = SurveyMonkey audience, V = Vipul Naik’s timeline, SSC = Slate Star Codex audience, AM = Wikipedia Analytics mailing list, H = heavy users (26 or more articles per week) of Wikipedia.
Response SM V SSC AM H
Explicitly seek Wikipedia 19% 60% 63% 57% 79%
Slight preference for Wikipedia 29% 39% 34% 43% 20%
Just click on top results 52% 1% 3% 0% 1%

One error on my part was that I didn’t include an option for people who avoided Wikipedia or did something else. This became apparent from the comments. For this reason, the “Just click on top results” options might be inflated. In addition, some comments indicated a mixed strategy of preferring Wikipedia for general overviews while avoiding it for specific inquiries, so allowing multiple selections might have been better for this question.

S1Q3: section vs whole page

This question is relevant for Vipul and me because the work Vipul funds is mainly whole-page creation. If people are mostly reading the introduction or a particular section like the “Criticisms” or “Reception” section (see S1Q5), then that forces us to consider spending more time on those sections, or to strengthen those sections on weak existing pages.

Responses to this question were fairly consistent across different audiences, as can be see in the following table.

Do you usually read a particular section of a page or the whole article? SM = SurveyMonkey audience, V = Vipul Naik’s timeline, SSC = Slate Star Codex audience, AM = Wikipedia Analytics mailing list.
Response SM V SSC AM
Section 73% 80% 74% 86%
Whole 34% 23% 33% 29%

Note that people were allowed to select more than one option for this question. The comments indicate that several people do a combination, where they read the introductory portion of an article, then narrow down to the section of their interest.

S1Q4: search functionality on Wikipedia and surprise at lack of Wikipedia pages

We asked about whether people use the search functionality on Wikipedia because we wanted to know more about people’s article discovery methods. The data is summarized in the following table.

How often do you use the search functionality on Wikipedia? SM = SurveyMonkey audience, V = Vipul Naik’s timeline, SSC = Slate Star Codex audience, AM = Wikipedia Analytics mailing list, H = heavy users (26 or more articles per week) of Wikipedia.
Response SM V SSC AM H
Several times per week 8% 14% 32% 57% 55%
About once per week 19% 17% 21% 14% 15%
About once per month 15% 13% 14% 0% 3%
About once per several months 13% 12% 9% 14% 5%
Never/almost never 45% 43% 24% 14% 23%

Many people noted here that rather than using Wikipedia’s search functionality, they use Google with “wiki” attached to their query, DuckDuckGo’s “!w” expression, or some browser configuration to allow a quick search on Wikipedia.

To be more thorough about discovering people’s content discovery methods, we should have asked about other methods as well. We did ask about the “See also” section in S1Q5.

Next, we asked how often people are surprised that there is no Wikipedia page on a topic to gauge to what extent people notice a “gap” between how Wikipedia exists today and how it could exist. We were curious about what articles people specifically found missing, so we followed up with S2Q4.

How often are you surprised that there is no Wikipedia page on a topic? SM = SurveyMonkey audience, V = Vipul Naik’s timeline, SSC = Slate Star Codex audience, AM = Wikipedia Analytics mailing list, H = heavy users (26 or more articles per week) of Wikipedia.
Response SM V SSC AM H
Several times per week 2% 0% 2% 29% 6%
About once per week 8% 22% 18% 14% 34%
About once per month 18% 36% 34% 29% 31%
About once per several months 21% 22% 27% 0% 19%
Never/almost never 52% 20% 19% 29% 10%

Two comments on this question (out of 59) – both from the SSC group – specifically bemoaned deletionism, with one comment calling deletionism “a cancer killing Wikipedia”.

S1Q5: behavior on pages

This question was intended to gauge how often people perform an action for a specific page; as such, the frequencies are expressed in page-relative terms.

The following table presents the scores for each response, which are weighted by the number of responses. The scores range from 1 (for every page) to 5 (never); in other words, the lower the number, the more frequently one does the thing.

For what fraction of pages you read do you do the following? Note that the responses have been shortened here; see the “Survey questions” section for the wording used in the survey. Responses are sorted by the values in the SSC column. SM = SurveyMonkey audience, V = Vipul Naik’s timeline, SSC = Slate Star Codex audience, AM = Wikipedia Analytics mailing list, H = heavy users (26 or more articles per week) of Wikipedia.
Response SM V SSC AM H
Check ≥1 citation 3.57 2.80 2.91 2.67 2.69
Look at “See also” 3.65 2.93 2.92 2.67 2.76
Read mostly for “Criticisms” or “Reception” 4.35 3.12 3.34 3.83 3.14
Click through ≥1 source to verify information 3.80 3.07 3.47 3.17 3.36
Share the page 4.11 3.72 3.86 3.67 3.79
Look at the talk page 4.31 4.28 4.03 3.00 3.86
Look at the editing history 4.35 4.32 4.12 3.33 3.92
Edit a page for grammatical/typographical errors 4.50 4.41 4.22 3.67 4.02
Edit a page to add new information 4.61 4.55 4.49 3.83 4.34
Look at editing history to verify author 4.50 4.65 4.48 3.67 4.73
Check how many pageviews a page is getting 4.63 4.88 4.96 3.17 4.92

The table above provides a good ranking of how often people perform these actions on pages, but not the distribution information (which would require three dimensions to present fully). In general, the more common actions (scores of 2.5–4) had responses that clustered among “For some pages”, “For very few pages”, and “Never”, while the less common actions (scores above 4) had responses that clustered mainly in “Never”.

One comment (out of 43) – from the SSC group, but a different individual from the two in S1Q4 – bemoaned deletionism.

S2Q1: number of Wikipedia pages read per week

Note the wording changes on this question for the second survey: “less” was changed to “fewer”, the clarification “at least one sentence of” was added, and we explicitly allowed any language. We have also presented the survey 1 results for the SurveyMonkey audience in the corresponding rows, but note that because of the change in wording, the correspondence isn’t exact.

How many distinct Wikipedia pages do you read (at least one sentence of) per week on average? SM = SurveyMonkey audience with no demographic filters, CEYP = College-educated young people of SurveyMonkey, S1SM = SurveyMonkey audience with no demographic filters from the first survey.
Response SM CEYP S1SM
Fewer than 1 37% 32% 42%
1 to 10 48% 64% 45%
11 to 25 7% 2% 13%
26 or more 7% 2% 0%

Comparing SM with S1SM, we see that probably because of the wording, the percentages have drifted in the direction of more pages read. It might be surprising that the young educated audience seems to have a smaller fraction of heavy users than the general population. However note that each group only had ~50 responses, and that we have no education information for the SM group.

S2Q2: multiple-choice of articles read

Our intention with this question was to see if people’s stated or recalled article frequencies matched the actual, revealed popularity of the articles. Therefore we present the pageview data along with the percentage of people who said they had read an article.

Which of these articles have you read (at least one sentence of) on Wikipedia (select all that apply)? SM = SurveyMonkey audience with no demographic filters, CEYP = College-educated young people of SurveyMonkey. Columns “2016” and “2015” are desktop pageviews in millions. Note that the 2016 pageviews only include pageviews through the end of June. The rows are sorted by the values in the CEYP column followed by those in the SM column.
Response SM CEYP 2016 2015
None 37% 40%
World War II 17% 22% 2.6 6.5
Barack Obama 17% 20% 3.0 7.7
United States 17% 18% 4.3 9.6
Donald Trump 15% 18% 14.0 6.6
Taylor Swift 9% 18% 1.7 5.3
Bernie Sanders 17% 16% 4.3 3.8
Japan 11% 16% 1.6 3.7
Adele 6% 16% 2.0 4.0
Hillary Clinton 19% 14% 2.8 1.5
China 13% 14% 1.9 5.2
The Beatles 11% 14% 1.4 3.0
Katy Perry 9% 12% 0.8 2.4
Google 15% 10% 3.0 9.0
India 13% 10% 2.4 6.4
Justin Bieber 4% 8% 1.6 3.0
Justin Trudeau 9% 6% 1.1 3.0

Below are four plots of the data. Note that r_s denotes Spearman’s rank correlation coefficient. Spearman’s rank correlation coefficient is used instead of Pearson’s r because the former is less affected by outliers. Note also that the percentage of respondents who viewed a page counts each respondent once, whereas the number of pageviews does not have this restriction (i.e. duplicate pageviews count), so we wouldn’t expect the relationship to be entirely linear even if the survey audiences were perfectly representative of the general population.

SM vs 2016 pageviews

SM vs 2016 pageviews

SM vs 2015 pageviews

SM vs 2015 pageviews

CEYP vs 2016 pageviews

CEYP vs 2016 pageviews

CEYP vs 2015 pageviews

CEYP vs 2015 pageviews

S2Q3: free response of articles read

The most common response was along the lines of “None”, “I don’t know”, “I don’t remember”, or similar. Among the more useful responses were:

S2Q4: free response of surprise at lack of Wikipedia pages

As with the previous question, the most common response was along the lines of “None”, “I don’t know”, “I don’t remember”, “Doesn’t happen”, or similar.

The most useful responses were classes of things: “particular words”, “French plays/books”, “Random people”, “obscure people”, “Specific list pages of movie genres”, “Foreign actors”, “various insect pages”, and so forth.

S3Q1 (Google Consumer Surveys)

The survey was circulated to a target size of 500 in the United States (no demographic filters), and received 501 responses.

Since there was only one question, but we obtained data filtered by demographics in many different ways, we present this table with the columns denoting responses and the rows denoting the audience segments. We also include the S1Q1SM, S2Q1SM, and S2Q1CEYP responses for easy comparison. Note that S1Q1SM did not include the “at least one sentence of” caveat. We believe that adding this caveat would push people’s estimates upward.

If you view the Google Consumer Surveys results online you will also see the 95% confidence intervals for each of the segments. Note that percentages in a row may not add up to 100% due to rounding or due to people entering “Other” responses. For the entire GCS audience, every pair of options had a statistically significant difference, but for some subsegments, this was not true.

Audience segment Fewer than 1 1 to 10 11 to 25 26 or more
S1Q1SM (N = 62) 42% 45% 13% 0%
S2Q1SM (N = 54) 37% 48% 7% 7%
S2Q1CEYP (N = 50) 32% 64% 2% 2%
GCS all (N = 501) 47% 35% 12% 6%
GCS male (N = 205) 41% 38% 16% 5%
GCS female (N = 208) 52% 34% 10% 5%
GCS 18–24 (N = 54) 33% 46% 13% 7%
GCS 25–34 (N = 71) 41% 37% 16% 7%
GCS 35–44 (N = 69) 51% 35% 10% 4%
GCS 45–54 (N = 77) 46% 40% 12% 3%
GCS 55–64 (N = 69) 57% 32% 7% 4%
GCS 65+ (N = 50) 52% 24% 18% 4%
GCS Urban (N = 176) 44% 35% 14% 7%
GCS Suburban (N = 224) 50% 34% 10% 6%
GCS Rural (N = 86) 44% 35% 14% 6%
GCS $0–24K (N = 49) 41% 37% 16% 6%
GCS $25–49K (N = 253) 53% 30% 10% 6%
GCS $50–74K (N = 132) 42% 39% 13% 6%
GCS $75–99K (N = 37) 43% 35% 11% 11%
GCS $100–149K (N = 11) 9% 64% 18% 9%
GCS $150K+ (N = 4) 25% 75% 0% 0%

We can see that the overall GCS data vindicates the broad conclusions we drew from SurveyMonkey data. Moreover, most GCS segments with a sufficiently large number of responses (50 or more) display a similar trend as the overall data. One exception is that younger audiences seem to be slightly less likely to use Wikipedia very little (i.e. fall in the “Fewer than 1” category), and older audiences seem slightly more likely to use Wikipedia very little.

SurveyMonkey allows exporting of response summaries. Here are the exports for each of the audiences.

The Google Consumer Surveys survey results are available online at https://www.google.com/insights/consumersurveys/view?survey=o3iworx2rcfixmn2x5shtlppci&question=1&filter=&rw=1.

Survey-making lessons

Not having any experience designing surveys, and wanting some rough results quickly, I decided not to look into survey-making best practices beyond the feedback from Vipul. As the first survey progressed, it became clear that there were several deficiencies in that survey:

  • Question 1 did not specify what counts as reading a page.
  • We did not specify which language Wikipedias we were considering (multiple people noted how they read other language Wikipedias other than the English Wikipedia).
  • Question 2 did not include an option for people who avoid Wikipedia or do something else entirely.
  • We did not include an option to allow people to release their survey results.

Further questions

The two surveys we’ve done so far provide some insight into how people use Wikipedia, but we are still far from understanding the value of Wikipedia pageviews. Some remaining questions:

  • Could it be possible that even on non-obscure topics, most of the views are by “elites” (i.e. those with outsized impact on the world)? This could mean pageviews are more valuable than previously thought.
  • On S2Q1, why did our data show that CEYP was less engaged with Wikipedia than SM? Is this a limitation of the small number of responses or of SurveyMonkey’s audiences?

Further reading

Acknowledgements

Thanks to Vipul Naik for collaboration on this project and feedback while writing this document, and for supplying the summary section, and thanks to Ethan Bashkansky for reviewing the document. All imperfections are my own.

The writing of this document was sponsored by Vipul Naik. Vipul Naik also paid SurveyMonkey (for the cost of SurveyMonkey Audience) and Google Consumer Surveys.

Document source and versions

The source files used to compile this document are available in a GitHub Gist. The Git repository of the Gist contains all versions of this document since its first publication.

This document is available in the following formats:

License

This document is released to the public domain.

Suggestion: jog instead of walk

3 [deleted] 13 July 2016 05:34PM

A bit of background: my manager literally runs around the office when she needs something so she can return to her desk asap. While watching her one day I thought about trying my own experiment of jogging from place to place.

I chose jogging instead of running as I figured the expenditure of energy and anaerobic respiration required for a full on run/sprint would take too much energy for the seconds saved and distance traveled, while jogging would feel a more natural transition of energy and still save some time.

So far, the only 'drawback' I've seen is the occasional confused look from a passerby (truthfully though, most people don't seem to care). Comparing this to the increase of overall energy and positive emotion I've felt, it's been well worth it. There also might be times and places where jogging would be inappropriate, so use discretion of course.

Also, you don't have to start out with jogging right away! I run daily anyway so that's why I transitioned right away. You could start with a brisker walk and work your way up.

If you do try it out, please feel free to let me know how it works out for you! :)

tDCS, Neuroscientists' Open Letter To DIY Brain Hackers

6 morganism 12 July 2016 07:37PM

"The evidence of harm would be the evidence that you can hurt some cognitive functions with the same stimulation protocols that help another cognitive function. But they're completely correct that we don't have any evidence saying you're definitely hurting yourself. We do have evidence that you're definitely changing your brain."

interview:

http://www.wbur.org/commonhealth/2016/07/11/caution-brain-hacking

 

Paper:

http://onlinelibrary.wiley.com/doi/10.1002/ana.24689/references

 

I was aware of the variability of responses to stim, but not the paper that leveraging one brain function could impair another. This was also written to give the docs some info to help inform their patients.

edit

I'll also tuck this in here, as i posted it to open thread.

Texting changes brain waves to new, previously unknown, pattern.

http://sciencebulletin.org/archives/2623.html

Makes me wonder if they were using spell check, or the new, shortend speak. By using constructed kernels, or images of words and concepts, it looks like machine learning retrieval or construction is already  being practiced here ?

Open thread, Jul. 11 - Jul. 17, 2016

4 MrMind 11 July 2016 07:09AM

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.


Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

A Very Concrete Model of Learning From Regrets

1 SquirrelInHell 09 July 2016 11:30AM

Warning 1: This post is written in the form of Java-like pseudocode.

If you have no knowledge of programming, you might have trouble understanding it.

(If you do, it still does not guarantee you will understand, but your chances are better.)

 

Warning 2: I have more than moderate, but less than high, confidence that this model is approximately correct.

It doesn't mean that my or anyone's brain works exactly in the way shown in the code, but rather that the flow of data in the brain is approximately as if it were using such an algorithm.

The word "approximately" includes stuff I don't (yet) know about, but also stuff I didn't include below to keep it simple.

I wrote this specifically for regrets, but processing of positive memories seems to have similar mechanics (with different constants).

 

Warning 3: There is little chance of finding any existing studies/data etc. that could directly validate or invalidate this model. (However if you know of any, I'm all ears.)

There might some stuff that is correlated, so if you know something mention it too.

 

class Brain
{
    ...
    
    // This represents a memory about a single event
    
    class Memory
    {
        ...
        
        float associatedEmotions; // positive or negative
    }
    
    // Your brain keeps track of this
    
    private Map<Memory, Float> memoriesRequireProcessing = new Map<>();
    
    // Add new stuff to the queue
    
    private void somethingHappened(Memory newMemory)
    {
        float affect = getAffectOfSituation(newMemory);
        
        newMemory.associatedEmotions = affect * 0.5;
    
        if (Math.abs(affect) > 0.1)
            memoriesRequireProcessing.add(newMemory, Math.abs(affect));
    }          // You have no control over how this works,     // but you can influence the confidence parameter     // (mostly indirectly, a little bit directly)          protected void learnedMyLesson(Memory m, float confidence)     {         float previousValue =             memoriesRequireProcessing.get(m);                  float nextValue = previousValue * (1.0 - confidence);                  if (nextValue > 0.1)             memoriesRequireProcessing.set(m, nextValue);         else             memoriesRequireProcessing.remove(m);     }          // You can consciously override this and do something else     //     // @return: judgement of success or failure          protected float ruminateOnMemory(Memory m)     {         // Depends on the situation, but the default is         // relatively low confidence                  learnedMyLesson(m, 0.1);                  // Substitute affect for judgement of success                  return getAffectOfSituation(m);     }          // This prompts some thoughts about a memory          private void rememberAbout(Memory m)     {         feelEmotion(m.associatedEmotions);              float judgement = ruminateOnMemory(m);                  m.associatedEmotions =             0.9 * m.associatedEmotions             + 0.2 * judgement;     }          // Your brain does this all the time          private void onIdle()     {         while (memoriesRequireProcessing.thereIsALotOfShit())         {             // Choose some memory paired with a high value                          Memory next = memoriesRequireProcessing.choose();                          rememberAbout(next);         }                  ...     }          ... }

New LW Meetup: Boise ID, Bay City MI

2 FrankAdamek 08 July 2016 03:25PM

This summary was posted to LW Main on July 8th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, New Hampshire, New York, Philadelphia, Research Triangle NC, San Francisco Bay Area, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

[LINK] Concrete problems in AI safety

15 Stuart_Armstrong 05 July 2016 09:33PM

From the Google Research blog:

We believe that AI technologies are likely to be overwhelmingly useful and beneficial for humanity. But part of being a responsible steward of any new technology is thinking through potential challenges and how best to address any associated risks. So today we’re publishing a technical paper, Concrete Problems in AI Safety, a collaboration among scientists at Google, OpenAI, Stanford and Berkeley.

While possible AI safety risks have received a lot of public attention, most previous discussion has been very hypothetical and speculative. We believe it’s essential to ground concerns in real machine learning research, and to start developing practical approaches for engineering AI systems that operate safely and reliably.

We’ve outlined five problems we think will be very important as we apply AI in more general circumstances. These are all forward thinking, long-term research questions -- minor issues today, but important to address for future systems:

  • Avoiding Negative Side Effects: How can we ensure that an AI system will not disturb its environment in negative ways while pursuing its goals, e.g. a cleaning robot knocking over a vase because it can clean faster by doing so?
  • Avoiding Reward Hacking: How can we avoid gaming of the reward function? For example, we don’t want this cleaning robot simply covering over messes with materials it can’t see through.
  • Scalable Oversight: How can we efficiently ensure that a given AI system respects aspects of the objective that are too expensive to be frequently evaluated during training? For example, if an AI system gets human feedback as it performs a task, it needs to use that feedback efficiently because asking too often would be annoying.
  • Safe Exploration: How do we ensure that an AI system doesn’t make exploratory moves with very negative repercussions? For example, maybe a cleaning robot should experiment with mopping strategies, but clearly it shouldn’t try putting a wet mop in an electrical outlet.
  • Robustness to Distributional Shift: How do we ensure that an AI system recognizes, and behaves robustly, when it’s in an environment very different from its training environment? For example, heuristics learned for a factory workfloor may not be safe enough for an office.

We go into more technical detail in the paper. The machine learning research community has already thought quite a bit about most of these problems and many related issues, but we think there’s a lot more work to be done.

We believe in rigorous, open, cross-institution work on how to build machine learning systems that work as intended. We’re eager to continue our collaborations with other research groups to make positive progress on AI.

Notes on Imagination and Suffering

5 SquirrelInHell 05 July 2016 02:28PM

Time: 22:56:47

I

This is going to be an exercise in speed writing a LW post.

Not writing posts at all seems to be worse than writing poorly edited posts.

It is currently hard for me to do anything that even resembles actual speed writing: even as I type this sentence, I have a very hard to resist urge to check it for grammar mistakes and make small corrections/improvements before I've even finished typing.

But to reduce the burden of writing, I predict it is going to be highly useful to develop the ability of actually writing a post as fast as I can type, without going back.

If this proves to have acceptable results, you can expect more regular posts from me in the future.

And possibly, if I develop the habit of writing regularly, I'll finally get to describing some of the topics on which I have (what I believe are) original and sizable clusters of knowledge, which is not easily available somewhere else.

But for now, just some thoughts on a very particular aspect of modelling how human brains think about a very particular thing.

This thing is immense suffering.

Time: 23:03:18

(Still slow!)

II

You might have heard this or similar from someone, possibly more than once in your life:

"you have no idea how I feel!"

or

"you can't even imagine how I feel!"

For me, this kind of phrase has always had the ring of a challenge. I have a potent imagination, and non-negligible experience in the affairs of humans. Therefore, I am certainly able to imagine how you feel, am I not?

Not so fast.

(Note added later: as Gram_Stone mentions, these kinds of statements tend to be used in epistemically unsound arguments, and as such can be presumed to be suspicious; however here, I am more concerned with the fact of the matter of how imagination works.)

Let's back up a little bit and recount some simple observations about imagining numbers.

You might be able to imagine and hold the image of five, six, nine, or even sixteen apples in your mind.

If I tell you to imagine something more complex, like pointed arrows arranged in a circle, you might be able to imagine four, or six, or maybe even eight of them.

If your brain is constructed differently from mine, you might easily go higher with the numbers.

But at some fairly small number, your mental machinery simply no longer has the capacity to imagine more shapes.

III

However, if I tell you that "you can't even imagine 35 apples!" it is obviously not an insult or a challenge, and what is more:

"imagining 35 apples" is NOT EQUAL to "comprehending in every detail what 35 apples are"

I.e. depending on how good your knowledge of natural numbers is, that is to say, if you passed the first class of primary school, you can analyse the situation of "35 apples" in every possible way, and imagine it partially - but not all of it at the same time.

Directly imagining apples is very similar to actually experiencing apples in your life, but it has a severe limitation.

You can experience 35 apples in your life, but you can't imagine all of them at once even if you saw them 3 seconds ago.

Meta: I think I'm getting better at not stopping when I write.

Time: 23:13:00

IV

But, you ask, what is the point of writing all this obvious stuff about apples?

Well, if you move to more emotionally charged topics, like someone's emotions, it is much harder to think about the situation in a clear way.

And if you have a clear model of how your brain processes this information, you might be able to respond in a more effective way.

In particular, you might be saved from feeling guilty or inadequate about not being able to imagine someone's feelings or suffering.

It is a simple fact about your brain that it has a limited capability to imagine emotion.

And especially with suffering, the amount of suffering you are able to experience IS OF A COMPLETELY DIFFERENT ORDER OF MAGNITUDE than the amount you are able to imagine, even with the best intentions and knowledge.

However, can you comprehend it?

V

From this model, it is also immediately obvious that the same thing happens when you think about your own suffering in the past.

We know generally that humans can't remember their emotions very well, and their memories don't correlate very well with reported experience-in-the-moment.

Based on my personal experience, I'll tentatively make some bolder claims.

If you have suffered a tremendous amount, and then enough time has passed to "get over it", your brain is not only unable to imagine how much you have suffered in the past:

it is also unable to comprehend the amount of suffering.

Yes, even if it's your own suffering.

And what is more, I propose that the exact mechanism of "getting over something" is more or less EQUIVALENT to losing the ability to comprehend that suffering.

The same would (I expect) hold in case of getting better after severe PTSD etc.

VI

So in this sense, a person telling you "you cannot even imagine how I feel" is right also with a less literal interpretation of their statement.

If you are a mentally healthy individual, not suffering any major traumas etc., I suggest your brain literally has a defense mechanism (that protects your precious mental health) that makes it impossible for you to not only imagine, but also fully comprehend the amounts of suffering you are being told about.

Time: 23:28:04

Publish!

Open thread, Jul. 04 - Jul. 10, 2016

4 MrMind 04 July 2016 07:02AM

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.


Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

Link: Re-reading Kahneman's Thinking, Fast and Slow

11 toomanymetas 04 July 2016 06:32AM

"A bit over four years ago I wrote a glowing review of Daniel Kahneman’s Thinking, Fast and Slow. I described it as a “magnificent book” and “one of the best books I have read”. I praised the way Kahneman threaded his story around the System 1 / System 2 dichotomy, and the coherence provided  by prospect theory.

What a difference four years makes. I will still describe Thinking, Fast and Slow as an excellent book – possibly the best behavioural science book available. But during that time a combination of my learning path and additional research in the behavioural sciences has led me to see Thinking, Fast and Slow as a book with many flaws."

Continued here: https://jasoncollins.org/2016/06/29/re-reading-kahnemans-thinking-fast-and-slow/

The map of future models

5 turchin 03 July 2016 01:17PM

TL;DR: Many models of the future exist. Several are relevant. Hyperbolic model is strongest, but too strange.

Our need: correct model of the future
Different people: different models = no communication.

Assumptions:
Model of the future = main driving force of historical process + graphic of changes
Model of the future determines global risks

The map: lists all main future models.
Structure: from fast growth – to slow growth models.
Pfd: http://immortality-roadmap.com/futuremodelseng.pdf

 

 

 

 

Zombies Redacted

33 Eliezer_Yudkowsky 02 July 2016 08:16PM

I looked at my old post Zombies! Zombies? and it seemed to have some extraneous content.  This is a redacted and slightly rewritten version.

continue reading »

New LW Meetup: Dallas

1 FrankAdamek 01 July 2016 03:41PM

This summary was posted to LW Main on July 1st. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Irregularly scheduled Less Wrong meetups are taking place in:

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Locations with regularly scheduled meetups: Austin, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, New Hampshire, New York, Philadelphia, Research Triangle NC, San Francisco Bay Area, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

continue reading »

Rationality Quotes July 2016

1 bbleeker 01 July 2016 09:02AM

Another month, another rationality quotes thread. The rules are:

  • Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
  • Post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
  • Do not quote yourself.
  • Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
  • No more than 5 quotes per person per monthly thread, please.

July 2016 Media Thread

2 ArisKatsaris 01 July 2016 06:52AM

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

  • Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.
  • If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.
  • Please post only under one of the already created subthreads, and never directly under the parent media thread.
  • Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.
  • Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.

View more: Prev | Next