Context
1. This is the second in a series of internal LessWrong 2.0 team document we are sharing publicly (with minimal editing) in an effort to help keep the community up to date with what we're thinking about and working on.
I suggest you first read this other document for context.
2. Caveat! This is internal document and does not represent any team consensus or conclusions; it was written by me (Ruby) alone and expresses my own in-progress understanding and reasoning. To the extent that the models/arguments of the other team members are included here, they've been filtered through me and aren't necessarily captured with high fidelity or strong endorsement. Since it was written on March 18th, it isn't even up to date with my own thinking
.
.
Epistemic status: Since the 18th when I first wrote this, I have many new lists and a lot more information. Yet this one still serves as a great intro into all the questions to be asked about Q&A and what it can and should be.
Originally written March 18, 2019
Related: Q&A Review + Case for a Marketplace
- HIGH LEVEL
- Is it actually the case that Q&A for serious research is this big, new, different thing which requires a big shift for people? Maybe it's not such an adjustment?
- How willing are people to do serious research work for others on the internet?
- RESEARCH PROCESS (and suitability for collaboration) <tease these out by talking through their recent research>
- Can "significant research" be partitioned into discrete questions?
- Or is it more that there is a deeper bigger question around which someone needs to become an expert, and that any question posed in downstream of the real question and can't be treated in isolation?
- Perhaps talk to the Ought folk about this.
- Do people have general open research questions they want vaguely want answered and are willing to have sit unanswered for a relatively long period of time?
- Or do they mainly have (and prioritize) research questions which are currently part of their work?
- How much interaction between the research requester and research contributor is required?
- Can someone take a research question and execute successfully on their own without too much feedback from the person requesting the research?
- If necessary, does Q&A facilitate this adequately? Are back and forth comments good enough?
- Are busy research requesters willing to put in the time to interact with people trying to contribute, contributors who they don't have know and haven't necessarily vetted?
- What kind of research questions are amenable to the format of LessWrong's Q&A?
- PERCEPTIONS AND PRIOR BELIEFS <should get answered semi-automatically interviews>
- Is the mix of research and less research-y questions on Q&A now causing people to not think of Q&A as a place for serious research questions?
- What are people's current impressions, expectations, anticipations of LW's Q&A, segmented by level of exposure?
- e.g. if I tell someone LessWrong has a Q&A with the goal of serious research progress, what do they imagine? What's their reaction?
- Do people think that they could be helped by Q&A? Do they want to use it?
- INCENTIVES, WILLINGNESS, & EXPERIENCE <get at these questions by talking through how interviewees might or might not use Q&A>
- How much (and what kind) of incentives are needed for contributors to want to contribute?
- Are bounties of cash prizes enough?
- If yes, is it because the money makes the effort worth it, OR
- it just that cash prizes are a costly signal is important and once that's clear, people would be glad to help?
- Is bounty complexity an actual issue?
- Are people doing an EV calculation with bounties such even if a nominal bounty is $500, people don't necessarily think they're worth a lot of work? Their EV is like $50
- How good does the ROI need to be for question askers to want to use the platform?
- How low does the time and attention cost need to be for question askers to want to use the platform?
- How much effort are question answerers willing to invest already?
- It does look like that some StackOverflow questions are very involved. So some people are willing to take time to answer things.
- A few of the questions/answers on Q&A right now are pretty involved. Not many, but a few.
- ADOPTION, POPULATION (TAM)
- What is the population of adequately skilled and available question answerers within the domains we care about? Is it enough to support a good Q&A ecosystem?
- How many people believe they're qualified? <probably need more general polling>
- What's the distribution of people in the 2x2 grid of "thinks they're qualified" x"actually qualified"?
- What user base of contributors do we have to reach before the question asker experience is good enough to retain users?
- OTHER <expect to come up in talking through their use of Q&A
- Is privacy a major issue for potential question askers?
- How do they feel if there are closed groups?
- Is trust in research quality an issue for question askers?
- What does it take to evaluate whether a research contribution is good?
- How much can it be done just by reading the contribution or will it require redoing serious work?
- Are question askers willing to do this?
- Are third parties willing to do the evaluation?
For a long time, I was an intellectual, and it worked out quite well for me. I've done very well to have a clear, comfortable writing style, I've done it many times. It's one of my main areas of self improvement, and it also strikes me as an amazing, quick to engage with the subject matter.
In retrospect, I was already way, very lucky in that I could just read an argument and find the flaws in it, even when I didn't really know what to do.
Now, I've tried very hard to be good at expressing my ideas in writing, and I still don't know how to give myself more than some effort. I do have some small amount of motivation, but no guarantee that I'll be the person who posts about the topic, and I don't have nearly as much ability as I'd like. If I were to take my friends and try to explain it, I don't think I'd be able to.
And finally - when it's my own beliefs - I start generating conversation like this:
Me: Do you think you're the best in the world?
Her: Consider me and my daughter. Our society works quite badly for our children [who don't enjoy cooking, do any science]
Her: But what's your field at work?
Me: People say they're the best and best in the world, but that's just a personal preference and not my field. It's a scientific field.
Her: So why do you think that?
Me: It may be true that I can do any science, but it sounds a bit... wrong.
Me: And, if you were to read the whole thing, did you really start?
Her: You have to read the whole thing.
Me: Let me start with the one I have:
Me: How do you all think I'm going to be on?
Her: If I could use any help at all, I probably would.
Me: How do you all think I'm going to get into any work?
Her: What do you mean, 'better yet' ? Because I've never done anything out of interest myself ? Because I've never done any interest in anything to my children?
Me: I'm going to start writing up a paper on my own future.
Her: I don't know, I do.