I feel like I should provide some data as someone who participated in a number of past bounties.
I don't think it's possible on LW. It's not a matter of money (ok, it is, in that I don't think anyone's likely to offer a compelling bounty that I expect to be able to win). It's not a matter of reliability of available offers (except that I don't expect ANY).
It's _is_ a question of reliability and trust, though. There are no organizations or people I trust enough to define a task well and make sure multiple aren't competing in some non-transparent way, so that I actually expect to get paid for the work posted on a discussion site. And I don't expect that I have enough track record for any bidder to prefer me for the kind of tasks you're talking about at the rates I expect. [edit to add] Nor do I have any tasks where I'd prefer a bounty or open-bid rather than finding a partner/employee and agreeing on specific terms.
It's also a question of what LW is for - posting and discussion of thought-provoking, well-researched, interestingly-modeled, and/or fun ideas is something that's very hard to measure in order to reward monetarily. Also, I'll be massively demotivated by thinking of this as a commercial site, even if I'm only in the free area.
My recommendation would be to use a different place to manage the tasks and the bid/ask process, and the acceptance of work and payment. Some tasks and their outputs might be appropriate to link here, but not the job management.
tl;dr: don't mix money into LW. Social and intellectual rewards are working pretty well, and putting commerce into it could well kill it.
There's some important points here that I'm going to address by rewriting the OP significantly.
Part of the impetus of our current thought process is there does seem to be a limit on the complexity of stuff that typically gets answered without bounties attached (but, we now have a track record of occasional bounty posts successfully motivating such work).
I could imagine it turning out that the correct balance involves "not building any additional site features, just allow it to be something that happens organically sometimes", so that it can happen but there's friction that prevents runaway Moloch processes.
I currently think there's room to slightly increase the option of monetary incentives without destroying everything but it's definitely something I'd want to think carefully about.
My answer (not necessarily endorsed by rest of team) to your question is something like "right now, it seems like for the most part, LessWrong motivates stuff that is either Insight Porn, or 'Insight That Is At Least Reasonably Competitive with Insight Porn.'"
And we actually have the collective orientation and many of the skills needed to work on real, important problems collaboratively. Much of those problems won't have the intrinsic feedb...
If answering the question takes weeks or months of work, won't the question have fallen off the frontpage by the time the research is done?
What motivates me is making an impact and getting quality feedback on my thinking. These both scale with the number of readers. If no one will read my answer, I'm not feeling very motivated.
I'm currently exploring a possible feature wherein question-authors, and moderators, can flag answers as "Top Answers", which trigger the question moving to the top of the home page, and adding the most recent "top answer" author as a co-author of the post.
Not 100% sure on the implementation details. Does that sound like that would help with this problem?
Well, the question asker will always see it (they'll receive a notification). The act of answering it will also:
a) put it on the recent discussion section
b) it'll also appear in on the slightly revamped Questions page, where both "Top Questions" and "Recent Activity" are sorted by which questions were most recently commented on. ("Top Questions" are "questions with 40 or more karma, sorted by recently commented/answered").
We'll be putting some work into figuring out how to keep questions take up "...
For the examples you give, the improvements you cite to intrinsic motivation + karma would be sufficient to motivate me for questions of the "take an afternoon of your time" type, which is approximately where my blogposts have been landing anyway. Further, several are already of the summarize papers/point to a list of sources type. On the long end of weeks or months, bounties in the hundreds would probably satisfy depending on the level of interest I have, which is the true variable in whether I engage.
It is hard to tell in the current format what kind of depth-of-answer the questioner is looking for, and what time frame would be appropriate for an answer. It is also hard to tell how well answered a question already is, which has a big impact on reading older questions or questions with many answers. Mostly I have been viewing questions at the same rate as blog posts, but it occurs to me that they don't age in the same way informative or exploratory posts do; the question is unresolved until it is.
Having some way to disentangle the content of this site from when it was posted would be handy.
It's still worth mentioning, but what sort of guidelines should be included when using the site and making it a better place?
For a few months I started donating a bit more money to LW instead of to EA, as a motivation to donate for reasons that don't sound very interesting to me, but for reasons tha
...It would help if the poster directly approaches or tags me as a relevant expert.
Given that there is some probability of winning a question, let's just guess it's 20% on any particular question I might try to answer. This suggests to me a bounty of 5x whatever I would be willing to answer the question for in order to make me willing to do it. Assuming a question takes about a day of work (8 hours) to answer fully and successfully, and given our 5x multiplier, I'd be willing to try to answer a question I wasn't already excited to answer for other reasons if it paid about $1800.
Many others may have lower opportunity costs, though (and I undercounted a bit because I assume any question I would answer would deliver me at least some sense of value beyond the money; otherwise my number would probably jump up closer to $2500).
Yeah, there's two issues this points at that we've been thinking about:
1. "bounties" come with an issue where you're not sure you'll succeed, so if you're actually relying on it for "real money" (instead of using the money as an indicator that someone cared which might motivate you enough to do it for fun), you need much more money for it to work
2. I actually expect a "well functioning" Q&A system works by having lots of people tackle small parts of a problem, in ways that are harder to assign cred...
If people provided this as a service, they might be risk-averse (it might make sense for people to be risk-averse with their runway), which means you'd have to pay more than hourly rate/chance of winning.
This might not be a problem, as long as the market does the cool thing markets do: allowing you to find someone with a lower opportunity cost than you for doing something.
In order to be motivated, I would like to have a good idea of the impact the work would be making. I would like to see a clear explanation of the process taken to come up with the question and a list of who in LW supports this question as being an effective target of attention at this point in time and why. Maybe this could be documented in the question post and maybe there could be rounds that potential questions to go through for community members to vote/discuss/rate them. Maybe there could be a backlog of other questions that have not been chosen yet with reasons why they have not been chosen yet to help new questions arise. I would also like to know which other LW users are working on it (to avoid duplication of efforts) and if there are good opportunities for delegating work among multiple community members.
I like the idea of sub-questions. It might be interesting to have a display in the form of a graph with vertices as question/answers and directed edges as indicating a sub-super relationship between questions/answers. I think this would help us get a big picture view of the progress made and how it was achieved.
Since there is only so much that can be done by one community, I think it could in some cases be useful to have questions that are intended to be handed off to external parties like academic groups or certain organizations or renowned individuals after we do enough investigatory work.
If this blog's "hard questions" have utility, they should be novel, important, and answerable.
Important questions are highly likely to be known already among experts in the relevant field. If they're answerable, one of those experts is likely already working on it with more rigor than you're capable of extracting from a crowd of anonymous bloggers. I think, then, that any questions you ask have a high probability of being redundant, unimportant, or unanswerable (at least to a useful degree of rigor). Unfortunately, you're unlikely to know that in advance unless you vet the questions with experts in the relevant literature.
And at that point, you're starting to look like an unaccountable, opaque, disorganized, and underresourced anonymously peer-reviewed journal.
It might be interesting to explore the possibility that a wiki-written or amateur-sourced peer reviewed journal could have some utility, especially if it focused on a topic that is not so dependent on the expensive and often opaque process of gathering empirical data. I expect that anyone who can advance the field of mathematics is probably already a PhD mathematician. So philosophy, decision theory, something like that?
Developing a process to help an anonymous crowd of blog enthusiasts turn their labor into a respectable product would be useful and motivating. I would start by making your next "hard question" what specific topic such a PRJ could usefully focus on.
Your premises seem strange to me – questions are either important and already worked on, or not important? Already-worked-on-questions don't need answers? Both of these seem false.
If an expert somewhere knows the answer to something, I still often need to know the answer myself (because it's a piece of a broader puzzle that I care about, which the expert doesn't necessarily care about). I still need someone to go find the answer, distill it, and to help put it into a new context.
The LW community historically has tackled questions that were important, and that few other people were working on (in particular related to human rationality, AI alignment and effective altruism)
I would like more concrete examples of nontrivial questions people might be interested in. Too much of this conversation is too abstract, and I worry people are imagining different things.
Toward that end, here are a few research projects I've either taken on or considered, which I would have been happy to outsource and which seem like a good fit for the format:
Thanks, and yeah these are approximately the same order-of-magnitude-of-difficulty that was I imagining as a "hard" question (some seem to require more specialized knowledge though, which I'm not sure of the viability of)
Some additional examples (answering Chris_Leong's question here I guess), are:
Ruby has now written a post that explains some more of the background thinking that underlay this post. It's not super polished but if you read this question and felt "...this seems to be missing context?", here is some of it.
This whole discussion is pretty abstract. I'd be quite interested to know what kind of content you are trying to encourage. Is it just high quality articles in general or on specific topics?
Another idea for improving the quality of content, although not necessarily directly related to questions, would be to have a Less Wrong "Magazine". For this proposal, a certain number of articles would be accepted in every issue. The competitive nature and the existence of an editor would lead to higher quality content production. This would be rewarded by the submitted content presumably receiving more attention and prestige as people would know that more effort went into producing these articles.
I think the question, narrowly interpreted as "what would cause me to spend more time on the object-level answering questions on LW" doesn't capture most of the exciting things that happen when you build an economy around something. In particular, that suddenly makes various auxiliary work valuable. Examples:
I have an upcoming post where I describe in more detail what I think is required to make this work.
This all seems plausible, but for it to work it seemed like it needed to cross the initial threshold of "are people actually willing to do this, and what will it take for them to do so?". For this question I'm most interested in getting a sense of what things are necessary for this sort of project to get off the ground.
I think the answer will be highly dependent on the question. The opportunity cost of someone answering a q-n on yoga (being a yoga expert) is very different from someone answering an investment question.
Sure. But (for you, in particular), what are some examples of types of questions and the amount you'd need, at different degrees of difficulty?
(Might also be a good time to share what fields of expertise you have, to get a sense of what domains LW might be particularly good at thinking seriously about)
Sure. I know something about general CS stuff, ML, applied Bayesian stats and finance. Generally I would not be answering questions for a bounty (I'm well compensated to do this at work and I don't want *more work*) but would spend some time if I think it helped people of contributed to an important body of knowledge. For me it comes from a different "time budget". I realize many people would feel differently but there's probably a class of people like me.
New features such as the ability to spawn related questions that break down a confusing question into an easier.
Why would that be any better than just mentioning related questions in a comment, or compiling links to the subquestions in your answer?
Two clusters of reasons:
1. Nudges/incentives.
We didn't technically need a questions feature in the first place (there's nothing stopping you from writing questions as posts), but having an explicit feature sends a strong signal that this is an encouraged norm.
We also didn't technically need to implement "answers" as a type separate from comments, but we did so to help ensure that people would approach questions with a different mindset than typical posts. (i.e. actually try to figure a thing out, rather than just sort of meander around on the internet).
I actually considered not only adding Related Questions, but making them the default way of interacting with question posts (rather than submitting an answer), based on the notion that people seemed to be rushing to answer hard questions when the correct next step was to break it down further. (We ultimately decided not to do this)
2. Enabling other features and architecture
Another reason we created answers (rather than just using comments), was that an Answer type makes it a bit more sensible to do things like "mark a question as Answered" (so that future people who search for the question will find a convenient Question/Answer pair). We haven't actually built that feature yet but still plan to.
Similarly, we're interested in a Related Questions because they suggest ways of more easily rearranging question pages and question sequences. For an Open Question, you can look at a high level overview of what subquestions have been answered and which haven't, which suggests a different way of engaging with the overall topic.
I think the general setup should be "all posts" at least, since it's so straightforward to look at a post separately from a list of each concept.
I think I have a specific concept for something I'm trying to say, but I didn't know how to describe that concept.
I think I've solved the problem of having to explicitly list several things to say in order to get to the kind of answer which results in more answers. If I just don't feel like doing this by the time you get to the next one, I guess it'd be useful to have my own concept.
That's what actually makes the post a better concept.
The main problem is with getting your concept to work together that's not how things really work together. I don't think it really helps to have that concept to work together on an underlying, just a) you can't, a) I don't think you could work together on a new concept, and b) even if it doesn't, you have to have the concept in your area of expertise to build it into a new concept.
So I'm hoping that it doesn't sound too insane to list a concept and then tell me how to do it, without which the concepts are useless.
Some thoughts on the latter thought, which I do in a few places:
Overall, it seems like this has been more generally useful, and I'm now aware that having several threads of thought seems easier and more natural to many people in some contexts, but has this explanation as the thing to remember? I don't think it is, though. I also hope for the rest of you to find it
Edit: Significantly rewritten. Original question was more specifically oriented around money-as-a-motivator.
One of the questions (ha) that we are asking ourselves on the LW team is "can the questions feature be bootstrapped into a scalable way of making intellectual progress on things that matter."
Motivations
Intrinsic vs Extrinsic
I'd cluster most knobs-to-turn here into "intrinsic motivation" and "extrinsic motivation."
Intrinsic motivation covers things like "the question is interesting, and specified in a way that is achievable, and fun to answer."
Extrinsic motivation can include things like "karma rewards, financial rewards, and other things that explicitly yield higher status for
(Things like "I feel a vague warm glow because I answered the question of someone I respect and they liked the answer" can blur the line between intrinsic and extrinsic motivation)
Improving Intrinsic Motivation
Right now I think there's room to improve the flow of answering questions:
Bounties and Reliability
A lot of questions are just hard to answer – realistically, you need a lot of time, at least some of the time won't be intrinsically fun, and the warm glow of success won't add up to "a few days to a few months worth of work."
So we're thinking of adding some more official support for bounties. There have been some pretty successful bounty-driven content on LW (such as the AI Alignment Prize, the Weird Aliens Question, and Understanding Information Cascades), which have motivated more attention on questions.
Costly signaling of value
They showcase that the author of the question cares about the answer. Even if the money is still relatively minor, it reaffirms that if you work on the question, someone will actually derive value from it, which can be an actual important part of intrinsic motivation (as well as a somewhat legible-but-artificial status game you to more easily play, which I'd classify as extrinsic)
Serious times requires livable-money
In some cases you just actually need to put serious time into solving it to succeed, which means you either need to have already arranged your life such that you can spend serious time answering questions on LW, or you need "answering hard questions on LW" to actually provide you with enough financial support to do so.
This requires not just "enough" money, but enough reliability of money that "quit your day job" (or get a day job that pays-less-but-gives-more-flexiblity) is a an actual option.
What would it take?
So, with all that in mind...
What would it take for you (you, personally), to start treating "answer serious LW questions" as a thing you do semi-regularly, and/or put serious time into?
My assumptions (possibly incorrect) here are that you need a few things (in some combination)
Some types of intellectual labor I'm imagining here (which may or may not all fit neatly into the "questions" framework).
"Serious" questions could range from "take an afternoon of your time" to "take weeks or months of research", and I'm curious what the actual going rate for those two ends of the spectrum are, for LW readers who are a plausible fit for this type of distributed work.