I think this is an intriguing idea. It reminds me of the discussion of vague language in Superforecasters: the intelligence community put a lot of effort into optimizing language in its reports, such as "possibly", "likely", "almost certainly", etc. only to later realize that they didn't know what those words meant (in terms of probabilities) even after discussing word choice quite a bit. Someone went around asking analysts what was meant by the words and got very different probabilities from different people. Similarly, being careful about describing epistemic status is likely better than not doing so, but the words may not have as clear a meaning as you think; describing what you actually did seems like a good way to keep yourself honest.
This is sort of stream of conscious-y because I didn't want to force myself to do so much that I ended up going 'ugh I don't have time for this right now I'll do it later.'
This seems like an important failure mode. People may not be so interested in writing if they also have to indicate their amount of effort. :p
Another problem I see is: "epistemic effort" may not play as well with signalling games as "epistemic status". Putting your specific efforts out there rather than a degree of confidence can make something look scatter-brained that is actually well-conceived. For example, "thought about it for 5 minutes" on your post doesn't indicate the degree of support the idea has from background knowledge and experience. Your actual post indicates that. But, the real reasons you think something will work will often be hard to summarize in a small blurb and will instead go into the content of the post itself.
I think what I'll do is keep using the "epistemic status" tag, starting with a vague status such as "confident" or "speculative", and then providing more detail with the notion of "epistemic effort" in mind.
"Epistemic status" metadata plays two roles: first, it can be used to suggest to a reader how seriously they should consider a set of ideas. Second, though, it can have an effect on signalling games, as you suggest. Those who lack social confidence can find it harder to contribute to discussions, and having the ability to qualify statements with tags like "epistemic status: not confident" makes it easier for them to contribute without feeling like they're trying to be the center of attention.
"Epistemic effort" metadata fulfill...
Epistemic Effort: Thought seriously for 5 minutes about it. Thought a bit about how to test it empirically. Spelled out my model a little bit. I'm >80% confident this is worth trying and seeing what happens. Spent 45 min writing post.
I've been pleased to see "Epistemic Status" hit a critical mass of adoption - I think it's a good habit for us to have. In addition to letting you know how seriously to take an individual post, it sends a signal about what sort of discussion you want to have, and helps remind other people to think about their own thinking.
I have a suggestion for an evolution of it - "Epistemic Effort" instead of status. Instead of "how confident you are", it's more of a measure of "what steps did you actually take to make sure this was accurate?" with some examples including:
[Edit: the intention with these examples is for it to start with things that are fairly easy to do to get people in the habit of thinking about how to think better, but to have it quickly escalate to "empirical tests, hard to fake evidence and exposure to falsifiability"]
A few reasons I think this (most of these reasons are "things that seem likely to me" but which I haven't made any formal effort to test - they come from some background in game design and reading some books on habit formation, most of which weren't very well cited)
Results of thinking about it for 5 minutes.
Next actions, if you found this post persuasive:
Next time you're writing any kind of post intended to communicate an idea (whether on Less Wrong, Tumblr or Facebook), try adding "Epistemic Effort: " to the beginning of it. If it was intended to be a quick, lightweight post, just write it in its quick, lightweight form.
After the quick, lightweight post is complete, think about whether it'd be worth doing something as simple as "set a 5 minute timer and think about how to refine/refute the idea". If not, just write "thought about it musingly" after Epistemic Status. If so, start thinking about it more seriously and see where it leads.
While thinking about it for 5 minutes, some questions worth asking yourself: