Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Lumifer 13 June 2017 04:51:31PM 0 points [-]

a rigorously mathematically defined system for calculating calibration given someone's past prediction history

That's absolutely straightforward Bayes updating.

Comment author: ProofOfLogic 13 June 2017 05:12:23PM *  2 points [-]

Not exactly.

(1) What is the family of calibration curves you're updating on? These are functions from stated probabilities to 'true' probabilities, so the class of possible functions is quite large. Do we want a parametric family? A non-parametric family? We would like something which is mathematically convenient, looks as much like typical calibration curves as possible, but which has a good ability to fit anomalous curves as well when those come up.

(2) What is the prior oven this family of curves? It may not matter too much if we plan on using a lot of data, but if we want to estimate people's calibration quickly, it would be nice to have a decent prior. This suggests a hierarchical Bayesian approach (where we estimate a good prior distribution via a higher-order prior).

(3) As mentioned by cousin_it, we would actually want to estimate different calibration curves for different topics. This suggests adding at least one more level to the hierarchical Bayesian model, so that we can simultaneously estimate the general distribution of calibration curves in the population, the all-subject calibration curve for an individual, and the single-subject calibration curve for an individual. At this point, one might prefer to shut one's eyes and ignore the complexity of the problem.

Comment author: lmn 27 April 2017 04:05:47AM 2 points [-]

First I don't think conflating blame and "bad person" is necessarily helpful.

Sociopaths exist, and may need to be truly avoided and outed as a "bad person" (although pro-social psychopaths also exist; being a sociopath doesn't automatically make you a bad person).

The most case is what is traditionally called "being tempted by sin", e.g., someone procrastinating and not doing what he was supposed to do.

Since we assume that people do have somewhat common goals

However, not entirely common goals. A group of people can all agree that something must get done while each one also wants to get the most credit for the least amount of work. And don't get me started on situations where most of the participants are only there for a paycheck, a.k.a., the real world.

The blame model seems something like this. There are strong social norms which reliably distinguish good actions from bad actions, in a way which almost everyone involved can agree on. These norms are assumed to be understood. When someone violates these norms, the appropriate response is some form of social punishment, ranging from mild reprimand to deciding that they're a bad person and ostracizing them.

As I see the blame model is about enforcing skin in the game after the fact. When something objectively bad happens, e.g., a bridge collapses, profits collapse, a car accident, it's necessary to enforce skin in that game on those whose decisions were responsible for bringing it about, i.e., make the person responsible for a risky decision bear the downside risk, especially if he would have received the benefits if it had succeeded, e.g., a CEO who decided on a risky strategy and would have gotten a big bonus if it had succeeded should also bear a cost for failure. Of course, in the question of who is responsible social norms might be relevant, e.g., if two cars collided in an intersection, the person who ran the red light is the one responsible.

Comment author: ProofOfLogic 03 May 2017 08:54:25PM 0 points [-]

First I don't think conflating blame and "bad person" is necessarily helpful.

OK, yeah, your view of blame as social incentive (skin-in-the-game) seems superior.

The most case is what is traditionally called "being tempted by sin", e.g., someone procrastinating and not doing what he was supposed to do.

I agree that imposing social costs can be a useful way of reducing this, but I think we would probably have disagreements about how often and in what cases. I think a lot of cases where people blame other people for their failings are more harmful than helpful, and push people away from each other in the long term.

And don't get me started on situations where most of the participants are only there for a paycheck, a.k.a., the real world.

It sounds like we both agree that this is a nightmare scenario in terms of creating effective teams and good environments for people, albeit common.

However, even when the primary motive is money, there's some social glue holding things together. I recommend the book The Moral Economy, which discusses how capitalist societies rely to a large extend on the goodwill of the populace. As mutual trust decreases, transaction costs increase. The most direct effect is the cost of security; shops in different neighborhoods require different amounts of it. This is often cited as the reason the diamond industry is dominated by Hasidic Jews; they save on security cost due to the high level of trust they can have as part of a community. Some of this trust comes from imposing social costs, but some of it also comes from common goals of the community members.

The Moral Economy argues that the lesson of the impossibility theorems of mechanism design is that it would not be possible to run a society on properly aligned incentives alone. There is no way to impose the right costs to get a society of selfish agents to behave. Instead, a mechanism designer in the real world has to recognize, utilize, and foster people's altruistic and otherwise pro-social tendencies. It is also shown empirically that designing incentives as if people were selfish tends to make people act more selfish in many cases.

So, I will try and watch out for blame being a useful social mechanism in the way you describe. I'm probably underestimating the number of cases where imposed social costs are useful precisely because they don't end up being applied (IE, implicit threats). At present I still think it would be better if people were both less quick to employ blame, and less concerned about other people blaming them (making more room for self-motivation).

Comment author: lmn 24 April 2017 11:40:38PM 1 point [-]

Of course, if looked at the kind of responsibility that is compatible with blame, you'd notice it's a lot more in line with the common sense notion of the term.

Comment author: ProofOfLogic 26 April 2017 09:21:17PM *  0 points [-]

Well, yes, and I think that's mostly unfortunate. The model of interaction in which people seek to blame each other seems worse -- that is, less effective for meeting the needs and achieving the goals of those involved -- than the one where constructive criticism is employed.

The blame model seems something like this. There are strong social norms which reliably distinguish good actions from bad actions, in a way which almost everyone involved can agree on. These norms are assumed to be understood. When someone violates these norms, the appropriate response is some form of social punishment, ranging from mild reprimand to deciding that they're a bad person and ostracizing them.

The constructive criticism model, on the other hand, assumes that there are some common group goals and norms, but different individuals may have different individual goals and preferences, and these might not be fully known, and the group norms might not be fully understood by everyone. When someone does something you don't like, it could be because they don't know about your preferences, they don't know about a group norm, they don't understand the situation as well as you and so fail to see a consequence of an action which you see, etc. Since we assume that people do have somewhat common goals, we don't have to enforce norm violations with punishment -- by default, we assume people already care about each other enough that they would have respected each other's wishes in an ideal situation. Perhaps they made a mistake because they lacked a skill (which is where the constructive feedback comes in), or didn't understand the situation, your preferences, or the existing norms. Or, perhaps, they have an overriding reason for doing what they did. Social punishment (even the mild social punishment associated with most cases of blame) often doesn't fix anything and may make things worse by escalating the conflict or creating hard feelings.

If you discuss the problem and find that they didn't misunderstand or lack a necessary skill or have an overriding reason that you can agree with, and aren't interested in doing differently in the future, then perhaps you don't have enough commonality in your goals to interact. This is still different from the blame model, where sufficiently bad violations mark someone as a "bad person" to be avoided. You may still wish them the best; you simply don't expect fruitful interactions with them.

That being said, there are cases where you might really judge someone to be a "bad person" in the more common sense, or where you really do want to impose social costs on some actions. Sociopaths exist, and may need to be truly avoided and outed as a "bad person" (although pro-social psychopaths also exist; being a sociopath doesn't automatically make you a bad person). However, it seems to me as if most people have overactive bad-person detectors in this regard, which harm other interactions. I don't think this is because easily-tripped bad-person detectors are on the optimal setting given the high cost of failing to detect sociopaths. I think it's because the concept of blame conflates the very different concepts involved in cheater-detection/sociopath-detection and situations where less adversarial responses are more appropriate.

(Response also posted back to the blog.)

Comment author: lmn 24 April 2017 11:02:29PM 2 points [-]

I found this bit ammusing:

[Paragraph extorting the reader to practice heroic responsibility.]

Don’t impose this kind of responsibility on someone else. I’ve heard “take responsibility” used as a command far more often than I’ve heard it used as a motto.

Comment author: ProofOfLogic 24 April 2017 11:26:28PM 1 point [-]

Edited to "You can’t really impose this kind of responsibility on someone else. It’s compatible with constructive criticism, but not with blame." to try to make the point clearer.

[Link] Chaos and Consequentialism

1 ProofOfLogic 24 April 2017 08:43PM
Comment author: ProofOfLogic 19 April 2017 05:27:50PM 1 point [-]

Noticing the things one could be noticing. Reconstructing the field of mnemonics from personal experience. Applied phenomenology. Working toward an understanding of what one's brain is actually doing.

(Commenting in noun phrases. Conveying associations without making assertions.)

Comment author: moridinamael 14 April 2017 05:51:52PM 2 points [-]

Yeah, LW would probably not be the place to try this. I would guess that most potential karma systems only truly function correctly with a sufficient population of users, a sufficient number of people reading each post. LW has atrophied too much for this.

Comment author: ProofOfLogic 14 April 2017 10:24:47PM 0 points [-]

I really like the idea, but agree that it is sadly not the right thing here. It would be a fun addition to an Arbital-like site.

Comment author: Lumifer 13 April 2017 03:22:43PM 1 point [-]

It's a bit of a misleading name: this is not moderation, auto or not, this is a signaling system focused on showing intent and feedback.

In the online environment it basically gives you a set of easy-to-express signals with more or less defined meaning. You can build something like that into the karma system (as Slashdot did very long time ago with its +funny +insightful) or just use it as shorthand in the body of the comments.

Comment author: ProofOfLogic 14 April 2017 03:59:32AM 0 points [-]

These signals could be used outside of automoderation. I didn't focus on the moderation aspect. Automoderation itself really does seem like a moderation system, though. It is an alternate way to address the concerns which would normally be addressed by a moderator.

Comment author: Zvi 13 April 2017 12:52:17PM 2 points [-]

My impression of these ideas is that they carry a high complexity cost, in terms of everyone involved having to track a lot of signals and a lot of things, and the barriers to entry being higher. Automoderation already is on the high end of reasonable complexity costs. A similar analogy could be made with bandwidth costs, as explicit communication is attention taxing.

I do agree that these are important contrasts, and important things to communicate; in a one-on-one conversation, it is good to say explicitly which mode you are in.

Another note is that it is not obvious that people agreeing for different reasons should be presumed to be stronger evidence than agreeing for the same reason. It means there is another reason out there, but also means that the person did not agree with your reason, which may or may not be because they did not know about your reason.

Comment author: ProofOfLogic 14 April 2017 03:56:17AM 0 points [-]

True, I didn't think about the added burden. This is especially important for a group with frequent newcomers.

I try hard to communicate these distinctions, and distinctions about amount and type of evidence, in conversation. However, it does seem like something more concrete could help propagate norms of making these sorts of distinctions.

And, you make a good point about these distinctions not always indicating the evidence difference that I claimed. I'll edit to add a note about that.

[Link] Thoughts on Automoderation

2 ProofOfLogic 12 April 2017 09:29PM

View more: Next