Suppose I were to say that the American legal system is a criminal organization. The usual response would be that this is a crazy accusation.

Now, suppose I were to point out that it is standard practice for American lawyers to advise their clients to lie under oath in certain circumstances. I expect that this would still generally be perceived as a heterodox, emotionally overwrought, and perhaps hysterical conspiracy theory.

Then, suppose I were to further clarify that people accepting a plea bargain are expected to affirm under oath that no one made threats or promises to induce them to plead guilty, and that the American criminal justice system is heavily reliant on plea bargains. This might be conceded as literally true, but with the proviso that since everyone does it, I shouldn't use extreme language like "lie" and "fraud."

This isn't about lawyers - some cases in other fields: 

In American medicine it is routine to officially certify that a standard of care was provided, that cannot possibly have been provided (e.g. some policies as to the timing of medication and tests can't be carried out given how many patients each nurse has to care for, but it's less trouble to fudge it as long as something vaguely resembling the officially desired outcome happened). The system relies on widespread willingness to falsify records, and would (temporarily) grind to a halt if people were to simply refuse to lie. But I expect that if I were to straightforwardly summarize this - that the American hospital system is built on lies - I mostly expect this to be evaluated as an attack, rather than a description. But of course if any one person refuses to lie, the proximate consequences may be bad.

Likewise for the psychiatric system.

In Simulacra and Subjectivity, the part that reads "while you cannot acquire a physician’s privileges and social role simply by providing clear evidence of your ability to heal others" was, in an early draft, "physicians are actually nothing but a social class with specific privileges, social roles, and barriers to entry." These are expressions of the same thought, but the draft version is a direct, simple theoretical assertion, while the latter merely provides evidence for the assertion. I had to be coy on purpose in order to distract the reader from a potential fight.

The End User License Agreements we almost all falsely certify that we've read in order to use the updated version of any software we have are of course familiar. And when I worked in the corporate world, I routinely had to affirm in writing that I understood and was following policies that were nowhere in evidence. But of course if I'd personally refused to lie, the proximate consequences would have been counterproductive.

The Silicon Valley startup scene - as attested in Zvi's post, the show Silicon Valley, the New Yorker profile on Y Combinator (my analysis), and plenty of anecdotal evidence - uses business metrics as a theatrical prop to appeal to investors, not an accounting device to make profitable decisions on the object level.

The general argumentative pattern is:

A: X is a fraudulent enterprise.
B: How can you say that?!
A: X relies on asserting Y when we know Y to be false.
B: But X produces benefit Z, and besides, everyone says Y and the system wouldn't work without it, so it's not reasonable to call it fraud.

This wouldn't be as much of a problem if terms like "fraud", "lie," "deception" were unambiguously attack words, with a literal meaning of "ought to be scapegoated as deviant." The problem is that there's simultaneously the definition that the dictionaries claim the word has, with a literal meaning independent of any call to action.

There is a clear conflict between the use of language to punish offenders, and the use of language to describe problems, and there is great need for a language that can describe problems.

For instance, if I wanted to understand how to interpret statistics generated by the medical system, I would need a short, simple way to refer to any significant tendency to generate false reports. If the available simple terms were also attack words, the process would become much more complicated.

Related: Model-building and scapegoating, Blame games, Judgment, Punishment, and the Information-Suppression Field, AGAINST LIE INFLATION, Maybe Lying Doesn't Exist

New Comment
38 comments, sorted by Click to highlight new comments since:
In Simulacra and Subjectivity, the part that reads "while you cannot acquire a physician’s privileges and social role simply by providing clear evidence of your ability to heal others" was, in an early draft, "physicians are actually nothing but a social class with specific privileges, social roles, and barriers to entry." These are expressions of the same thought, but the draft version is a direct, simple theoretical assertion, while the latter merely provides evidence for the assertion. I had to be coy on purpose in order to distract the reader from a potential fight.

I want to quibble with this a little bit (and maybe this is that fight you were trying to avoid), but to me the draft version doesn't seem so direct and simple.

In a sense it's simple, but if I just read that statement in isolation, it's less clear to me as a reader what you mean by it. Maybe largely because I'm not sure what you mean by the "nothing but". If you took out the "nothing but", I would agree that it's a clear and direct (and true!) statement. But with the "nothing but" it seems obviously false on many interpretations, so I'm not quite sure how to make sense of it.

In contrast, the "while you cannot acquire..." version seems much clearer to me about what it's claiming and complaining about.

Even setting aside the norm-enforcing functions of language, "the American hospital system is built on lies" is a pretty vague and unhelpful way of pointing to a problem. It could mean any number of things.

But I do think you have a good model of how people in fact respond to different language.

Yeah, "built on lies" is far from a straightforward summary - it emphasises the importance of lies far beyond what you've argued for.

The system relies on widespread willingness to falsify records, and would (temporarily) grind to a halt if people were to simply refuse to lie.

The hospital system also relies on widespread willingness to take out the trash, and would (temporarily) grind to a halt if people were to simply refuse to dispose of trash. Does it mean that "the hospital system is built on trash disposal"? (Analogy mostly, but not entirely, serious).

everyone says Y and the system wouldn't work without it, so it's not reasonable to call it fraud.

This seems like a pretty reasonable argument against X being fraudulent. If X are making claims that everyone knows are false, then there's no element of deception, which is important for (at least my layman's understanding of) fraud. Compare: a sports fan proclaiming that their team is the greatest. Is this fraud?

If X are making claims that everyone knows are false, then there's no element of deception

"Everyone knows" is an interesting phrase. If literally everyone knew, what would be the function of making the claim? How do you end up with a system that wouldn't work without false assertions, and yet allegedly "everyone" knows that the assertions are false? It seems more likely that the reason the system wouldn't work without false assertions, is because someone is actually fooled. If the people who do know are motivated to prevent it from becoming common knowledge, "It's not deceptive because everyone knows" would be a tempting rationalization for maintaining the status quo.

If literally everyone knew, what would be the function of making the claim? How do you end up with a system that wouldn't work without false assertions, and yet allegedly "everyone" knows that the assertions are false?

This is answered in Benquo's last post, take a look at stages 3 and 4 to see how this situation can arise.

https://www.lesswrong.com/posts/fEX7G2N7CtmZQ3eB5/simulacra-and-subjectivity

I think this conversation might be suffering from ambiguity in the term "knows"; it doesn't mean the same thing across simulacrum levels. In fact, it's not clear how someone operating above SL2 can "know" anything in the standard philosophical sense. There's know-how, and there's the holding of opinions that lower SL people would agree with, but as a function of social reality, not with real "aboutness" pointing to underlying reality.

"Everyone knows" is an interesting phrase.

This is key. There's a very weird kind of knowing - somewhere between amnesia and willfully ignoring the problem - when bad data is aggregated into statistics, and those who know that the data is bad decide to rely on the statistics anyway, because it's the best they have.

This can even be entirely honest. Even if everybody really does have common knowledge that X is a lie, they probably don't agree on what the actual truth is, and acting as if the known lie X is true can be a compromise position to get stuff done, as long as X isn't too far away from the truth.

How do you end up with a system that wouldn't work without false assertions, and yet allegedly "everyone" knows that the assertions are false?

One way this might happen:

  1. Someone designs a process that requires X to happen. (This process might be entirely sensible, at the time.)
  2. This rule is embodied in a necessary component of the process (e.g. it's coded into software, or it's one sentence in a large legal document that also serves many other necessary purposes)
  3. Circumstances change so that either the original reason for X no longer applies, or some higher priority trumps the need for X.
  4. People in the field who are trying to keep the process running in the face of changing circumstances decide it is necessary to ignore the rule requiring X to happen, as a triage measure
  5. But the embodied component still prevents proceeding to the next step unless someone attests that X has happened, and the embodiment is harder to change than participants' behavior, so people feed false inputs into the component
  6. Knowledge of this work-around spreads, possibly until literally every person involved in the process is fully aware of it
  7. Since no clear harm is presently occurring, no one devotes resources to redesigning the component

(You could argue that "the software is being fooled", but that takes us back to "I don't think most people would call that fraud".)

I'm sure there are also many situations where someone is being fooled and "everyone knows" is just a comforting lie.

It's not something I'd say as a complete summary without context, but it's something that would be pretty frequently seized on and evaluated out of context even when - in context - it ought to be quite clear to a naive reader what specific patterns it's summarizing.

One answer: #toolazytolink Taboo Your Words when they could be seen as an attack. Instead of "lie" use "document reasonably expected results in lieu of actual measures outcomes" or whatever the specific instance is.

Yeah. You can use language that is unambiguously not attack language, it just takes more effort to avoid common words. In this respect it's not much unlike how discussing lots of other things seriously requires avoiding common but confused words!

The answer to your title question is pretty clearly "no". "Crime" is itself an ambiguous word, and will be motte-and-baileyed (unintentionally in many cases) among "violates a law as written", "causes serious societal harm", and "is commonly prosecuted as a crime".

This isn't (just) a problem with language - confusing words can be mitigated by using more words (sometimes a lot more). It's a problem with simultaneous divergent motives for communication. People WANT multiple things from that reporting: they want liability protection, they want personal deniability, they want actual measurement, they want punishment for defectors, and they want to save time on things they don't find valuable. There's no language or wording fix for that.

This seems related to the noncentral fallacy. A descriptor may be technically accurate but have misleading connotations.

Agreed, it seems very similar to (maybe exactly like) the "Martin Luther King was a criminal" example from there.

It seems to me that you're taking the position opposite MLK's, and my position is pretty much MLK's.

MLK never equivocated about whether he was disobedient towards US law. He just asked people to accept the legitimacy of the justice over that of US law. As he wrote in Letter from a Birmingham Jail:

You express a great deal of anxiety over our willingness to break laws. This is certainly a legitimate concern. Since we so diligently urge people to obey the Supreme Court's decision of 1954 outlawing segregation in the public schools, it is rather strange and paradoxical to find us consciously breaking laws. One may well ask, "How can you advocate breaking some laws and obeying others?" The answer is found in the fact that there are two types of laws: there are just laws, and there are unjust laws. I would agree with St. Augustine that "An unjust law is no law at all."
Now, what is the difference between the two? How does one determine when a law is just or unjust? A just law is a man-made code that squares with the moral law, or the law of God. An unjust law is a code that is out of harmony with the moral law. To put it in the terms of St. Thomas Aquinas, an unjust law is a human law that is not rooted in eternal and natural law. Any law that uplifts human personality is just. Any law that degrades human personality is unjust. All segregation statutes are unjust because segregation distorts the soul and damages the personality. It gives the segregator a false sense of superiority and the segregated a false sense of inferiority. To use the words of Martin Buber, the great Jewish philosopher, segregation substitutes an "I - it" relationship for the "I - thou" relationship and ends up relegating persons to the status of things. So segregation is not only politically, economically, and sociologically unsound, but it is morally wrong and sinful. Paul Tillich has said that sin is separation. Isn't segregation an existential expression of man's tragic separation, an expression of his awful estrangement, his terrible sinfulness? So I can urge men to obey the 1954 decision of the Supreme Court because it is morally right, and I can urge them to disobey segregation ordinances because they are morally wrong.
Let us turn to a more concrete example of just and unjust laws. An unjust law is a code that a majority inflicts on a minority that is not binding on itself. This is difference made legal. On the other hand, a just law is a code that a majority compels a minority to follow, and that it is willing to follow itself. This is sameness made legal.
Let me give another explanation. An unjust law is a code inflicted upon a minority which that minority had no part in enacting or creating because it did not have the unhampered right to vote. Who can say that the legislature of Alabama which set up the segregation laws was democratically elected? Throughout the state of Alabama all types of conniving methods are used to prevent Negroes from becoming registered voters, and there are some counties without a single Negro registered to vote, despite the fact that the Negroes constitute a majority of the population. Can any law set up in such a state be considered democratically structured?
These are just a few examples of unjust and just laws. There are some instances when a law is just on its face and unjust in its application. For instance, I was arrested Friday on a charge of parading without a permit. Now, there is nothing wrong with an ordinance which requires a permit for a parade, but when the ordinance is used to preserve segregation and to deny citizens the First Amendment privilege of peaceful assembly and peaceful protest, then it becomes unjust.
Of course, there is nothing new about this kind of civil disobedience. It was seen sublimely in the refusal of Shadrach, Meshach, and Abednego to obey the laws of Nebuchadnezzar because a higher moral law was involved. It was practiced superbly by the early Christians, who were willing to face hungry lions and the excruciating pain of chopping blocks before submitting to certain unjust laws of the Roman Empire. To a degree, academic freedom is a reality today because Socrates practiced civil disobedience.
We can never forget that everything Hitler did in Germany was "legal" and everything the Hungarian freedom fighters did in Hungary was "illegal." It was "illegal" to aid and comfort a Jew in Hitler's Germany. But I am sure that if I had lived in Germany during that time, I would have aided and comforted my Jewish brothers even though it was illegal. If I lived in a Communist country today where certain principles dear to the Christian faith are suppressed, I believe I would openly advocate disobeying these anti-religious laws.

This is an attempt at a principled, conceptual distinction between just and unjust laws. The idea that criminality becomes noncentral - and therefore the idea becomes not worth applying - because it's approved of by the majority is what King describes above as "difference made legal," and as such, the basic paradigm of injustice.

If someone disapproves of MLK because he was a criminal, they disapprove of him because he was disobedient to the US Government at the time, so they're taking a position in favor of unjust laws approved by the majority. Invoking the noncentral fallacy is effectively an appeal to democracy, favoring the current majority against the past one. This can look like justice if you focus attention on particular issues where the majority opinion has become more just, but is still fundamentally opposed to *principled* distinctions, which King stood for:

I must confess that over the last few years I have been gravely disappointed with the white moderate. I have almost reached the regrettable conclusion that the Negro's great stumbling block in the stride toward freedom is not the White Citizens Councillor or the Ku Klux Klanner but the white moderate who is more devoted to order than to justice; who prefers a negative peace which is the absence of tension to a positive peace which is the presence of justice; who constantly says, "I agree with you in the goal you seek, but I can't agree with your methods of direct action"; who paternalistically feels that he can set the timetable for another man's freedom; who lives by the myth of time; and who constantly advises the Negro to wait until a "more convenient season." Shallow understanding from people of good will is more frustrating than absolute misunderstanding from people of ill will. Lukewarm acceptance is much more bewildering than outright rejection.

in hindsight I think it was pretty disrespectful of you to use King as the example in the Noncentral Fallacy post.

The idea that criminality becomes noncentral - and therefore the idea becomes not worth applying - because it's approved of by the majority is what King describes above as "difference made legal," and as such, the basic paradigm of injustice.

I don't quite follow this. As I understand, MLK says that a "difference made legal" is when a majority enforces laws on a minority that it won't enforce on itself. The part about enforcing the law on the minority seems critical to that definition.

Whereas, disparate enforcement doesn't seem like it would necessarily be part of "non-central criminality". The majority might approve of something that's technically illegal for anyone, regardless of whether they're a member of the majority group. In particular, people who approve of MLK's civil disobedience may approve of the same style of civil disobedience, no matter who does it. Or they may not. It would depend on the person. But disparate enforcement certainly doesn't seem baked into the idea.

in hindsight I think it was pretty disrespectful of you to use King as the example in the Noncentral Fallacy post.

I think you have some valid points in your comment, but it seems a bit harsh to say that it was disrespectful for Scott not to think of them (if indeed he didn't) when he made his post. If you only thought of this in hindsight, was it disrespectful of you not to think of it before either? (Sure, we should hold the author of a post to a higher standard than the reader, but still, I think the point you're making here is actually relatively subtle, in the scheme of things, so it strikes me as an overstatement to call out someone as disrespectful for not thinking of it.)

If you only thought of this in hindsight, was it disrespectful of you not to think of it before either? (Sure, we should hold the author of a post to a higher standard than the reader, but still, I think the point you're making here is actually relatively subtle, in the scheme of things, so it strikes me as an overstatement to call out someone as disrespectful for not thinking of it.)

I'm not claiming to have been morally pure at the time. I'm just claiming that this was disrespectful to King. I'm not calling for anyone to be punished or shamed here, just trying to describe an unfortunate aspect of the post worth avoiding in the future, because it seems relevant to the point under discussion.

There was probably some unfortunate element of point-scoring in the way I brought it up, for which I am sorry.

Fair enough. Thanks for the reply.

What do you think the point of prosecutorial discretion (and other sorts of institutionalized hypocrisy) is, if not disparate enforcement? Does the majority have a coherent agenda that it's afraid to inform itself about?

I'm not claiming that disparate enforcement does not exist. Just that it doesn't seem central (heh) to the idea of non-central criminality.

One can have a conception of crime that typically includes theft, assault, etc, and also be in favor of civil disobedience in some cases, while also not being in favor of disparate enforcement. In fact, I would expect that to be the common case.

I guess your last line about the majority's agenda suggests you think that's not the common case. Is that right? Perhaps that's the core of our disagreement.

It seems to me precisely the opposite: my reading is that Benquo is driving exactly at how to talk about the problem of systemic falsification of information.

If the post is noncentral, what is the central thing instead?

What are these 'misleading connotations'?

[This comment is no longer endorsed by its author]Reply

Did you read the linked essay?

Yes. (Content-wise, the linked essay is also the post.**)

What part of the essay do you find to be misleading (explicitly or implicitly*)?

*Connotation

**The linked essay is at the address, but doesn't include the address, so actually the post includes more than the linked essay.

What part of the essay do you find to be misleading (explicitly or implicitly*)?

I didn't say the essay was misleading. I said a descriptor could be misleading. That's what the article I linked talks about, and the post discusses a similar phenomenon, where true descriptors are interpreted as attacks (and I claim that some of the examples in the post seem to be similar to Scott's examples of non-central members of categories).

The linked essay is at the address, but doesn't include the address, so actually the post includes more than the linked essay.

I do not understand what this sentence is saying. When you say "address", are you talking about the URL?

are you talking about the URL?

Yes.

I said a descriptor could be misleading.

What descriptors do you find to be misleading?

Same ones Scott talks about in his essay. E.g. "MLK was a criminal."

And remember, misleading connotations. The descriptor itself is technically correct.

Since whether or not something "is" a 'central' example is defined relative to an ontology, I am asking "what is your ontology", in particular that the OP's remarks serve as a 'non-central' usage?

I think this post was among the more crisp updates that helped me understand Benquo's worldview, and shifted my own. I think I still disagree with many of Benquo's next-steps or approach, but I'm actually not sure. Rereading this post is highlighting some areas I notice I'm confused about.

This post clearly articulates a problem with having language both have a function of "communicating about object level facts" and "political coalitions, attacks/defense, etc". It makes it really difficult to communicate about important true facts without poking at the social fabric, which often results in the social fabric poking back.

I'm still not sure what to to about this – the social fabric matters. But I do think the status quo of how language typically works (even on LW) is pretty bad. It's particularly relevant among various AI orgs who's members might have wildly different worldviews or strategies, who may think each other net negative, but who nonetheless (I think) are better of sharing information and collaborating in various ways anyway. (This isn't exactly about "crimes", but I think the subject matter of the post transfers to various other domains where stating your beliefs clearly can make people really upset)

Can crimes be discussed literally? makes a short case that when you straightforwardly describe misbehavior and wrongdoing, people commonly criticize the language you use, reading it as an attempt to build a coalition to attack the parties you're talking about. At the time I didn't think that this was my experience, and thought the post was probably wrong and confused. I don't remember when I changed my mind, but nowadays I'm much more aware of requests on me to not talk about what a person or group has done or is doing. I find myself the subject of such requests quite a lot, and I think past versions of myself would have thought these requests reasonable. Anyway, my point is this post was right about something important, so I give is a +4.

(This review is taken from my post Ben Pace's Controversial Picks for the 2020 Review.)

I'm interested in hearing more about experiences you've had (perhaps in vague terms, if need be?). I'm a bit surprised that you didn't resonate with this when first reading the post.

Why is it a useful description to call the legal system a criminal organization, to define physicians as a social class, to describe X as a fraudulent enterprise, or other things in that class?

The only reason I can see is to make an attack against the thing you are describing, because if you do not intend the attack then the description is not useful- if describing an institution as a criminal organization doesn't mean that it should be dismantled (or better), then describing an institution as a criminal organization means nothing.

If saying that a title is nothing more than a social class doesn't also say that the title is without significant meaning, then saying that a title is nothing more than a social class is an exercise in definitional naval-gazing (by which I mean both a somewhat hyperbolic sense of the literal meaning and also as a dismissal of the practice of using tautological descriptions that parse as attacks without meaning them as attacks).

And so forth.


If you want to describe the problems, you have to describe, or at least cogently assert the problems. The problem that you are asserting about the American legal system seems to be related to the idea that it accepts some kinds of implicit coercion in plea bargains, but forbids other forms of coercion, and makes people claim that they haven't been coerced at all when accepting plea bargains? You never actually said why that's bad, and I honestly don't believe that it is bad- Sure, it's a little bit hypocritical; it might even be a violation of law. And the practice of plea bargaining is problematic from the start- but it would be even more problematic if there was no attempt to ensure that suspects were free from untoward coercion.

When you suggest that physicians are "actually nothing but" a social class with certain characteristics, you aren't communicating anything about the core point that being good at healing people is insufficient to become a physician. Those two words are the attacking part of the statement: "Physicians are a social class with specific privileges, social roles, and barriers to entry." is not an attack, it's an uncontroversial background statement that could easily lead into a point.

I can see why fudging times in healthcare might be bad, since recording inaccurate information about treatment could possibly result in making decisions based on inaccurate information, and developing bad norms about "insignificant" falsifications risks people falsifying more significant details. But I have to draw from my own knowledge to reach that conclusion; you seem to stop at "the system would grind to a halt if people simply refused to [follow the normal behavior in the system]", without reference to the bad things that the normal behavior in the system might cause.


"X relies on asserting Y, when we know Y to be false" is not intrinsically a problem. In most cases where "Asserting Y when it is common knowledge that Y is false" is a problem, it is a problem because it can lead to asserting Z when it is not common knowledge that Z is false and where belief in Z might be a problem.

You made a jump from:

asserting Y, when we know Y to be false

to

Asserting Y when it is common knowledge that Y is false

The difference between the two may be relevant to the contents of the OP.

The latter is a strict subset of the former, but it is the big one if "everybody knows that Y is false".

In the cases where asserting Y when Y is false, but it is not common knowledge that Y is false, the path to a problem likely passes through "People actually believe Y, because they have been told Y by someone who knows if Y is true."

In linguistics, this can be explained by the difference between the denotative meaning of a word and the connotative meaning. The denotative meaning of fraud is to obtain something by means of deception. But the connotative meaning is 'something society sees as criminal.' We don't have very good ways of communicating without connotations, especially when we need to say something straightforward that is going to trigger someone when they assume implications to what you're saying. (Intentions = what you mean, implications = what people think you mean).

Ever read an explainer article in which a writer is covering something someone has said which society sees as offensive? First the writer repeats what a shamed person has said, condemns them for it to show the audience that the writer is on the people's side, and then goes on to make the same assertion as the shamed person's, only in words that don't trigger the audience.

Or think of how in conversations, you might need to preface a statement by adding, "technically!" to make the hearer evaporate their preconceptions.

This review is not very charitable, because I think the meaning of the post is different than how they present it.

The things it describes at the beginning are clearly true, with plea-bargaining being institutionalized lying, but this is, in the end, a poorly written plea to not say so. It's not that I don't see a point. It would be nice to be able to simply describe things as they are without it being a fight, but that isn't how things would work out. Words like 'lie' and 'fraud' have their extremely negative connotations because they are extremely negative things. Anything that is a synonym for them will pick up the same connotation, because people will still hate them. (I have previously determined that I would not be able to make a plea bargain, because I would not be able to admit I did something that I didn't. I may, of course, be wrong about that since I haven't experienced it.)

If it did succeed, that would not be a good thing, and would encourage people to lie more. (Social shame and punishment is what keeps the major offenders from being even worse. We may let far too many get away with deceit, but that is hardly a good reason to make it easier and less punishable.)

I am reminded of something I heard about a simulation of corruption; even in a highly corrupt society, sufficient punishment for caught for corruption can cause the rest of society to stop being corrupt on rare occasions, since people keep their heads down when it seems likely they will get caught if they keep it up, and this makes it easier to find and make examples of the remaining corrupt actors. Once corruption ceases, it may be quite a while before it becomes corrupt again (though it will eventually in the real world.)

This is far from the first post trying to remove the shame from being a known liar, and it is far from the last. I wish it was though.

On a quality of writing note, it seems to end without having made a sufficient point. (Everything is very surface level, hiding the details of how their suggestion would work in practice. I suspect they know that, but it could just be the previously mentioned lack of charitability.)