I was reading the Slate Star Codex review of The Cult of Smart. Part of it discusses charter vs state schools and the allegations of fraud of various kinds undermining charter schools record of better achievement. Reading it, I realized that I took for granted that public schools engage in systematic fraud in a variety of ways. I don't think this is something everyone understands, hence this post.
I went to a state school in the UK. State schools are rated on a 1 - 4 scale from unsatisfactory to outstanding. My school was rated good, meaning a 3. A few memories which stand out. During my first week I saw one of the boys in my class who was 11 at the time held up against the wall in a corridor while a 16 year old put a shiv to his throat and robbed him. He handed over his wallet and keys. A year or two later and I remember seeing a small boy who struggled with depression held up by the throat against a locker and slapped in the face by a troublemaker from the same class in front of everyone just before we went in to the classroom. I remember classes which were filled start to finish with people shouting and talking. Neither of the first two events were common but they also weren't uncommon. No one was surprised to witness them. It's worth emphasizing again that my school was above average, in fact quite far above average, and in a middle class area. It's also worth noting that I was mostly in top ability streamed classes, meaning my classroom experience was likely far better than average.
There were many ways in which the school and teachers gamed the system to boost their measured performance. One way was to do exams for students. I was on a bottom set language class for French. After two years I literally couldn't speak a single sentence in french and maybe knew 20 words in total. I still passed my exams. How? We did the tests in class. Often the teacher would go through them with us. Literally giving us the test and then going through each question on the whiteboard and telling us what to write. A different year and a different teacher, this time the teacher would sit next to us and write the answers down. Why sit next to us? It was the bottom set so people often wouldn't even bother to write down the answer if they were told it. This kind of thing was normal, so much so that I, and I think most people there, didn't realize anything unusual was happening.
Another way schools game metrics is to cheat inspections. A major component of how schools are judged in the UK is through independent inspections carried out by an independent quasi-governmental organization called Ofsted. Now, you may imagine that these inspections would be unannounced, so as to best get a real image of how a school works. Not the case. They're scheduled well in advance. Before every inspection, a few things would happen in my school:
- The worst troublemaker kids would be taken aside and put in a special room where inspectors wouldn't see them. Either that or they would just be told not to come into school at all on that day.
- All of us were told in assembly that an inspection was coming and to be on our best behavior on that day. Often teachers would have conversations with less serious troublemakers and impress on them that they would behave on that day or face consequences afterwards.
- Teachers would put a great deal more effort into their lesson plans than was normal. Classroom behavior management would also be far stricter. Because of these and other measures my school during an inspection was utterly different than my school on a normal day. On some level this isn't surprising. If teachers' promotions and management's jobs depend on good inspection results and inspections are easy to game, people will game them. Incentives drive behavior. But it's still sad.
Another way the stats were gamed was by not recording bad behavior. When a school gives a detention or suspends/expels a student, there's a record of it. This is especially true of suspensions, students being sent home or expulsions. The more of these you have, the worse you look as a school. The solution then is obvious, don't punish people or punish them in non-recorded ways. Again, in my school it was completely normal for students in lower sets to swear at the teacher, talk over them or disrupt the class for everyone else. It was normal for someone to be aggressive and abusive towards others and to face at most a 40 minute detention, but even getting a detention would be unusual.
I realize that one data point is not enough to draw solid general conclusions. My own perception is that this kind of fraud wasn't specific to my school. My cousin went to a state school fairly nearby. He's 4 years younger than me. During one of my winters back from undergrad we discussed his school and his experiences mirrored mine. His exact words regarding inspections were "I learned 4 times more that day than any other day that year. It was amazing". I talked to a few British students at university, although specifically the not middle/upper class ones who would have gone to public schools. They had gone to schools similar to mine in different parts of the country and their stories were similar and often worse. Two particularly funny examples from my friends' experiences stand out. A teacher in year 9 walked up to a student who was talking, picked them up and threw them out of an (open) first floor window. My friend sitting in class noticed two boys making fun of him and then proceeded to get up in the middle of class while the teacher was talking, walk to their table, flip the table upwards to hit them in the face before going to sit down again when the teacher told him to. (Remember, my friend was a studious, sporty Asian kid and not a troublemaker. This kind of thing is normal in that environment). Comedic stories aside, my experiences in school, while not universal, seem fairly common in the UK and from what I've read of the statistics, bad US schools are far, far worse.
I'm unsure what my point here is. I think I have two:
- Charters may cook their books in various ways. In the UK, State schools do too. I would be surprised if it wasn't also the case in the US.
- I think that I feel like a lot of commentators on places like SSC have fairly middle class experiences of fairly good schools and that bleeds into how their comparison between state vs charter schools. It's just good to remember that it's not those nice middle class schools that charters typically replace.
Crossposted to my blog at https://dissent.blog/2021/02/20/how-my-school-gamed-the-stats/
I don't actively "want to continue", in that it seems to me that the whole content of this discussion is you saying or implying that I've badly misrepresented how affluent my area is, and me pointing out in various ways that that isn't so.
However, your last paragraph seems once again like an accusation of inconsistency, so let me clarify.
"Upper middle class" means different things in different places. In the US, "class" is largely (but not wholly) about wealth. In the UK, "class" is largely (but not wholly) about social background. These are less different than that makes them sound because the relevant differences in social background are mostly driven by the wealth of one's forebears, and in both societies there is a strong correlation between that and one's own wealth.
The US has a more wealth-based notion of class and is also richer. So being "upper middle class" in the US means a level of wealth that would make you quite rich in the UK.
The UK has a more social-background-based notion of class, which in particular is strongly influenced by the existence of a (statistically very small) aristocratic class. So "upper class" in the UK means a smaller, more-elite fraction of the population than in the US, and "upper middle class" is pulled in the same direction. So being "upper middle class" in the UK typically (but not always, because of the wealth/background distinction) means being at a distinctly higher percentile of wealth than it does in the US.
The combined effect of these things is to put the typical "upper middle class" person or family at something like the same level of wealth in the two countries, although there's plenty of fuzziness and variability.
(Perhaps I have by now made it clear that I do have some idea how social class works in the UK, enough so that you might believe me if I tell you that (1) I am definitely lower-upper-middle-class and (2) my household income is somewhere around the 95th percentile.)
So, issue 1 is that you're wanting to call my neighbourhood "upper middle class" even though the people here don't fit either the UK or the US notion of "upper middle class", because you think it matches the definition you'd get if you applied the US-based notion directly to the UK despite the substantial differences between the two societies.
This would be (annoying but) excusable given that your main point was to suggest that my daughter's school may be more atypical than I was claiming. But there's more.
That percentile-ranking tool is not ranking individuals, it is ranking areas of the country. Areas vary less than individuals do, and (e.g.) an 80th-percentile area is not composed mostly of 80th-percentile individuals. On the other hand, terms like "upper middle class" are descriptions of individuals and families, and only secondarily of areas. If calling an area "upper middle class" (or "upper class", or whatever) means anything, it should mean an area whose people are mostly of the class in question.
Almost no areas are "upper class" or even "upper middle class".
(Perhaps an analogy may help. Suppose you have an area where the Jewish population is at the 95th percentile of areas in the country. Would you call it "very Jewish"? You probably shouldn't, because I bet that 95th-percentile population is still <5%. I submit that "upper middle class" is like "Jewish" in this respect.)
In a typical 80th-percentile area, most people are middle-middle-class. That's unusual; in a more typical area a large fraction will be lower on the socioeconomic scale. How might you describe such an area? Well, maybe as a "nice middle-class area", for instance. Which happens to be exactly the term I used.
"But £36k is distinctly higher than a typical middle-middle-class salary in the UK." It's not all that much higher; median UK household income is £30k. Second, the £36k figure is a mean, not a median. Mean incomes are always higher than median incomes. UK mean household income turns out to be about £37k, if I've done my calculations correctly. Both the area mean of ~£36k and the national mean of ~£37k are described as "equivalised" and I don't know whether that means the same thing in both cases, but the point is that this area is in fact about as well-off as the UK as a whole. No contradiction with the 80th-percentile thing; most areas are (when you calculate in terms of means) a bit poorer than average and a few are substantially richer.
So, issue 2 is that you're taking terms that describe individuals and applying them to areas in a profoundly misleading way. You can call an 80th-percentile individual "upper middle class" if you like, though actually most classifications wouldn't call them that either in the UK or in the US; but an 80th-percentile area is still not an "upper middle class" area. That's not how the words work.)