Problems in Education

65 Post author: ThinkOfTheChildren 08 April 2013 09:29PM

Post will be returning in Main, after a rewrite by the company's writing staff. Citations Galore.

Comments (318)

Comment author: jaime2000 01 November 2015 01:00:49PM 2 points [-]

Since the post never came back (much less with "citations galore"), here's a mirror.

Comment author: Douglas_Knight 06 October 2016 04:49:40AM 0 points [-]

It came back here

Comment author: Houshalter 22 July 2013 04:43:34PM 2 points [-]

I don't suppose anyone saved the original version of this before it was edited?

Comment author: buybuydandavis 13 April 2013 11:20:21AM 3 points [-]

What would be the point of the hoax?

Ha ha, people believed a story about a fucked up bureaucracy! How foolish they are! Bureaucracies are the perfect will of God here on earth!

A fictional account of a screwed up bureaucracy, or a screwed up school district, would hardly prove that they don't exist.

The post seemed a little strange at some points, and a little like spam for "bayesian predictive models for mathematics placement". Maybe some company has a patent on that?

Whatever. Hoax or no hoax, lots of good stories and good times.

Comment author: AustinParish 11 April 2013 06:25:57PM *  5 points [-]

Currently, my firm and its allies are trying to push the government into forcing the schools to use a Bayesian prediction model, in which you feed an individual student's test scores for the past 5 years and it spits out their probability of success in the advanced classes, and you keep putting the students with the highest probability of success in the top classes until you run out of teachers.

This is good, and I hope that such models are implemented. However, when I hear the phrase "problems in education," these sorts of placement problems are not what comes to mind first.

Having personally taught at a massively failing inner city high school for several years (where only 2% of students were white, and only 10% met state goals for education), the few "advanced classes" that my school offered were, indeed, filled by students who had achieved top scores in a variety of metrics (top, as compared to the other students at the school). I taught such an advanced class, as well as several general placement classes. The administration assigned students to each class without using a Bayesian model, but I honestly don't think the resulting student distributions would have changed much if they had used one.

The problem was never making sure that the students with the highest probability of success made it into the advanced classes; my administration, for whatever its other failings, had mostly solved that one. The consuming, stultifying problem was that in my advanced 10th grade classes, only a few of my students could read even at an 8th grade level. The situation was even worse in my general classes, where most students read at a 6th grade level.

Comment author: FFFvolleymama 11 April 2013 12:49:38AM 1 point [-]

I saw an example of exactly this today from an independent sports program that partnered with the county public schools, repeating that the outcome measure was that they "provided this opportunity"; that "meetings were held"; that "students received flyers", etc. etc.

Now I understand that which confused me thanks to this article. Kudos!

Comment author: mwengler 15 April 2013 07:39:42PM 1 point [-]

Of course if the purpose of the grant was to fund a sports program and to publicize it with flyers, then these would indeed be the outcomes sought. Not every program is a research program.

Comment author: CAE_Jones 10 April 2013 09:54:08PM *  2 points [-]

I live in Arkansas (you may remember us as the state that threw a fit over desegregation roughly a Jesus-lifetime ago), in a region that is pretty economically strong but still has distinct socioeconomic classes. I'm pretty confident that this post describes reality pretty well, based on 1, my direct observations as a student, 2, what I've heard from other students, past and present, and 3, what I've heard from teachers and principles, retired and practicing.

[edit] To be more clear, I think there's almost certainly a county or several in the United States where the article could apply, but not necessarily a majority. Whether or not the OP is being completely honest, on the other hand, I don't know (the username seems designed with emotional impact in mind, which is rarely a good sign).[/edit]

Comment author: Bugmaster 10 April 2013 07:59:42PM 1 point [-]

However, this process yields sheer lunacy, mostly because of the ridiculous ineptitude of every single person involved. ... she was flabbergasted, and explained that the evaluators job was to collude with the grant proposal submitter...

I think you're mistaken, and that most of the people involved in this process are, in fact, acting quite rationally and efficiently. They are merely solving a different problem. Your goal is to improve education; their goal is to channel as much money as possible into the pockets of everyone involved. The system you describe works quite well for that purpose.

You could argue that such a system is immoral, but it is eminently rational (at least in the short term), since it provides a way to acquire significant resources at very low risk. This is better than, for example, betting on the stock market or creating a startup.

Comment author: [deleted] 10 April 2013 07:23:41PM *  4 points [-]

I wish I had computer acess to write out a longer reply to this, for now see educationrealists response and his blog in general. I was torn wether to upvote or downvote the article as I don't know whether it was exploiting or exposing key weaknesses of community rationality here.

Comment author: OrphanWilde 10 April 2013 04:06:40PM 47 points [-]

Did some research. The claim that the proposals are poorly written leaps out at me as immediately true. Here's a website with successful grant applications, to be used as models to write them:

http://www.k12grants.org/samples/samples_index.htm

This is the first grant I pulled up (it's not the first, but it -was- the first I felt competent to evaluate, concerning primarily technology):

http://www.k12grants.org/samples/grantkay.pdf

First, the horrible spelling, grammar, and punctuation leap out at me immediately. Second, the claim in the post that grant proposals are written to describe what they're doing, rather than what they're intending to achieve, holds up, for this grant at least.

http://www.k12grants.org/samples/MH%20grant.pdf

This proposal is the best-written I encountered. It describes the specific problems it intends to resolve and the specific solutions it intends to use. Unfortunately, the only evidence it introduces is the evidence that there is a problem. It doesn't provide any evidence that its solutions work. Its stated "Method of Evaluation", moreover, exactly mirrors the claims made in this post - it evaluates whether or not its solutions are implemented, NOT whether or not the problem is solved. (Goal #5 seems like an exception, but remember the stated problem is mental health issues.

http://www.k12grants.org/samples/TARGET.pdf

This proposal is the best I've encountered. It is horribly written, however. (You can skip past the pages and pages of documentation about how exactly the money will be spent to read the goals.) The accountability section has this (this is a proposal, essentially, to buy more modern computers for students and teachers, and to hire support staff):

We will begin by taking benchmarks of our current situation with regard to number of computers per student (including the capability of that equipment), number of teachers and students currently using the PLATO or other learning systems, number of teachers using the Web as a training and communication tool, and student scores on the TABE and PLATO assessment tests. On a quarterly basis, we will review computer ratios and teachers/students using PLATO. Every semester we will use questionnaires and surveys, as well as observation, of staff to get feedback on the impact of professional development activities. Student scores will be reviewed after every semester. Results will be tabulated and communicated to school staff, superintendents, Advisory Boards and Texans Can! staff annually. Where indicated, adjustments in curriculum and instruction will be made to ensure that student performance continues to improve

Note that the accountability, as it pertains to this grant, is - wait for it - to make sure the grant money is spent as expected. You can change curriculum and instruction -without- the computers, remember. (I think this is a pretty sensible grant request, but the accountability measures it proposes provide no actual accountability. "Did we say what we were going to do? Yes? Then our grant was a success!")

http://www.k12grants.org/samples/FLAP%20Narrative.pdf

Setting aside the fact that I've seen better writing from middle school students, this is actually a decently written grant. It has specific goals, implementations, and even has accountability. (Although it does seem confused about who or what is accountable to who or what; the accountability section reads rather like the author's understanding of accountability meant the ability of students to measure their own improvements in performance. Notably omitted is a suggestion that the program's success/improvement rate be compared to non-program success/improvement rates.)

http://www.k12grants.org/samples/2003_Library&Literacy.pdf

A well-written grant I can't find fault with. (Except maybe its questionable notion of scientific evidence.)

So - some of the grants here definitely show symptoms of the problems indicated in the post. Some don't. A couple of these had no business being granted. ALL of these grants were successful applications - that is, the grants were granted.

After this exercise, my position shifted from "This post is credible" to "This post exaggerates the extent of the problem to some degree, but remains a valid criticism of the grant system as it exists."

And I tried to find a grant similar to the iPod/Makeover grant, and found this:

http://www.msmagiera.com/ipad-grant

Okay, not exactly analogous, as it at least pertains to education. However, given the grant's self-evaluation criteria, student scores could plummet and the project could still call itself an overall success. (Actual improvement in student abilities only accounts for a fourth of their apparently unweighted criteria.)

Comment author: mwengler 11 April 2013 12:44:44AM 2 points [-]

http://www.k12grants.org/samples/grantkay.pdf

First, the horrible spelling, grammar, and punctuation leap out at me immediately.

I just read that grant in its entirety. I noticed one possible typo, but did not find other bad grammar or spelling.

Second, the claim in the post that grant proposals are written to describe what they're doing, rather than what they're intending to achieve, holds up, for this grant at least.

The are asking for a grant to get equipment, primarily computers and software, for use in teaching students. It is not really a research project. What is the outcome hoped for from a grant like that? That students will be taught using these computers. They make a feint at claiming it will raise grades or enrolement, but really if I were a science teacher, my real goal would be to get the stuff and sit students down in front of it and teach them with it. I think that is pretty accurately reflected.

I'll look at the ipad grant, and kudos for finding the site and bringing me that much closer to real contact with the kinds of grants under discussion.

Comment author: Decius 11 April 2013 04:37:57AM 4 points [-]

Quotes from the .pdf, with my corrections:

With request to the Boulder Valley School District Science Content Standards, Nederland Middle Senior High School is highly motivated to reform their Science Department.

respect, regard, or reference.

Teachers will attend training sessions to become proficient with the technology and learn ho to integrate this technology into their classroom.

how

In order to meet the Science Content Standards set forth by Boulder Valley School District, our Science Department requires hardware. software, training and curriculum.

hardware, software

With a Microcomputer Based Laboratory (MBL) students can perform sophisticated experiments, collect and manipulate data, share their findings with classmates and do in depth analysis of natural phenomena.

in–depth

Productivity in the laboratory will be increased due to computers performing the data manipulation, enabling students more time to concentrate on scientific principals and concepts.

principles

With a more hands-on approach to science, many students who lose interest in science past the graduation requirements could find science to be more relevant to their day to day lives.

day–to–day

This science lab will be in place alter the grant period is over.

after

Team Labs will receive $30,175 for science curriculum, software and probeware. In addition, they will receive $2.250 for teacher training.

$2,250

Consistently 'moneys' is used where 'money' or 'monies' seems correct to me; I did not count this as an error despite not following a strict style guide. Most other 'errors' are very reasonably scanning errors rather than writing errors; the only error that couldn't plausibly be a scanner error would be 'principal' for 'principle'.

Overall, the writing was simplistic, sentences were short and simple, and would pass a technical writing test. Presented as a model for what complexity and intelligence level of grants are approved, that is very informative. Grant proposals (apparently) should be simple, repetitive, and full of Capitalized Buzzwords that are Important to the Right People.

Comment author: OrphanWilde 11 April 2013 07:43:59PM 2 points [-]

I actually counted the really short sentences heavily against them mentally, probably too much. Owing to the way I parse sentences, reading the grant was like listening to William Shatner at his... not quite hammiest, but pretty close.

As far as the 2.250 thing, that's actually not that uncommon outside English-speaking nations; see http://en.wikipedia.org/wiki/Decimal_point#Countries_using_Arabic_numerals_with_decimal_comma which lists countries which use decimals as thousands separators and commas as decimal marks. (That may actually help to explain the short sentences, come to think of it.)

An alternative explanation is offered here: http://lesswrong.com/lw/h5z/problems_in_education/8qqj (Specifically, that the document may have been electronically scanned; this could also account for other apparent spelling mistakes. Handwriting recognition is getting better, but is still far from perfect.)

Comment author: glenra 11 April 2013 03:58:17AM *  7 points [-]

First, the horrible spelling, grammar, and punctuation leap out at me immediately.

Me too. Good thing they're not trying to improve writing ability!

I just read that grant in its entirety. I noticed one possible typo, but did not find other bad grammar or spelling.

The VERY FIRST SENTENCE has minor punctuation issues and refers to "Excellence in Leaning (sp) Through Technology" - I refuse to believe that the original Senate bill being referred to failed to spell the word "Learning" correctly in its title. :-)

The second sentence puts a space before the colon for no apparent reason.

"The moneys this school is requesting" => should probably be "money", though I'd accept argument to the contrary. "With request to <standard>..." => should probably be "With RESPECT to"

"This shows community support for improvement and a move forward with the support of a technology plan." => You can tell what the writer is trying to say, but the writer is not actually saying it; the sentence is just broken.

"Teachers will...learn ho to integrate this technology" => should be "learn HOW to integrate..."

That's just the first page, and it's not even ALL the issues on the first page. Fortunately, the following pages are much better than the abstract page (which was painful). The second page is missing a bunch of hyphens - that's a problem throughout - but otherwise not too bad.

Third page: "A desired outcome of this project is an increase in tile number of students taking high level science." => change "tile number" to "total number" and possibly change "high level" to "high-level"

"By using MBL’s, less time is required" => change "MBL's" to "MBLs" - it's not a possessive.

"The purchase of this equipment would be in support of Colorado economy." has a missing article; change it to => "would support THE Colorado economy"

"accommodate this set Up." => "setup".

Under IMPACT: "By obtaining these funds and implementing this program more students will be able to participate in hands on leaning" => again, it should be LEARNING, not LEANING. Also it's "hands-on", not "hands on"

"This science lab will be in place alter the grant period is over." => AFTER the grant period, not ALTER.

Much of this suggests a very bad writer - less than 8th-grade level - who is using a spell-checker. But there some other mistakes that seem like the document might have been electronically scanned. For instance, the budget mentions "guides for teachers arid students" => should obviously be "teachers AND students" but I can't imagine a human writer accidentally writing "arid" for "and" and "ri" does look an awful lot like "n".

Comment author: SomeCallMeTim 11 April 2013 05:20:51PM 6 points [-]

Agree with everything but:

"By using MBL’s, less time is required" => change "MBL's" to "MBLs" - it's not a possessive.

If you look in an old enough style guide (the current standard is as you say), it will say to use an apostrophe when you pluralize an acronym. Wikipedia agrees.

Comment author: Michelle_Z 10 April 2013 07:02:38PM 1 point [-]

How, exactly is it, that these people get hired in the first place?

Comment author: gwern 10 April 2013 07:20:34PM 10 points [-]

I'll put it this way: in the average GRE scores by intended field, education ranks below philosophy & STEM in every subtest, and various forms of education rank very low (early childhood education is, out of 50 groups, second from the bottom in 2 subtests and fifth from the bottom in the last subtest).

Comment author: buybuydandavis 13 April 2013 10:32:47AM 1 point [-]

Those who can't do...

Another category that jumped out at me - see all the public/education/business administration near the bottom. These are the people running institutions. The world is the way it is for a reason.

Comment author: educationrealist 11 April 2013 06:37:54AM 3 points [-]

Average GRE is useless. Elementary teachers have far lower GRE scores than secondary school teachers, and are about average in verbal and below average in math. Secondary school content teachers are above average in verbal and average in math. However, close to half of all secondary school teachers get higher than 600 on the math section, which is more than the number of math and science teachers. While I suppose it's possible that math and science teachers have terrible math scores and the English/history teachers are scoring those 600+ scores, I'm figuring it's far more likely that math and science high school teachers have eminently respectable GRE scores in math, and that English/history teachers have higher than average verbal.

Anyone who claims that teachers are stupid is using propaganda instead of ETS data.

<A href="http://educationrealist.wordpress.com/2012/01/15/teacher-quality-pseudofacts-part-ii/">Cite for SAT scores</a> and for <a href="http://educationrealist.wordpress.com/2012/01/12/teacher-quality-pseudofacts/">GRE scores</a>

Comment author: glenra 11 April 2013 06:02:32PM 3 points [-]

What makes "higher than 600 on ONE section" a cutoff above which counts as an "eminently respectable" score?

Anyone who claims that teachers are stupid is using propaganda instead of ETS data.

Would you accept "mediocre"? ;-)

Comment author: Matt_Simpson 11 April 2013 01:03:56AM *  0 points [-]

I notice that I'm confused: the maximum score on the Quantitative section is 800 (at that time), and Ph.D. econ programs won't even consider you if you're under a 780. The quantitative exam is actually really easy for math types. When you sign up for the GRE, you get a free CD with 2 practice exams. When I took it, I took the first practice exam without studying at all and got a 760 or so on the quantitiative section (within 10 pts). After studying I got a 800 on the second practice exam and on the actual exam, I got a 790. The questions were basic algebra for the most part with a bit of calculus and basic stat at the top end and a tricky question here and there. The exam was easy - really easy. I was a math major at a tiny / terrible liberal arts school; nothing like MIT or any self respecting state school. So it seems like it should be easy for anyone with a halfway decent mathematics background.

Now you're telling me people intending to major in econ in grad school average a 706, and people intending to major in math average a 733? That's low. Really low relative to my expectations. I would have expected a 730 in econ and maybe a 760 in math.

Possible explanations:

1) Tons of applicants who don't want to believe that they aren't cut out for their field create a long tail on the low side while the high side is capped at 800.

2) Master's programs are, in general, more lenient and there are a large number of people who only intend to go to them, creating the same sort of long tail effect as above in 1).

3) There's way more low-tier graduate programs than I thought in both fields willing to accept the average or even below average student.

4) Weirdness in how these fields are classified (e.g. I don't see statistics there anywhere, is that included in math?)

5) the quantitative section of the standard GRE actually doesn't matter if you're headed to a math or physics program (someone in that field care to comment?). Note: the quantitative section of the standard GRE does matter in econ, but typically only as a way to make the first cut (usually at 760 or 780, depending on the school). I don't know much of the details here though.

6) very few people actually study for the GRE like I did - i.e. buy a prep book and work through it. This depresses their scores even though they're much better quantitatively than I am.

Unsurprisingly since these are in when-I-though-of-them order, 1)-3) appeal to me the most, but 5) and 6) also seem plausible. I don't see why 4) would bias the scores down instead of up so it seems unlikely a priori.

Comment author: gwern 11 April 2013 01:32:42AM 1 point [-]

I was actually too lazy to study for my GRE, so I think I got like in the 600s on the math section (it had been a long time since I had studied any of that stuff); I realized while taking it that this was a stupid mistake and I was perfectly capable of answering everything, but the GRE cost too much for me to want to take it a second time. Oh well.

Statistics does not seem to be broken out in the latest GRE scores I found: https://www.ets.org/s/gre/pdf/gre_guide.pdf I think statistics is almost always part of the math department.

My guess is that there are a lot of grad schools (consider law schools, the standard advice is to not bother unless you can make the top 10, yet there are scores if not hundreds of active law schools), and few actually intend to do a PhD.

Comment author: Pentashagon 10 April 2013 08:00:02PM 1 point [-]

Education even ranks below Religion in every category. Also, Economics is only quantitatively better than Religion. </abuse of ranked lists of things>

Comment author: Matt_Simpson 10 April 2013 09:47:15PM *  5 points [-]

Not surprising, given my experience. Most religion majors I've met were relatively smart and often made fun of the more fundamentalist/evangelical types who typically were turned off by their religion classes. Religion majors seemed like philosophy-lite majors (which is consistent with the rankings).

Edit: Also, relative to Religion, econ has a bunch of poor english speakers that pull the other two categories down. (Note: the "analytical" section is/was actually a couple of very short essays)

Comment author: gwern 10 April 2013 08:13:08PM 4 points [-]

Also, Economics is only quantitatively better than Religion.

Yes, given that economics is apparently one of the most lucrative fields around going by Caplan's recent post on majors, it's interesting that the econ students aren't ranked even higher.

Comment author: Eliezer_Yudkowsky 10 April 2013 09:28:32PM 3 points [-]

Chicken-and-egg problem: Non-economics majors don't think economically enough to choose fields on the basis of their remuneration?

Comment author: Matt_Simpson 10 April 2013 09:46:14PM 0 points [-]

That seems to explain why Econ majors get a premium, but that doesn't seem to explain why econ majors don't rank higher, or am I missing something?

Comment author: knb 09 April 2013 11:57:02PM *  18 points [-]

This post is popular not because it is accurate, but because it repeats the popular misconceptions about the US education system, and tells both left and right what they want to hear:

Of course, the biggest myth that the media reporting of PISA scores propagates is that the American public school system is horrible. The liberal left in U.S and in Europe loves this myth, because they get to demand more government spending, and at the same time get to gloat about how much smarter Europeans are than Americans. The right also kind of likes the myth, because they get to blame social problems on the government, and scare the public about Chinese competitiveness. We all know that Asian students beat Americans students, which "proves" that they must have a better education system. This inference is considered common sense among public intellectuals. Well, expect for the fact that Asian kids in the American school system actually score slightly better than Asian kids in North-East-Asia!

American students generally outperform their racial group in other countries. White Americans have higher PISA scores than any European country except statistical outlier Finland. Asian Americans beat every Asian country, and are second only to the wealthy, elite Chinese city of Shanghai (another statistical outlier).

Hispanic Americans are mostly Mexican-Americans, but outscore Mexico by a healthy 41 points. They are only 15 points behind Spain--and note that many Hispanic-Americans are recent immigrants and don't speak English as a first language (but had to take the test in English), while Spaniards take the test in Spanish.

African-Americans outperform Trinidad (Trinidad is a developed country with a high per capita GDP, and has a substantially African population, which makes them perhaps the most comparable group.)

This really seems to disconfirm both the liberal and conservative talking points. The US education system is not underfunded, as liberals say, nor is it underperforming, as conservatives say. It also is not correct that the US is systematically failing racial minorities due to institutionalized racism (as the OP claims).

The picture I have of the US education system is that there are a large number of smart, dedicated, people spending a lot of money trying get the best outcomes they can with the students they have to work with. This is all irreconcilable with the claims the OP makes.

Comment author: buybuydandavis 13 April 2013 10:57:47AM 0 points [-]

American students generally outperform their racial group in other countries.

I don't see that supported by the data, since other countries aren't broken out by race.

Also, how biased is the sample? What percentage of Americans take the test? And Europeans?

Comment author: [deleted] 13 April 2013 11:03:09AM 0 points [-]
Comment author: buybuydandavis 13 April 2013 11:59:52AM 1 point [-]

In all countries, schools are sampled, attempting to control for some variables. Around 5000 students, 160 schools in the US. Note that schools may/may not choose to participate, and the same with students. PISA has minimum standards for acceptance as sampled selection, in an attempt to avoid the obvious bias that countries would have an interest to produce in their samples.

http://nces.ed.gov/surveys/pisa/faq.asp

Given the optional sampling, and obvious incentives, I'm skeptical that this is particularly accurate, and an apples to apples comparison.

Also, homeschoolers, one of the strongest demographics for ethnic and intact family reasons, are likely missed by this sample, skewing results in the US down. They appear to be about 4% of the total population, disproportionately white and in two parent households.

Comment author: [deleted] 13 April 2013 12:18:25PM *  1 point [-]

Note that schools may/may not choose to participate, and the same with students.

They control for schools opting not to participate; see the section under substitute schools.

It's standard research ethics that minors (and their guardians) be given the option to refuse to participate in a study.

Also, homeschoolers, one of the strongest demographics for ethnic and intact family reasons, are likely missed by this sample, skewing results in the US down. They appear to be about 4% of the total population, disproportionately white and in two parent households.

Now I'm confused. The original inference Sailer drew from PISA was that American students outperform their racial group in other countries. You're claiming the study will be biased against white Americans. If anything you should be annoyed at Sailer for trying to support his racial performance narrative using a study that didn't really focus on it.

Regarding homeschooling in particular, it'd be nearly impossible to develop an international study on the same scale of PISA (which you already want to reject as too small) merely because homeschooling isn't prevalent in most of the PISA-participating countries.

Comment author: buybuydandavis 13 April 2013 11:45:51PM *  1 point [-]

They control for schools opting not to participate; see the section under substitute schools.

I'm aware that they tried to control, but the bounds are large and open to exploitation by the countries who so choose to do so.

And the homeschool issue should strengthen his conclusion. I was just noting a factor he hadn't controlled for.

Both factors I pointed to would tend to mean that the relative rank for the US is in reality better than listed, IMO, but the wide bounds of substitution adjustment makes it very hard to be confident in the results.

Comment author: Decius 11 April 2013 04:17:34AM 1 point [-]

and note that many Hispanic-Americans are recent immigrants and don't speak English as a first language (but had to take the test in English), while Spaniards take the test in Spanish.

In states with high Hispanic-American populations, that portion is false.

Comment author: PrawnOfFate 10 April 2013 06:01:57PM 8 points [-]

Just imagine...there are countries where education can be discussed without bringing in race at all...

Comment author: AspiringRationalist 10 April 2013 04:24:00AM -1 points [-]

American students generally outperform their racial group in other countries. White Americans have higher PISA scores than any European country except statistical outlier Finland. Asian Americans beat every Asian country, and are second only to the wealthy, elite Chinese city of Shanghai (another statistical outlier).

Given that there are racial gaps in the test scores, it's not fair to compare the average of white Americans against the nation-wide averages of European countries, since they also have significant non-white populations.

Comment author: knb 10 April 2013 05:21:54AM *  10 points [-]

If you follow my first link, you can see the author's analysis is demographically neutralized (it excludes 1st and 2nd generation immigrants in European countries, and compares to white Americans). In this ranking, American whites substantially outperform the European average, and only 2 small European countries (Switzerland and Finland) noticeably outrank American whites. US whites are outscoring the EU-15 (basically the core nations of the EU, before it expanded into Eastern Europe), by a substantial amount.

The second image is not demographically neutralized, but European countries have far, far lower non-white percentages than the United States. For example, Germany is about 10% non-white as of 2010.

Comment author: [deleted] 10 April 2013 05:06:33AM *  2 points [-]

The image appears to come from Steve Sailer, who is not the most reliable source in the history of reliable sources. Identifying as anti-establishment media seems to correlate with poor epistemic hygiene.

On the other hand, I wouldn't be surprised if some version of this is true after correcting for this, as your typical European country has an ethnic majority over 80%.

Comment author: CarlShulman 10 April 2013 12:40:09AM 10 points [-]

The U.S. educational system can be better than most other countries' (assuming higher performance is not due to some other factor) and yet have much room for improvement. The U.S. economy has higher GDP per capita than almost all other countries, and yet it keeps growing, and there are many areas where policy is clearly forsaking GDP.

Comment author: buybuydandavis 13 April 2013 10:58:59AM 1 point [-]

Doesn't the US spend a lot more per pupil?

Comment author: CarlShulman 13 April 2013 06:29:22PM *  1 point [-]

The U.S. spends much more per manual laborer, nanny, etc. The wage level is higher, and immigration restrictions prevent wages from equalizing across national borders. You have to ask whether the U.S. has more or better teachers, or textbooks/facilities/amenities, as opposed to paying more for similar or lesser inputs.

Also, spending reflects other factors to some degree, e.g. it is more labor-intensive, and thus expensive, to educate children with learning disabilities or other serious problems.

Comment author: buybuydandavis 13 April 2013 10:50:57PM 1 point [-]

Wage levels much higher than Northern European countries? Really?

More to the point, teacher's pay much higher?

Comment author: CarlShulman 13 April 2013 11:11:35PM 1 point [-]

Here is a chart of teacher salaries as a share of GDP per capita, and here is a tablepercapita) of GDP per capita across countries.

The US spends a lower share of GDP than many other countries, but off a higher GDP base.

Comment author: Desrtopa 10 April 2013 12:18:17AM *  14 points [-]

The picture I have of the US education system is that there are a large number of smart, dedicated, people spending a lot of money trying get the best outcomes they can with the students they have to work with. This is all irreconcilable with the claims the OP makes.

Not so irreconcilable, if you don't suppose that "a lot" means "most."

The current average likelihood of a high school freshman in America making it to graduation is about 78%, and that's the best it's been in quite a while.

At the public high school I went to, it was a pretty big deal if a year passed where someone failed to graduate, and students would ask each other, not if they were planning to go to college, but what college they planned to go to. The only student I ever asked or heard asked that question who said they weren't planning to go to college, went to college. And not a two-year or community college, but a pretty decent state college.

That was a good high school, but it wasn't by any means renowned. With schools like that bringing up the national average, consider the state of the schools dragging down the national average.

Comment author: OrphanWilde 09 April 2013 11:12:04PM *  21 points [-]

I don't know about the rest of the country, but this fails entirely to be surprising [eta: given the region of the country I grew up in, Texas]. My family has seen too much shit go on in schools.

The most egregious case that happened -to us- was a school that put one of my siblings in a fucked-up experiment (paid for by a grant!) without my parents' consent, and indeed told all the children involved not to tell their parents or bad things would happen to them (we grew up being taught very firmly to question authority, so of course my parents found out, and a shitstorm was raised - with nothing ultimately happening. One of several reasons we moved out of that school district.). The experiment involved shit like telling the (extremely young) children to imagine they were in a crashing airplane, and there's nothing they could do, they were going to die, and how did they feel about this?

Some of it seems exaggerated, but the basics - half-assed school grants funding ridiculous shit - ring a little too true to me to outright reject the post. I've seen too many things happen in schools that remain completely unreported on, like prayer in school, to think that the scarcity of information on the internet means anything, as well.

Comment author: buybuydandavis 13 April 2013 10:47:51AM 1 point [-]

The experiment involved shit like telling the (extremely young) children to imagine they were in a crashing airplane, and there's nothing they could do, they were going to die, and how did they feel about this?

The laughter brought tears to my eyes.

Obviously, that's an obscene thing to do to children. Shades of the Milgram Experiment.

we grew up being taught very firmly to question authority

Martian.

Comment author: NancyLebovitz 09 April 2013 06:07:32PM 6 points [-]

I'm torn between thinking that if this is a hoax, the hoaxer should be banned with extreme prejudice, and hoping that there will be another hoax designed to appeal to right-wingers.

Comment author: fubarobfusco 11 April 2013 02:04:14AM 1 point [-]

I initially considered this post pretty credible, but then this thread happened; and I subsequently realized that it was kind of interesting that the post's description of the education system seemed to systematically have something in it to offend pretty much everyone's political views. That struck me as ... odd.

I find myself not having much of an opinion on whether it is true or not; but then, I don't expect to take any particular action based on whether it is true, so I feel pretty safe not caring.

Comment author: [deleted] 10 April 2013 06:35:50PM 1 point [-]

Seconded.

Comment author: mwengler 10 April 2013 12:15:05PM 4 points [-]

I'm torn between thinking that if this is a hoax, the hoaxer should be banned with extreme prejudice, and hoping that there will be another hoax designed to appeal to right-wingers.

I come down strongly on the "hoax" side because I spend a lot of time "reviewing" the emails that my father and other relatives exchange. These are of the sort Obama born in Kenya, Obama dissed dead soldiers and their families, Obama pushing Sharia law, Obama hates Flag pins.

As far as I'm concerned, I have seen 100s of the hoaxes designed to appeal to right wingers. You can see them too: go to snopes.com, search on Obama and False ,stop reading when you get bored.

As to banning, if we really are supposed to be learning rationality here, how does it help to erase all evidence that in large numbers we got tricked? And it didn't even take Omega to do it to us, it was just another Beta like ourselves? If this does turn out to be a hoax designed to appeal to us, it should be taught as something we need to watch out for.

Comment author: IlyaShpitser 09 April 2013 07:48:17PM 7 points [-]

The sign of a good Usenet troll post is that it is a mirror held up to as many groups as possible.

Comment author: Randy_M 09 April 2013 06:43:59PM 8 points [-]

I'm surprised you think the appeal of the OP is confined to left-wingers. The bad guys are all government beaurocrats, the current boogey men of the right and a group championed by the left.

Comment author: NancyLebovitz 09 April 2013 07:24:47PM 2 points [-]

You've got a point. The OP would appeal to both-- I was probably biased by the left-wing appeal being at the end of the post.

Comment author: ThinkOfTheChildren 09 April 2013 06:40:28PM 0 points [-]

That's interesting.

If this were a hoax, it would certainly appeal to right-wingers. In general, the way the school board is debating this issue, the democrats are in favor of teacher recommendations and "helping the poor black kids", whereas the republicans (although, on the school board, they're all teapartiers) are the ones running with the "Data Driven Decisions D^3" slogan.

Comment author: Randy_M 09 April 2013 07:16:43PM 0 points [-]

What specifically is the school board debating? Allow the Principal to keep some minority students in honors classes?

Comment author: Douglas_Knight 09 April 2013 06:36:40PM 0 points [-]

That would be the first 7 paragraphs.

Comment author: jimrandomh 09 April 2013 05:37:24PM 33 points [-]

I believe you're posting this because you want the problems fixed; but in order for that to happen, you need to make it happen. There are a variety of escalation strategies to consider: you might go to your state's secretary of education; you might go to your state's attorney general; you might go to the press. Note that NDAs usually do not hold up when you're making accusations of criminal activity, and some of the accusations you have made are criminal in nature. There are "whistleblower protection" laws, which vary from state to state. Also note that if someone is your lawyer, then you have the absolute iron-clad right to tell them everything ("attorney-client privilege") regardless of what you may have signed. And if what you say is true, then there are probably quite a few groups that would happy to provide you with a lawyer, or lawyers that would work on contingency. In any case, talking to a lawyer should be your next step before taking any major action; it's just a question of which one.

Your career as an auditor that people pay to evaluate themselves is completely doomed and you should sacrifice it before it explodes. Your career as an auditor that people pay to evaluate others has promise.

Also, you should realize that having a scandal ready to release in a controlled fashion gives you some powers, and you should think carefully about what your objectives are. Depending on the location, size, spin and time of release, you can take out any one person of authority, possibly (but not necessarily) as high as the state governor. If you have political savvy, you might maneuver into a position of authority yourself.

(To all the other commenters in this thread: this is one of those cases where you should be providing actionable options to the original poster, not going meta, not expressing outrage, not trying to collect the information to act in his place. Comment as a consequentialist, not as a conversationalist.)

Comment author: [deleted] 10 April 2013 01:52:04PM 1 point [-]

Your career as an auditor that people pay to evaluate themselves

As-yet-uninvented rationalist career niche?

Comment author: jimrandomh 09 April 2013 06:51:41PM 13 points [-]

Pulling a bunch of money out of the system with a lawsuit is not a winning outcome if it leaves the existing corrupt power structure in place. Be warned that for many lawyers, the goal will be money, not improvement. Do not use a lawyer whose goals are different than your own.

You should be collecting evidence. If you are ever alone with an incriminating document, photograph every page. You may later wish to allow these documents a chance to go missing (eg, by making a FOIA request) before you reveal that you've copied them. If you are in a state where it is legal to do so without telling anyone, record every interesting conversation. A recorded statement like "the evaluator's job is to collude with the submitter" is a political instakill if given to the press, and the threat of releasing it is significant leverage.

Pay attention to the specific people involved, especially the ones who are higher up, and the ones who are keeping themselves hidden. Try to figure out who's good, who's evil, who's smart, and who's dumb. Try to predict how each will react to scandal. Assume that by default, the evildoers will successfully deflect blame onto the stupid, unless you have specific incriminating evidence.

Comment author: TimS 11 April 2013 01:22:49AM *  3 points [-]

Pulling a bunch of money out of the system with a lawsuit is not a winning outcome if it leaves the existing corrupt power structure in place. Be warned that for many lawyers, the goal will be money, not improvement. Do not use a lawyer whose goals are different than your own.

This is good advice in general for dealing with the tort portion of the US legal system. But we shouldn't forget the general deterrence effect of money damages.

And in education law in particular, money damages for being a negligent educator are essential impossible to get, so the extreme money grubbers aren't in this market. In special education, money damages simply for having a stupid education plan are unavailable under the most important special education statute.

That said, the rest of this post is excellent advice.

Comment author: TimS 09 April 2013 05:58:06PM 11 points [-]

In particular, if any of the children evaluated are in special education, parents have fairly strong leverage to punish school districts who are doing stuff that no one could think would work. Attorneys and non-attorney advocates in all 50 US states are available to help the parents, who often have no idea what their rights are or that the school district is being foolish.

Separately, the American's with Disabilities Act prohibits retaliation against anyone who acts to protect a disabled individual's rights. Alas, proving this is a difficult matter. You might consider looking at my post on one way to document verbal statements in writing.

And for clarity, disabled in this context is a broader label than cognitively impaired, blind, or deaf. If a child has a medical condition that impairs their ability to learn, they are a "child with a disability" under the Individuals with Disabilities Education Act.


Disclaimer: This is a general statement of the law, not legal advice. Consult your own attorney, because reading this does not create an attorney-client relationship.

Comment author: mwengler 09 April 2013 05:23:44PM 19 points [-]

My top candidates for what is up here are: 1) fabrication as part of a social experiment on how credulous we are 2) fabrication by a sociopath with a very odd idea of self-entertainment 3) incredibly erroneous interpretation of what is going on by a crank

But it is SO full of red flags that I would be surprised if it is not intentional. Call it 66% chance it is intentional hoax.

And it is so far from the mark of a true post that I would be very surprised if it had more than a glancing connection to the truth, call it 95% that it is barely connected to actual facts.

I have kids in California public schools. I have read, over the years, many critiques of public schools and public funding generally. As bad as things are, they are quite obviously nowhere near as bad as this article suggests in the schools my kids have gone to and are now going to. Further, I am quite good friends with a long time teacher, administrator, and union officer in NYC. I by no means share her respect for the union and DO believe documented horror stories of "turkey farms" where truly impossibly bad teachers are stored while being paid rather than following the more expensive process of firing them. I do believe other horror stories. But I can tell you for sure, while things are not amazingly wonderful in California public schools, they are simply not even vaguely close to that bad in many real Calfifornia schools I am exposed to.

So at bare minimum, if there is any truth to the allegations in the original post, the idea that these things are universal, or at least pervasive in American public schools is wrong.

Next argument: many of us reading this board, and even being taken in by this post, went through the American public school system ourselves, and by my standards, (I'm 55) many of you went through quite recently. Many of us, I dare say, were in advanced classes. Does the OP fit even vaguely with what you saw with your own eyes? It is miles from my 40 year old experiences.

Next, there is a thriving critique of publicschools in this country. With the amount of negative attention public education has drawn, is it really plausible that NONE of this critique has discovered the depths of waste and stupidity described as routine by this post? It is not plausible to me.

Next, public spending and public education tends to be a pretty open process. If these are Government grants, there is a crap load of information that has to be public.

Finally, to make such extreme claims with absolutely NO linkage to any source other than the post itself, would require remarkable naivete about how an intelligent audience should perceive claims like this, an innocence which is belied by the beautiful craftsmanship of the post itself. Really, EVERY program discussed needs to be obfuscated? No agency involved can be mentioned?

I googled "black men ipad education grant" hit nothing similar to the OP claimed program.

The real question is how long before the trap is sprung and we are told we were naive to believe this at all and we are really no better than birthers and creationists when the story fits our fears. I think it is better than 50% we will get such a message, but we'll see.

Comment author: ThinkOfTheChildren 25 April 2013 08:02:10AM *  1 point [-]

I have kids in California public schools.

I have never worked in California, nor New York, and cannot speak for your experience.

Next argument: many of us reading this board, and even being taken in by this post, went through the American public school system ourselves, and by my standards, (I'm 55) many of you went through quite recently. Many of us, I dare say, were in advanced classes. Does the OP fit even vaguely with what you saw with your own eyes? It is miles from my 40 year old experiences.

Really? I myself went through the advanced classes. In my "Calculus AB" class, there were 28 whites, 1 hispanic, and 2 blacks. My school was probably around 30% black, 20% hispanic, 50% white. There are two possibilities. Either whites are 5/3 * 28/2 = 23 times more likely than blacks to be prepared for Calculus, or there is some kind of institutionalized racism going on.

Nobody issues grants to help the academically gifted kids who are already doing well. Most grants come as "dropout prevention grants", or are otherwise targeted at students unlikely to end up on Lesswrong. So I would ask you: in your advanced math classes, were minorities represented as a proportion of the school's population? Or was the ratio of the percentage of minorities in your school's population to the percentage of minorities in your advanced classes higher than 1? Perhaps higher than 2? For me it was 23.

The real question is how long before the trap is sprung and we are told we were naive to believe this at all and we are really no better than birthers and creationists when the story fits our fears. I think it is better than 50% we will get such a message, but we'll see.

Eh. I wish citations were easier to find; it's kind of ridiculous, honestly. Just trying to find math placement criteria for any given school system on the internet is impossible, much less a random assortment of school systems such that my location is anonymous.

Comment author: Desrtopa 09 April 2013 11:21:52PM 20 points [-]

The variation in educational standards and practices between districts in America is too large to make generalizing from one's own experience very useful except insofar as it demonstrates that the critiques given in the article cannot be universal.

When I talk to friends who went to decent schools (which is pretty much all of my friends,) their experiences, cynical though they might be about them, don't reflect the sort of scandal the OP describes. When I talk to acquaintances who work as teachers for seriously disadvantaged schools through programs like Teach For America, the general consensus appears to be "No matter how bad you think it is, it's always worse."

Comment author: gwern 09 April 2013 10:01:51PM *  13 points [-]

Next, there is a thriving critique of publicschools in this country. With the amount of negative attention public education has drawn, is it really plausible that NONE of this critique has discovered the depths of waste and stupidity described as routine by this post? It is not plausible to me.

Every scandal was at some point not yet known. Consider an apropos contemporary news event: the Memphis cheating ring, which embraced an entire school district in cheating far worse than merely sustained incompetence and racism. It apparently may have started as early as 1995, and only began coming out in 2009.

Comment author: Nominull 10 April 2013 10:36:56PM 1 point [-]

Wait, systematic cheating is far worse than systematic racism? That seems, uh, non-obvious to me.

Comment author: gjm 09 April 2013 09:29:59PM 1 point [-]

FWIW I estimate 30% chance something of the sort is going on; if so, my guess is that the OP is actually a ... well, I suppose he might use the term "racial realist" ... who wants to show how those lefties on Less Wrong will believe even the most ridiculous claims if they allow them to blame underperformance by "blacks and poors" on systematic mistreatment rather than natural inferiority.

Comment author: [deleted] 09 April 2013 04:22:59PM *  5 points [-]

The specific project I was evaluating had only gotten $800,000 out of the maximum $2m. Its strategy was to purchase the male students iPod Touches, the female students makeovers, manicures, and pedicures at a local beauty parlor, and all students were offered an additional iPod Touch or Makeover, respectively, if they passed the exam at the end of the current year.

Besides everything else, the iPod touch doesn't sound exactly like the kind of thing that already having one makes you more likely to want another. What the heck should I do with a second iPod touch if already have one? (Beside selling it or giving it to my sister, that is.)

Comment author: Document 11 April 2013 02:13:08AM 1 point [-]

One public and one private? One to listen to while you record with the other? One for practical and one for recreational use? Not that either of the uses you downplay aren't also good.

Comment author: [deleted] 11 April 2013 04:40:56PM 1 point [-]

Okay, the second sentence of my comment might be an exaggeration, but I stand by its first sentence.

Not that either of the uses you downplay aren't also good.

Yeah, but I'd kind-of prefer to be given the retail price of a new iPod touch in cash rather than be given a new iPod touch.

Comment author: Emile 09 April 2013 01:52:29PM 0 points [-]

Nice article (as usual, if true). Minor nitpick:

Anyway, this is an important problem that I'm working on, but literally I can only make a difference for my county in my state, and it's clear that the problem is everywhere.

I think you mean "in the US" - things are different in France, or in Finland, or in Japan, or in China. You may see some similar patterns, but I think the whole "black/white" thing is pretty specific to the US; you get complicated relationships between ethnic groups in many countries, but they work differently (look at Belgium, or Ireland, or Canada, or China, or Rwanda, or Egypt, or Singapore - though Australia seems to be one of the rare countries comparable to the US).

I think taking a more international perspective is useful not only to be more accessible to non-US readers, but also to detach the issue from local politics (especially with examples from countries whose politics are quite different), and get a larger sample size in which it's easier to see which theories hold water.

Comment author: wedrifid 09 April 2013 04:58:46PM 3 points [-]

though Australia seems to be one of the rare countries comparable to the US

That doesn't seem right. Australia's "black" group (which isn't labelled 'black') is more closely analogous to Native Americans than African Americans in history and current status in ethnic politics. It would be altogether ironic to equate Australian Aboriginals with African Americans simply because they have similar skin color.

Comment author: Emile 09 April 2013 05:16:53PM 1 point [-]

Can't say I disagree with that, but do you agree that the American black-white situation is closer to the Australian situation (in term of current political and social dynamics, not in terms of history) than it is to the Walloon-Flemish situation, or the Hutu-Tutsi situation, or the Han-Tibetan situation?

Comment author: wedrifid 09 April 2013 05:22:53PM *  1 point [-]

Can't say I disagree with that, but do you agree that the American black-white situation is closer to the Australian situation (in term of current political and social dynamics, not in terms of history) than it is to the Walloon-Flemish situation, or the Hutu-Tutsi situation, or the Han-Tibetan situation?

I'm afraid I can't speak with confidence about any of those other ethnic situations. I'm not particularly familiar with them. Australia and the United States are the two countries in which I have resided and the cultures I am least unfamiliar with. I'll take your word for it that the dynamics are more similar than the others mentioned.

Comment author: [deleted] 09 April 2013 04:19:38PM 5 points [-]

Huh. Even I, who am usually very peeved by that kind of things, automatically took “everywhere” to mean ‘in the US’ in that context without even noticing. (Possibly because the first paragraph explicitly mentioned the US twice, and because of the “county” and “state” earlier in the same sentence.)

Comment author: Kawoomba 09 April 2013 04:35:14PM 7 points [-]

Everywhere is all-quantified concerning the location, so obviously it means on other planets and in other civilizations as well. Elementary, my dear Watson.

Comment author: [deleted] 11 April 2013 04:56:38PM 3 points [-]

(In certain posts, I do have trouble determining whether the writer is talking about the US, the Anglosphere, the western world, the developed world, the whole world, or what.)

Comment author: educationrealist 09 April 2013 01:46:28PM 21 points [-]

Man, I registered just so I could vote and then it turns out there's something called karma.

This post is almost entirely nonsense. I give it "almost" simply because in certain all-URM school districts the corruption level is high. It's within the realm of possibility that "fake grants" to "fake grant programs" that are nothing more than chump change doled out by large employers who can wave the program in front of Jesse Jackson and his ilk--look! We're providing gravy!--so I won't call it an outright lie. But it's certainly not the norm. Did you notice that this guy acts like the education world is comprised solely of blacks and whites? If any element of his story is true, it's because he lives or works in an all black school district that is, indeed, corrupt. Detroit, New Jersey somewhere, or the like. And that's a generous interpretation.

The second half of his post is so risible I'm amazed anyone takes it seriously. We live in a world where, as I write this, federal settlements are forced on schools that suspend or expel minorities at a higher rate, never mind the details, and anyone believes that schools assign classes by race? It's not just wrong. It's an outright LIE. Even in very rich schools that have low income URM students (and I can think of five within 20 miles of my home), the pressure to integrate classes when the kids are unprepared is huge. Principals are at risk for losing AP classes if they don't put enough URMs in them. They face lawsuits if they do use tests to assign kids to advanced classes, much less if they assigned by race. As for the idea that black students do well if the teachers like them there? Please. Teachers have next to no say as to their assignments---it's one area in which principals have a great deal of control.

Every word beginning with "unfortunately" is such a lie I'm astonished anyone would credit it.

Comment author: ThinkOfTheChildren 25 April 2013 07:40:17AM *  1 point [-]

I apologize for the late response.

As for the idea that black students do well if the teachers like them there? Please. Teachers have next to no say as to their assignments---it's one area in which principals have a great deal of control.

I do not know where you come from, but I have personally reviewed the math placement criteria of hundreds of middle schools and high schools. Teacher recommendations are always on the list, whereas I have never seen a school which used "principal recommendations". Wake County, NC's placement criteria: http://www.wcpss.net/policy-files/series/policies/5611-bp.html Alamance County's placement criteria: http://tinyurl.com/d35dtfy I will find more if you'd like me to, but teacher recommendations are plainly listed. In my experience, principals generally back their math teachers when it comes to which students get placed where.

We live in a world where, as I write this, federal settlements are forced on schools that suspend or expel minorities at a higher rate, never mind the details, and anyone believes that schools assign classes by race? It's not just wrong. It's an outright LIE.

The schools do not outright assign math placement based on race; it is slightly more subtle than this. An example would be Wake County, in North Carolina. Wake County used a model called the "effectiveness index". A student is given a score based on: 1) Their previous test scores 2) Their income level (trinary: free lunch, reduced-price lunch, normal) 3) Their race. If two students with exactly equal grades and test scores were evaluated using the effectiveness index, with one student being a poor black, and another being a middle-class white, the former would be given a lower residual score, and therefore would be less likely to be placed into an advanced class. These scores were also used to determine how well a school is doing at teaching. If the poor black student did as well as the white student, the difference between his score and his effectiveness index residual would be larger than the white student's, and so the school would be rewarded for overcoming the "risk factors" of being poor and black and managing to instruct him anyway. Wake county is currently doing away with the effectiveness index, replacing it with EVAAS, a system which takes into account nothing but test scores. Source: http://content.news14.com/pdf/sas_report.pdf

Can you point me to a federal settlement forced on a school that suspends or expels minorities at a higher rate? I ask because in all of the school districts I have worked with, the schools did suspend minorities at a higher rate, and I have yet to see any consequences for this.

Principals are at risk for losing AP classes if they don't put enough URMs in them.

This, as well, I would like to see a citation for.

This post is almost entirely nonsense. I give it "almost" simply because in certain all-URM school districts the corruption level is high. It's within the realm of possibility that "fake grants" to "fake grant programs" that are nothing more than chump change doled out by large employers who can wave the program in front of Jesse Jackson and his ilk--look! We're providing gravy!--so I won't call it an outright lie. But it's certainly not the norm. Did you notice that this guy acts like the education world is comprised solely of blacks and whites? If any element of his story is true, it's because he lives or works in an all black school district that is, indeed, corrupt. Detroit, New Jersey somewhere, or the like. And that's a generous interpretation.

The school districts I have worked with have varied from being 90% black to 3% black. You are right in that I should have said "minority" rather than "black", for hispanics, native americans, and other minorities are at a similar disadvantage. However, I've seen enough districts in enough states that I, at least, believe the traits I ascribed to the education system to be nearly universal.

Comment author: Decius 11 April 2013 04:56:22AM 2 points [-]

Do you have citeable evidence that principals are facing lawsuits for using 'teacher recommendations' to either assign students to limited slots, or to discourage students from competing for those slots (e.g. tell students that a teacher recommendation is required, knowing that one group of students will see it as more of a bar to entry than another, resulting in a smaller proportion of that group even competing for the slot).

Because either of those actions are indistinguishable from using race as a factor in determining access to classes.

Comment author: Randy_M 09 April 2013 03:26:29PM 9 points [-]

Upvoted because I'd like to see the OP address your questions.

Comment author: Raemon 09 April 2013 03:03:52PM *  9 points [-]

I actually think there's a decent chance this story is a hoax, but not because it is remotely implausible. It sounds exactly like everything I've heard about the NYC school system.

Comment author: Emile 09 April 2013 02:59:09PM 23 points [-]

I upvoted this comment, because I'm interested in hearing a dissenting view on this, but ... I find this to be pretty poor dissent.

You should tone down your accusations, and especially make them more precise - on the face of it, I'm not sure to what extent the things that you're saying (like "the pressure to integrate classes when the kids are unprepared is huge") actually contradict the OP, as opposed to merely being evidence that supports a different interpretation (and you'll find arguments for both sides on any disagreement).

Mostly, from my French point of view, I'm seeing American politics cloud up issues here, and I would much rather see a dispassionate discussion of the facts rather than flinging accusations back and forth. Too much "THIS IS A LIE AND YOU ARE ALL IDIOTS", not enough "this particular specific statement appears to be false, and here is why".

Comment author: [deleted] 10 April 2013 02:38:39AM 5 points [-]

Likewise upvoted, likewise would prefer higher-quality criticism.

Given how incredibly important education is and how few citations there are here (at the moment, from both 'sides'), forgetting to actually think about something for five minutes before updating your beliefs would be a very bad idea.

Comment author: Viliam_Bur 09 April 2013 01:59:05PM *  14 points [-]

Is it possible that different parts of USA have different situation, because of a different state, different county, or just depending on whether the parents in the specific school are politically savvy, know their rights and fight for them?

Sometimes the official rules are the same for everyone, and yet what actually happens, depends more on the local culture. Maybe the lawsuits get big media attention, but in reality they happen rarely and require a lot of effort on parents' side (or a coincidence that some political group decides to push this cause), so most parents don't even try.

Comment author: Desrtopa 09 April 2013 02:07:30PM 10 points [-]

Is it possible that different parts of USA have different situation, because of a different state, different county, or just depending on whether the parents in the specific school are politically savvy, know their rights and fight for them?

In a country where some school districts have higher college acceptance rates than others have high school graduation rates, I would say this is a near-certainty.

Comment author: NancyLebovitz 09 April 2013 11:50:08AM 3 points [-]

I voted credible/outsider so that I could see the poll results. I'd have gone for plausible/outsider or preferably "no strong opinion, but I want to see the results" if either had been available.

Comment author: [deleted] 09 April 2013 04:12:51PM 0 points [-]

I was in the same situation as you, and I flipped a coin between “exaggerated” and “credible”.

Comment author: Viliam_Bur 09 April 2013 11:03:57AM 6 points [-]

Some people have expressed some doubts about this story, and because it is anonymous, we can't verify it directly. I would like to use this opportunity to explore our models of the school system, and especially the difference between the models of insiders and outsiders.

This poll asks a pair of questions. The first question is about how the story fits your model of educational system. The second question is whether you are an "school system insider". That means whether you ever had a full-time job or a part-time job related to the school system; whether you were a teacher or a director of a school, an employee of a Department of Education, a school inspector, or an employee of a company working for education as much as the author of this article. (It is not enough to be a student, a janitor in a school, or an employee of a company which sold one product to schools but sells most of its products elsewhere.)

Submitting...

Comment author: CronoDAS 10 April 2013 10:19:03PM 3 points [-]

My thoughts: having seen lots of people trying to write things and failing miserably - my father the professor repeatedly finds that large numbers of students don't follow directions when writing lab reports - I'm not surprised by a claim that people would do stupid things when writing grant applications and then not understand what the problem is when it was pointed out to them.

Comment author: mwengler 09 April 2013 05:26:11PM *  1 point [-]

I voted earlier then came back to this post to see how it was going. How do I view the results? I tried voting again, but it won't let me, and I'm sorry, I can't guess how to view the poll results anymore.

EDIT: I figured it out, on the off chance anybody else wonders, hit the chain-link looking icon at the bottom of the poll to go to a page where you see results (at least if you have voted). That link points to: http://lesswrong.com/r/discussion/lw/h5z/problems_in_education/8qa5

Comment author: wedrifid 09 April 2013 05:11:35PM 1 point [-]

That means whether you ever had a full-time job or a part-time job related to the school system; whether you were a teacher or a director of a school, an employee of a Department of Education, a school inspector, or an employee of a company working for education as much as the author of this article

I'm a school system insider by this standard. This makes the survey rather broken because I know very little about schools in your country. I recommend correcting your language.

Comment author: prase 09 April 2013 11:34:47PM 1 point [-]

I know very little about schools in your country

Whose country? Viliam_Bur's country is most probably not the same country as the OP author's.

Comment author: wedrifid 10 April 2013 02:03:40AM 1 point [-]

Whose country? Viliam_Bur's country is most probably not the same country as the OP author's.

In that case the survey makes even less sense to me.

Comment author: Viliam_Bur 10 April 2013 08:01:40AM 4 points [-]

The educational systems seems to me similar enough in different countries. When I read stories of teachers in Britain or USA, of course there are differences, but the similarities are also obvious. Some of the stories could have as well happened in my class.

The biggest differences are: How the school is organized. (Are teachers in a union? Who appoints the director? Is there a management layer between the director and the teachers?) Which minorities underperform, and what political consequences does it have. (Black students? Romani students? Are there ethnical quotas? Affirmative action? How often are teachers accused of racism; what is the typical reason and typical consequences?)

Another significant difference is the level of violence at school -- but this varies also within the country, and changes during time. (How often and how severly do students attack each other? Do students attack teachers? Do parents attack teachers? What are the consequences for the agressor?)

And here are the similarities: Bureaucracy. Decisions made by people who don't have a clue, and often have zero educational experience. Supervision and assessment according to unintelligible or actively harmful criteria. Pseudoscience, and aversion to measuring outcomes. (Tests are bad. Teachers shouldn't explain, but entertain. If any recommended technique doesn't work, it is always the teacher's fault, never a problem with the technique. Knowledge is a lost purpose, the true goal of the school system is to make students happy.) People making big money selling pseudoscience to schools. Random minor changes in school system to make voters see that politicians care about their children. Textbooks containing nonsense. Parents treating teachers as babysitters.

Comment author: wedrifid 10 April 2013 08:32:42AM *  2 points [-]

And here are the similarities: Bureaucracy. Decisions made by people who don't have a clue, and often have zero educational experience. Supervision and assessment according to unintelligible or actively harmful criteria. Pseudoscience, and aversion to measuring outcomes. (Tests are bad. Teachers shouldn't explain, but entertain. If any recommended technique doesn't work, it is always the teacher's fault, never a problem with the technique. Knowledge is a lost purpose, the true goal of the school system is to make students happy.) People making big money selling pseudoscience to schools. Random minor changes in school system to make voters see that politicians care about their children. Textbooks containing nonsense. Parents treating teachers as babysitters.

I mostly agree (strongly) with this. However the "Tests are bad" part in particular doesn't seem to be completely general. More testing and measurement seems to be the direction things have been going here.

Comment author: Decius 11 April 2013 03:47:50AM 0 points [-]

I read "Tests are bad" as "The tests do not accurately measure".

Comment author: wedrifid 10 April 2013 08:29:21AM 0 points [-]

The educational systems seems to me similar enough in different countries.

Ok, I've answered the survey adopting this assumption. I chose "Very difficult to believe, school system insider". But note that I would also have found the prom segregation difficult to believe if not for the somewhat credible sources so discount the results as appropriate.

Comment author: Randy_M 09 April 2013 03:34:21PM 2 points [-]

I have taught in a school and subbed in others. I had very little say about who could come into my classroom, though I was only a first year teacher. (Quit after 1 year). I don't have any reason to believe education beaurocracy is generally bright or honest, but can't say I've seen that kind of thing in the grant process, possibly just because I wasn't involved in it.

Comment author: Tenoke 09 April 2013 12:24:37PM 7 points [-]

You move from 'very difficult to believe' to 'seems like a very exaggerated version of true events,' which is almost the same thing but from that you directly jump to 'This story is credible' without any middle ground. I am sorry but this seems like a really poor questionnaire design.

Comment author: Viliam_Bur 09 April 2013 01:45:46PM *  2 points [-]

Once the poll is made, the answers cannot be edited. The difference was supposed to mean approximately a) I don't believe this could ever happen, b) I believe it can happen exceptionally, but not all the time as the author claims, c) I believe this can be the way system works.

In other words, the second option is like: "I believe that with so many grant proposals, once in a while a crazy thing passes unnoticed; but I don't believe that it happens all the time, not even half of the time -- you have probably seen one or two bad cases, and now you exaggerate to make your case more appealing".

Comment author: BT_Uytya 15 April 2013 08:04:00PM *  -1 points [-]

So, you are asking me to condition on belief that the author of this post isn't a troll who crafted this story in such a way to be especially appealing to the LW community. Am I right? If I am, you probably should edit your original post to make your idea more clear.

Comment author: fubarobfusco 09 April 2013 05:56:14PM -1 points [-]

b) I believe it can happen exceptionally, but not all the time as the author claims

That sounds very different from "This story seems like a very exaggerated version of true events". One is about frequency (how often do things in the author's intended reference class occur?) while the other is about severity (how bad are the events that actually happened to the author?)

Comment author: Raemon 09 April 2013 02:50:47PM *  3 points [-]

I found myself wanting to say "I think this sort of grant proposal thing happens maybe 25% of the time, but not all the time, the way the post implies."

I also wished there was some kind of gradation for "school insider/outsider". I'm an outsider, but I talk a lot with a friend who teaches full time. I showed this article to her and she said "yes, yes, yes. This is basically how it is."

I actually DO still assign substantial probability to this being a hoax, despite it matching my understanding - we know the this is sockpuppet account, created ostensibly for NDA anonymity. But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.

Comment author: ChristianKl 11 April 2013 10:15:47PM 1 point [-]

But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.

Which people? Stylometry might us allow to work out whether someone's writting style matches this post. Anonymity is hard.

Comment author: gwern 11 April 2013 11:08:09PM *  4 points [-]

Yes, we could probably de-anonymize OP; I have some passing familiarity with stylometrics, so I considered trying that myself. I decided not to because so far no one has produced any smoking guns that this is false (school districts vary massively in quality across the USA & I have already mentioned an existing and far more shocking instance of school failure/corruption), if it is true then I approve of whistleblowing and have no interest in attacking the OP*, de-anonymizing probably would not set a good precedent, and if it were false - well, I do not disapprove of red team tests of LW (and would be hypocritical to disapprove) and so far this seems to be limited to LW.

* Similarly, it's been suggested to me by a few people that it would be an interesting project to try to de-anonymize Satoshi Nakamoto or La Griffe du Lion. I am not sure I could, and even if I could, I would choose not to since I either approve of their work or find their material interesting.

If this post ever looked like it was both false and not a limited-scale test, like it was something else (an entrapment of an off-site person? an attempt to discredit LW entirely with a Sokal-style attack?), then I might change my mind. But so far, that does not seem to be the case since I see nothing indicating this post has been picked up by the Drudge Report, Hacker News, Fox News, Breitbart, etc.

Comment author: Emile 09 April 2013 04:09:41PM *  0 points [-]

Yeah, I also generally consider that posts like this have around 5% chances of being a hoax followed by a gloating "they swallowed it" update (here or somewhere else), though this post doesn't have any huge red flags (there doesn't seem to be any huge gloating potential, I mean it's basically just venting).

Comment author: Viliam_Bur 09 April 2013 03:26:28PM 3 points [-]

But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.

Spreading false data as a "test of rationality" would be actively harmful. But I can imagine people misunderstanding that.

Rationality is a method of working with the data you have. You should update on evidence correctly, instead of updating incorrectly. You should be able to recognize that this specific piece of evidence contradicts the model based on all other evidence, which makes this specific piece of evidence suspicious. But also should estimate your degree of certainty in a given model.

It is proper to say "I defy the data" when one's model is based on a lot of reliable evidence. Saying it more often would be overconfidence, not rationality.

Comment author: Raemon 09 April 2013 03:29:22PM 1 point [-]

I was assuming that if it were a hoax, they'd let people know in another few days with a gloating update.

Comment author: Desrtopa 09 April 2013 05:38:16PM 2 points [-]

I agree that this would be the most likely course of action if the essay is a hoax, but I think it would still risk being harmful overall, since retractions generally don't result in an appropriate corresponding decrease in confidence in the material that was originally presented. I'd expect Less Wrong members in general to be better at reducing their confidence in a retracted claim than most people, but better is not necessarily good enough.

Comment author: bentarm 09 April 2013 10:39:04AM *  2 points [-]

The specific project I was evaluating had only gotten $800,000 out of the maximum $2m. Its strategy was to purchase the male students iPod Touches, the female students makeovers, manicures, and pedicures at a local beauty parlor, and all students were offered an additional iPod Touch or Makeover, respectively, if they passed the exam at the end of the current year.... only 25% (14/56) of the students targeted by the program had failed the reading exam in the first place.

$800,000/56 students = $14,000 per student. Those are some expensive iPod touches!

Comment author: falenas108 09 April 2013 12:59:46PM 5 points [-]

See this part of the post:

I described in rigorous detail everything the man had done wrong, put in a strong recommendation to not award him grant money in the future, and suggested that some sort of corruption investigation be conducted to see if he had committed any crimes (23 iPods + 23 Makeovers does not total to $800,000, after all).

Comment author: somervta 11 April 2013 05:41:52AM 2 points [-]

Interestingly, 23+23 != 56

Comment author: buybuydandavis 09 April 2013 07:39:49AM 10 points [-]

make sure the criteria for math placement is based on achievement data

Make sure you collect achievement data. Bayesian calculations are fine and dandy, but I'd declare victory if they collected the data and let people see it.

Comment author: Kawoomba 09 April 2013 06:24:18AM 6 points [-]

Even an effective program that actually, verifiably works would have its problems: It would (as it stands) target standardized test scores of some sort, which then automatically lose some of their previous reliability as surrogate parameters. That effect has a name, which eludes me, can anyone supply it? (Loss of reliability when a variable is targetted directly and thus becomes subject to manipulations.)

Comment author: JGWeissman 09 April 2013 06:29:55AM 15 points [-]

That effect has a name, which eludes me, can anyone supply it? (Loss of reliability when a variable is targetted directly and thus becomes subject to manipulations.)

You are thinking of Goodhart's Law.

Comment author: Kawoomba 09 April 2013 06:33:08AM 1 point [-]

Cheers!

Comment author: James_Miller 09 April 2013 05:11:35AM *  11 points [-]

If you could prove this stuff you could become a hero to a lot of people.

Edit: I now think this post is probably a hoax. As EY writes "Your strength as a rationalist is your ability to be more confused by fiction than by reality."

Comment author: ThinkOfTheChildren 25 April 2013 08:05:30AM 0 points [-]

Please look through the comments where I have replied to criticisms; I have tried to find relevant citations.

Comment author: Decius 11 April 2013 04:48:02AM *  0 points [-]

Your confusion is not edit: strong evidence.

Comment author: James_Miller 11 April 2013 01:49:29PM 1 point [-]

My not understanding how something could have happened is evidence for it not having happened.

Comment author: Decius 11 April 2013 08:26:18PM 0 points [-]

If you understand how something could happen, how strong is that evidence that it happened?

Comment author: James_Miller 11 April 2013 09:24:32PM 1 point [-]

Some. I tell you that I mixed chemicals X and Y to make Z. You are initially 99% confident that X and Y don't make Z and so think I'm probably lying. Then you read that something in the air in the state in which I live cause X and Y to create Z. Won't your estimate of my having told the truth go up?

Comment author: Decius 12 April 2013 03:57:42AM *  0 points [-]

I would be even more certain that X and Y don't make Z, and that you were mistaken. I would believe that you mixed X and Y and a and made Z, where a is the characteristic in the air which I was unaware of prior.

There is the very weak effect that I am more likely to understand how something happens if it is possible than if it is impossible, and things which are possible are more likely to happen than things that are impossible. Therefore I am more likely to be confused in general by things that didn't happen than by things that did- but not more likely to be confused by things that didn't happen but are possible than by things which did happen and are possible.

My biggest doubt comes from the fact that there should be trivial to reference to at least one grant which is literally as bad as the example given; this could be done without compromising anonymity, given that FOIA requests can originate from any source. Because the details of one grant as bad as the iPod/makeover grant would be fairly weak evidence that almost all grants are horrible, the absence of any in my research is fairly strong evidence that not almost all grants in the nation are horrible.

Which is not to say that there couldn't be districts where horrible grants are the norm, or clearly fraudulent grants.

Finally, the biggest inconsistencies I found in the original post were
A) That an apparently literate and intelligent person though that a state-standardized test was an accurate measure of literacy,
B) That a school with a test results problem would still have a 75% pass rate among lowest-class students, and
C) That he never mentions being told by his supervisor that his job was specifically not to evaluate if the goals were appropriate (that being the job of the department issuing the grant, prior to issuing the grant; if they said that giving students iPods was the goal of the grant, it was sufficient), but only to evaluate whether the goals written into the grant were met. Instead the author describes the conversation as being one of 'colluding'
D) (weak) By law, in every state, schools do not give out lists of students who are on free/reduced meal programs nor of students who failed tests. It is possible that the administrator in question simply violated the law; that the data was provided in a technically non-personally-identifiable manner, such as student ID numbers that qualified for meal programs; or some combination of the two.

Comment author: James_Miller 12 April 2013 03:33:06PM *  2 points [-]

I think that (A) is true because of Spearman's g. The evidence for g is overwhelming.

Comment author: Decius 12 April 2013 04:31:08PM 0 points [-]

Did you just claim that g correlates well enough with two specific s to measure one s (ability to determine the answer expected by the writer of a test) and provide results for a different one (literacy) in the general sense?

Because my position is that most standardized tests measure a combination of the intended subject and the ability of the taker to figure out the test writer; part of this comes from my observed ability to consistently outperform people with an equal or better knowledge of the subject being tested on many different tests, and most of it comes from my ability to explicitly recognize the test author's thought patterns in determining which options were available in multiple choice tests and figure the correct answer to a large number of their questions by looking only at the possible answers.

Comment author: James_Miller 12 April 2013 04:44:45PM *  2 points [-]

Yes. I did a huge amount of reading on IQ to write this.

Comment author: Decius 13 April 2013 01:07:06AM 0 points [-]

Great. We can use test any skill to accurately measure any other skill now, right? It is impossible for someone to have great math skills and poor English literacy, because of general intelligence?

That requires more extraordinary evidence for me to believe than I have seen. What is the most extraordinary citable data that you encountered in your research indicating that specific skills are in general interchangeable?

Comment author: Bakkot 09 April 2013 05:10:03AM 9 points [-]

I'd be very interested in a citation on

the evidence shows that teacher recommendations have zero correlation with aptitude in a field

Comment author: Kawoomba 09 April 2013 06:49:13AM 15 points [-]

Since OP's clearly a bit venting, I'd give him some charitable leeway and interpret 'zero' as 'so small as to not be relevant'.

Comment author: Kaj_Sotala 09 April 2013 06:35:42AM *  3 points [-]

Seconded. A relatively low correlation I could believe, but none? As a friend pointed out, this would imply that if there's a math prodigy in the class, the teacher would be just as likely to recommend advanced classes as they would be to recommend the student needing extra help with basic stuff? I could accept prodigies slacking off due to boredom and therefore sometimes getting mistaken for people with bad skills, but 50-50?

Comment author: Decius 11 April 2013 04:10:24AM 7 points [-]

Would you believe that many teachers use 'effort' as an explicit factor in assigning grades? As in, someone who understands the material without putting forth visible effort is assigned a lower grade than someone who visibly struggles to have the same level of understanding.

Comment author: Viliam_Bur 11 April 2013 07:19:49AM 4 points [-]

Would you believe that many teachers use 'effort' as an explicit factor in assigning grades?

They are officially required to (in Slovakia). But it is just one on many confusing, sometimes mutually contradicting, mostly applause-lights criteria.

(From my memory: The grades have to reflect knowledge, they have to reflect effort, they have to be motivating, and they have to respect human rights, whatever that means. And a dozen other conditions.)

As in, someone who understands the material without putting forth visible effort is assigned a lower grade than someone who visibly struggles to have the same level of understanding.

Yes, this is explicitly required by and explicitly forbidden by the rules. Welcome in the world of educational system!

Comment author: Decius 11 April 2013 08:23:50PM 2 points [-]

One more data point; was it a politician with no educator qualifications that wrote the requirements?

Comment author: Viliam_Bur 12 April 2013 07:43:16AM 2 points [-]

I don't know, but my guess would be that a group of bureaucrats with zero educational experience from the Department of Education prepared the document, and some minister just signed it, because it seemed okay (it contained all the applause lights).

Comment author: Decius 11 April 2013 04:05:14AM *  1 point [-]

See also: Pygmalion in the Classroom.

Simply put, when teachers expect students to do well and show intellectual growth, they do; when teachers do not have such expectations, performance and growth are not so encouraged and may in fact be discouraged in a variety of ways

It's entirely reasonable that teacher's ratings of children's academic abilities &tc cause future achievement.

Comment author: gwern 11 April 2013 04:50:37AM 3 points [-]

It's entirely reasonable that teacher's ratings of children's academic abilities &tc cause future achievement.

No, it's not. Did you read Carl's comment in this same thread?

Comment author: CarlShulman 11 April 2013 10:10:20PM *  0 points [-]

That's going further than I did. It's a reasonable prior, and the evidence is at least consistent with weak effects.

Comment author: gwern 11 April 2013 10:54:15PM 1 point [-]

Eh. Decius was clearly thinking of, and still is thinking of, substantial and longlasting effects rather than the almost trivially small disappearing effects confirmed by the followups and meta-analyses. That is completely unreasonable a view to hold after reading that review, and I would suggest that even that small nonzero effect is dubious since it seems that few to none of the studies fully accounted for the accuracy issue and there is obviously publication bias at play.

Comment author: Decius 11 April 2013 04:58:46AM *  2 points [-]

It's been demonstrated by controlled research that students who have teachers who expect them to perform better than their peers do, even when the expectations of the teachers are not founded on fact.

In the Oak School experiment discussed in this book teachers were led to believe that certain students, selected at random, were likely to be showing signs of a spurt in intellectual growth and development. The results were startling. At the end of the year, the students of whom the teaches had these expectations showed significantly greater gains in intellectual growth than did those in the control group.

Comment author: CarlShulman 11 April 2013 05:46:19AM *  6 points [-]

The Jussim et al review of that literature is worth reading. Expectations do seem to have causal impact, but the effect is usually small relative to measures of past performance and ability, and teacher expectations tend to reflect past performance more.

The review covers some serious challenges to the effect sizes claimed by Rosenthal and coauthors, such as effect sizes declining with sample size and publication bias. Or, regarding the original Pygmalion/Oak School experiment:

Snow (1995) also pointed out that the intelligence test used in Pygmalion was only normed for scores between 60 and 160. If one excluded all scores outside this range, the expectancy effect disappeared. Moreover, there were five "bloomers" with wild IQ score gains: 17-110, 18-122, 133-202, 111-208, and 113-211. If one simply excluded these five bizarre gains, the difference between the bloomers and the controls evaporated.

As an aside, Rosenthal pioneered meta-analysis in psychology because the effect only replicated a third of the time in the published literature (despite the presence of publication bias and QRPs). In doing so he promulgated a test for publication bias which implicitly assumed the absence of any publication bias, and so almost always output the conclusion that no publication bias was present. These methods were eagerly adopted by the parapsychology community, as the same methodology that appeared to show strong expectancy effects also appeared to show ESP in the ganzfeld psychic experiment, as Rosenthal (1986) agreed.

Since I think that the ESP literature reflects the scale of apparent effect that can be shown in the absence of a real effect, purely through publication bias, experimenter bias, optional stopping, and other questionable research practices, this makes me suspicious of the stronger claims about expectation effects.

Comment author: Decius 11 April 2013 06:06:03AM 4 points [-]

I don't think the sample of experiments reviewed is large enough to evaluate sample size versus effect size; throw out the outliers and there's nothing left.

I'm now heavily concerned about the validity of the IQ test used; however, that's more due to the 8 point increase in the control group, when no increase is expected. I'll have to dig further, exclude any of the controls with out-of-band scores and redo the math.

One result of the meta-analysis, however, is that experimentally-induced changes to teacher expectation have a small casual effect on student performance; another result is that non-induced teacher expectations correlate well with performance in the same year, and less well with long term performance. I would rephrase that as 'Teacher expectations of student performance in their class tend to be accurate, but correlate poorly with student performance in other classes.'

In any case, thanks for the link. I'm going to have to spend some time determining how much I should change my mind with this new evidence, but my gut feeling is that the objectively worst possible data (my own experience with performing well when expected to perform well, and performing poorly when expected to perform poorly), will continue to dominate my personal opinion on the matter.

Comment author: gwern 11 April 2013 09:13:54PM *  2 points [-]

I don't think the sample of experiments reviewed is large enough to evaluate sample size versus effect size; throw out the outliers and there's nothing left.

The first Rosenthal meta-analysis used 345 studies. That is pretty big. And the individual studies listed in table 17.1 have large n, ranging from 79 to 5000+.

I'm now heavily concerned about the validity of the IQ test used; however, that's more due to the 8 point increase in the control group, when no increase is expected.

No, that's not a problem that should concern you. Children IQ scores are less stable than older people's scores, test-retest effects will give you a number of IQ points (that's why one uses controls), and children are constantly growing.

What should concern you is that the researchers involved were willing to pass on and champion a result driven solely by obviously impossible nonsensical meaningless data. A kid going from 18 IQ to 122? or 113 to 211? This can't even be explained by incompetence in failing to exclude scores from kids refusing to cooperate, because tests in general (much less the specific test they used!) are never normed from 18 to 211. (How do you get a sample big enough to norm as high as 7.4 standard deviations?)

Worrying about the control's gains and not the actual data is like reading a physics paper reporting that they measured the speed of several neutrinos at 50 hogsheads per milifortnight, and saying 'Hm, yes, but are they sure they properly corrected for GPS clock skew and did accurately record the flight time of their control photons?"

Comment author: Decius 12 April 2013 04:06:20AM 0 points [-]

Unstable IQ scores should provide a net zero; an average increase of half a standard deviation across the entire population already means that the norms are fucked.

Therefore, the IQ test used simply wasn't properly normed; if we assume that it was equally improperly normed for all students in the study, we still see an increase of 4 points based on teachers being told to expect more. Whether an increase of 4 points is statistically significant on that (improperly normed) test is a new question.

Comment author: CarlShulman 11 April 2013 06:17:52AM *  6 points [-]

I'm going to have to spend some time determining how much I should change my mind with this new evidence, but my gut feeling is that the objectively worst possible data (my own experience with performing well when expected to perform well, and performing poorly when expected to perform poorly), will continue to dominate my personal opinion on the matter.

Upvoted for candor.

Comment author: DanArmak 09 April 2013 08:27:32PM 1 point [-]

I wonder if the teachers making the predictions were the same ones who then taught the students, and examined them to determine the outcome.

Comment author: CarlShulman 09 April 2013 08:38:08PM *  6 points [-]

Here's a review of the literature on teacher self-fulfilling prophecies from Lee Jussim, who is skeptical but finds that they occur and are of nontrivial magnitude, moreso for grades vs standardized tests, although they dissipate quickly and teacher judgments are more driven by accuracy than stereotypes in the aggregate.

Comment author: buybuydandavis 09 April 2013 04:50:28AM 1 point [-]

Good data on the disparate racial outcomes for some advanced math teachers. What's the relative prevalence of the bigoted? Is this across other subjects as well? What region of the country are we talking about?

There were math teachers who absolutely refused to allow blacks/poors in their classroom, or worse, treated them in such a way as to cause them to fail, thus confirming their worldview.

They can't be fired/fined/reprimanded, or is there no will to do it?

Comment author: Nornagest 09 April 2013 05:45:26AM *  4 points [-]

They can't be fired/fined/reprimanded, or is there no will to do it?

As I understand it, it's exceptionally hard to fire teachers within the American school system -- it takes evidence of sexual misconduct or something similarly precipitous, and it's expensive, time-consuming, and legally hazardous. Even those charges aren't a sure bet. A teacher of mine in high school was suspended on sexual harassment charges -- well-founded ones from what I heard, although I have no direct knowledge -- leading to a lengthy punitive process that involved, among other things, investigators taking students out of their classes and interrogating them about the allegations. He was back in his classroom before the year was out.

Needless to say, ordinary incompetence won't do it. I don't think implicit racism would either, as long as it stayed implicit -- wearing a KKK hood into the classroom would probably be beyond the pale. Probably.

Comment author: buybuydandavis 09 April 2013 06:38:58AM 3 points [-]

I understand that it next to impossible to fire teachers, unless you hit on extreme hot button issues.

Sex with students is number 1. But I'd expect the long knives to come out for racism/sexism/homophobia as well, at least in some jurisdictions. Likely not in others. That's why I was asking about what region of the country we're talking about.

Comment author: Nornagest 09 April 2013 06:52:17AM *  1 point [-]

In all but the most liberal districts, and maybe even then depending on how cynical you are, I think I'd expect any of that to get a pass as long as plausible deniability existed. Unfortunately, that's plausible deniability from the standpoint of parents and administrators who generally aren't statistically literate nor inclined to take student impressions all that seriously, and that leaves quite a bit of leeway as long as the teacher in question is bright enough to couch their objections in the right terms.

You know and I know that if the bell curve on expected achievement is shaped such that 30% of the student population from some minority group should be admitted to an advanced math class, and 0% actually is, then after a couple of years that's as good as admitting racial prejudice. But I think that'd be a much harder sell to a review board, especially one that doesn't want to incur the wrath of the teachers' union or any further investigative costs.

Comment author: Decius 11 April 2013 03:40:19AM 2 points [-]

If you have 20 teachers who are fair, it would not be surprising for a statistical analysis to show that one of them is 95% likely to be unfair.

Comment author: Nornagest 11 April 2013 03:50:41AM 1 point [-]

Quite. If I was in a position to be firing people, though, I'd be shooting for considerably more than 95% confidence -- a level of confidence that should be achievable with the kind of data sets that the OP was talking about.

Comment author: Decius 11 April 2013 05:43:34AM 0 points [-]

What proportion of false positives would you shoot for? The current status quo seems to be towards 0% false positive, and it appears that some areas might actually reach that goal.

Comment author: Nornagest 11 April 2013 04:56:56PM *  1 point [-]

Well, that's a fairly complicated ethical question, isn't it? The general answer is "the point at which additional effort to reduce the number of false positives does more damage than it'd prevent"; I don't have the data to say exactly where that point is, but it's almost certainly higher than zero or epsilon.

Comment author: Decius 11 April 2013 08:17:27PM 1 point [-]

So you would fire good teachers at random if you could also fire some disproportionate amount of bad teachers? That's strictly rationally better if we care only about student outcomes, but is worse if you care only about fairness to teachers.

Comment author: Nornagest 12 April 2013 07:32:45PM *  6 points [-]

The school system (ostensibly) exists to educate students, not to provide teachers with jobs. Fairness to teachers matters insofar as job security prospects affect the baseline quality of teachers; being able to get rid of the worst ones isn't necessarily a good thing if you bring down the average in the process. But it's not ultimately what we're trying to maximize.

That's my true objection, I think. But there's a couple others you could raise if you prefer a more moderate approach. Firstly, if we're looking to factor in teachers' outcomes it doesn't make much sense to use a binary fair/unfair criterion; it'd make more sense to work out the consequential weight of the lost teaching jobs at an n% false positive rate and do the same for the problems caused by the incompetent or abusive teachers that end up remaining in the system, and calibrate according to where the curves cross.

Finally, a negligible false positive rate is really a rather unusual thing to be aiming for, and needs to be justified as such. Almost no other jobs carry that kind of security, of course, but we don't even aim that low in the criminal justice system; I'd think that if "beyond a reasonable doubt" is good enough for a murder trial with life imprisonment on the line, it ought to be good enough when a single job is at stake. The tenure system in higher education is supposed to insulate professors from political fashions, but that shouldn't matter as much in primary or secondary.

Comment author: buybuydandavis 09 April 2013 07:12:50AM 2 points [-]

I'd think that normally the problem is you don't have the data, and just have anecdotes.

Particularly for admittance to class, I think you've got a problem. Generally disparate selection outcomes are treated as prima facie evidence of prejudice. Something as well controlled as this - when I picture the scenario, it's either the guy in trouble, or everyone circling the wagons and chanting the tribal truth because they don't want to open the door to routine measurement of what they do.

If a teacher is incompetent and can't teach anyone he won't get fired, but if he only can't teach blacks? Maybe they'll circle the wagons there too. I could see it going either way, but relatively more likely trouble for the teacher compared to most non sexual transgressions.

Comment author: private_messaging 09 April 2013 04:35:09AM 6 points [-]

If you are a decision maker in education in your area, please, please, please look into the various Bayesian predictive models used for math placement;

Bayesian methods still can (and in practice, will) use race and the like as evidence, meaning, if you're black you need higher test scores and grades to qualify - they just don't entirely stone-wall you from qualifying, which is a step forward I guess.

The fair approach is to have an entrance exam for better math classes, blind to the race.

Comment author: Houshalter 13 April 2013 05:41:22PM 1 point [-]

I'm confused as to why race would matter if you already have the grades and test scores information. Race might be helpful in predicting what their previous grades and test scores were, but I doubt it would improve the accuracy much over a model that excluded it.

And if it really was true that race matters in who will benefit the most from the program, then so be it. Why would you not want to help those that benefited the most first, regardless what information was used to predict it?

Comment author: RolfAndreassen 09 April 2013 05:16:15PM 3 points [-]

if you're black you need higher test scores and grades to qualify

This seems to me mistaken. The reason race can be used as a proxy in the first place is that there is some correlation with performance on the standard tests. If you use the standard tests, then that entirely screens out all the race information; there is no additional information in race that you didn't get from the test. This is similar to checking whether the plane flies: All information from authority and from theory is screened out by the experiment.

More generally: If A is a proxy for B, and you use B, then trying to use A in addition is double-counting.

Now, perhaps you are arguing that race is a proxy for test scores and something else, and you can still extract the something-else? If so this should be made explicit.

Comment author: private_messaging 09 April 2013 07:18:32PM *  4 points [-]

Consider a test followed by a re-test (which we are trying to predict). To calculate expected score on the re-test you need to apply regression to the mean. For a population where you measured lower mean or (in high range) smaller variance, you'll have to regress more.

Of course, that mathematical fact doesn't make it non racist or morally right to do such adjustment. You could add a couple simple extra questions to the test, to obtain similar improvement in the accuracy. Or you could use some other side data instead - weight, height, and blood type, for example, there's a lot of other data you can use besides race, if the race is used but nothing else, that's because of tradition of racism, not because of some awesome rationality. It's fairly amusing to see how race realists justify racism with increased accuracy, but start complaining when you adjust your evaluation of them in much same manner using racism/non-racism as evidence...

edit: An important correction. The test-to-test variance may also differ between the groups. E.g. if we have some robots that always test the same, even if they have low mean, they'll have smaller regression to the mean than humans.

Comment author: RolfAndreassen 09 April 2013 08:48:45PM 0 points [-]

Consider a test followed by a re-test (which we are trying to predict). To calculate expected score on the re-test you need to apply regression to the mean.

This is only true if you assume there is some component of luck or guesswork to the score. I admit that this may be a good model for the kinds of tests you get in American high schools. However, it is not clear to me that "black people" is the correct population to use for the regression, because by construction you have an untypical member. Why not "high-scoring people" or "all students"?

Perhaps it would be helpful to construct an example using something other than race as the difference between populations, to avoid emotional entanglements?

Comment author: Douglas_Knight 09 April 2013 09:22:12PM 4 points [-]

Try neuroskeptic.

If there is no component of luck or guesswork or something that varies from test to test, then the retest will be exactly the same as the original test, but that's not what we see in pretty much any test. or any measurement of anything.

Comment author: DanielLC 09 April 2013 07:35:53AM *  0 points [-]

The fair approach is to have an entrance exam for better math classes, blind to the race.

Is it more important to be fair or accurate?

There are times where it's more important to be fair. For example, punishing a person because he's guilty discourages crime. Punishing someone because he's black does not. Thus, using the fact that he's black as evidence will mean more guilty people will go to jail, more innocents will avoid being jailed, and more people will commit crime.

I don't think that really applies here, though.

Comment author: private_messaging 09 April 2013 04:19:35PM *  2 points [-]

Well, what do you think about losing points because your profile photo has atypical proportions, or atypical posture? Points adjustment for round face, or for relative finger lengths? For having too many or too few facebook friends, likes, and so on? Weight, height, and blood type?

I don't think that really applies here, though.

Well, if you want to encourage education rather than encourage being white or having typical posture or other things like that, it does apply.

Comment author: Will_Sawin 10 April 2013 10:24:59PM 1 point [-]

One might say that the sanity waterline one has to cross to rationally handle test-score-based Bayesian predictions in a by-and-large rational way is much lower than the sanity waterline one has to cross to rationally handle relative-finger-length-based predictions, which itself is lower than the waterline for skin-color-based predictions.

Comment author: private_messaging 11 April 2013 05:09:25AM *  5 points [-]

What sanity? Everyone is pushing for measures that would be advantageous for themselves, opposing disadvantageous measures, and there's nothing particularly insane about that, it's just instinctive selfishness. The white 'nerds' for instance could be OK with adjustment for race, but very much not OK with adjustment for various looking odd metrics (which lump them together with the autistic). It's only Bayesian when it's someone else; when it's you losing points, that's you being lumped together with other people (on basis of some random trait that happens to be widely measured), which is of course bad and irrational and a bias (complete with examples of how it is inexact). Nothing insane about that either, it's just selfishness.

Meanwhile, I'd dare to guess you can get considerably larger boost in accuracy from adding a couple more questions to a test, or using data from some other standardized test.

Comment author: DanielLC 09 April 2013 11:05:02PM 2 points [-]

Well, if you want to encourage education rather than encourage being white or having typical posture or other things like that, it does apply.

If you're giving prizes to the best students to encourage them, then it applies. If you're trying to match the teaching style to the student, I don't think it does.

Comment author: fubarobfusco 09 April 2013 04:50:17AM 2 points [-]

higher test scores

Other way around. If you've begun with a socioeconomic disadvantage, then achieving a particular test score is an indicator of greater inherent ability, insofar as such a thing exists. Someone who can run a mile in five minutes while carrying a fifty-pound weight is a better runner than someone who can run the same mile in the same speed while carrying no additional weight.

Comment author: cousin_it 09 April 2013 09:46:43AM 4 points [-]

It seems to me that private_messaging is right and explains his point beautifully. Here's a Robin Hanson post making a similar point. Also see this discussion, especially Wei Dai's comment.

Comment author: gwern 10 April 2013 06:47:28PM 3 points [-]
Comment author: private_messaging 09 April 2013 07:02:23AM *  11 points [-]

That depends very much to specific priors and correlations.

If you're looking for the expected score on a re-test, you should apply regression towards the mean, and for a lower mean, that's more regression. A school may be interested in the probability of student success on a course, which is not a measure of inherent ability either but very much depends to the same disadvantages that lower the test score.

edit: that is to say, if you made a programming contest where the contestants write programs to predict re-test scores from score and a profile photo, given huge enough database of US students (split in two, one available to our contestants, one for the final test), winner code will literally measure skin albedo, and in some cases maybe also try to detect eyeglasses. Of course, the morale of the story is not that racism is good but that socially sometimes we don't want the most accurate guess.

edit2: Subtler measures may correlate too, besides the racial ones. E.g. angle between line connecting pupils of the eyes, and horizontal, the pupil dilation in the photo, use/non use of flash, strength of red eye effect, and who knows what else (how busy does the background look, maybe?). I don't think many people here want to have their math scores be adjusted depending to how they held their head in a photograph. edit3: ohh, and the image metadata, or noise signatures, that'd be a big one - is the image taken by an expensive camera? Get free points on your math test. And a free tip: squint. It will think you're asian or smart enough to squint.

Comment author: gwern 10 April 2013 06:49:13PM 1 point [-]

I think fubar may be right in a certain way: if you observe someone reaching a very high score while having a known poor environment (let's say you've tested them enough so one can ignore issues of <1 reliability causing a regression to the mean on subsequent retests), then you might then estimate that the non-environmental contributions must be unusually high - because something must be causing him to score very high, and it's sure not the environment. So for example, we might infer that his genes or prenatal environment or personality are better than average.

Comment author: private_messaging 10 April 2013 07:52:05PM *  1 point [-]

Yes. As I say, depends to what we are trying to predict and priors. Even with 1 test and significant regression, it's correct to infer higher non-environmental contribution, just not higher combination of environmental and non-environmental.

Comment author: blashimov 09 April 2013 04:21:31AM 17 points [-]

I would dearly like citations for everything - I would really like to know if I am still terrible at estimating how awful the world is.

Comment author: PrawnOfFate 10 April 2013 05:46:51PM -2 points [-]

Not the world, the US.

Comment author: blashimov 11 April 2013 04:40:58PM 2 points [-]
  1. Sure, in this case.
  2. The US is part of the world.
  3. I was expressing my thoughts on a general and common failure mode that I have attempted to correct, and that this article provides evidence I have failed to do so.
Comment author: wedrifid 09 April 2013 04:21:04AM 2 points [-]

There were math teachers who absolutely refused to allow blacks/poors in their classroom, or worse, treated them in such a way as to cause them to fail, thus confirming their worldview.

This is now? Not 100 years ago?

Comment author: drethelin 09 April 2013 05:31:33AM 10 points [-]
Comment author: [deleted] 09 April 2013 04:44:12AM 1 point [-]

There were certainly double standards when I was in secondary.

Comment author: NancyLebovitz 09 April 2013 11:44:46AM 1 point [-]

When was that?

Comment author: [deleted] 09 April 2013 01:36:10PM 1 point [-]

Early 2000's.

Comment author: buybuydandavis 09 April 2013 03:56:24AM *  23 points [-]

Technical person meets a bureaucracy. Good clean fun, like the Mr. Bill show. I wish I had been there when Thomas Sowell interned for the Department of Labor.

The only things about your story that surprised me was that you weren't shit canned within a month, and that an actual company exists that would hire you. You, and by extension them, rocked the boat and survived. That's not what anyone is paying you for. You're there to validate that they're doing the right thing. I don't know how you and your company have survived this long, but I'd like to thank you all for saving some students from the regularly scheduled destruction of their lives.

As for your conversations with the bureaucracy, do you really think their confusion was in not understanding your point? I'd guess that any confusion they had was in how you had a job there at all, while you were busy saying things that shouldn't be said. I think you were the one not "getting it".

Every so often someone says something that opens a new world to me. I'll pass on the new world to you.

The purpose of a bureaucracy is to further the interests of the bureaucracy, whatever goals they give lip service to. But even theoretically, you don't have the lip service goal right. That goal is not to help students. It's to remake society so that it looks right, primarily as measured by equality of outcomes for groups. Helping a white child is helping the white group, thereby making group inequality worse.

If you think in racial groups, everything makes sense. Blacks are more likely to have poor achievement, therefore you help blacks, regardless of achievement. But also, if you judge them by group, then you conclude they aren't ready for the advanced math classes either.

Why not use test scores? Because test scores are objective measurements. Can't allow those into the school system. Then the bureaucracy's performance can be judged, as you demonstrate. Can't have that.

And of course the proposal's execution is the goal of the proposal. Well, it's really the grant itself which is the goal, but they couldn't write "receive check" as the goal, so they write what they plan to do, something entirely in their power. If they wrote that the goal was the delivery of some objective measurement, someone crazy person might measure it and determine that they had failed. Can't have that. Who in the system would possibly want that? Only people like you who just don't "get it".

An anecdote from corporate bureaucracy. I was in a meeting with two managers, where they were describing back and forth what some other woman did. Except their story was completely false. And we all knew it was false. Having the foolishness of youth, if not the years and health, I came out and said "But she didn't really do that, right?" Silence. For a moment. Then the subject was changed and they moved on. People in bureaucracies spend much of the day telling each other social truths that are epistemically false. I don't say lying, because as George Constanza would say, it's not a lie, if you believe it. Their standard of truth is the socially useful. They forgot they had let a fool into the room whose standard was epistemic truth. That buffoon just doesn't get it. We won't invite him to more of these meetings.

Comment author: Randy_M 09 April 2013 03:20:55PM 5 points [-]

"As for your conversations with the bureaucracy, do you really think their confusion was in not understanding your point? I'd guess that any confusion they had was in how you had a job there at all, while you were busy saying things that shouldn't be said. I think you were the one not "getting it"."

I'm torn myself; I could see it being due to self-interested playing dumb, but also the genuine kind given that we are talking about gross failings of basic education, after all, which has to have consequences down the line.

Comment author: ThoughtSpeed 09 April 2013 05:33:02AM 4 points [-]

Not to lower signal-to-noise, but - I really liked this comment. It shows of a fine mind made cynical, a delicate sarcasm born of an impinging upon by a horrific, Cthulhian reality.

"People are crazy, the world is mad."

Comment author: buybuydandavis 11 April 2013 10:55:12PM 2 points [-]

Thank you.

At first I didn't agree with the "horrific, Cthulhian reality", but I think that's one of the Orwellian problems.

Bureaucracy's are infuriating and frustrating and horrific, but they are the way they are for reasons, like gravity. If you're willing to stare into the abyss, it's not hard at all to see why things are the way they are. An institution is a machine, indifferent to our wants and intentions. Make the machine wrong, and it will be a meat grinder for everyone involved.

But's there's the other, more human Orwellian horror. People really are different in the head, running different algorithms. Or more to the point, they aren't aliens, I am. That manager's meeting was a peek behind the curtain to true and habitual Doublethink. In their heads, the epistemic truth algorithm was completely inoperative. The Lie was completely True until I opened my yap. People can be frustrating and infuriating - but they too are what they are for reasons, like gravity, and they're not mysterious at all if you stare into their abyss.

The Hitch had a catchy phrase - "just two chromosomes away from a chimpanzee". We aren't so smart, rational, or sane. All sorts of "mysteries" dissolve in the light of that.

Comment author: Eliezer_Yudkowsky 09 April 2013 03:41:38AM 30 points [-]

The specific project I was evaluating had only gotten $800,000 out of the maximum $2m. Its strategy was to purchase the male students iPod Touches, the female students makeovers, manicures, and pedicures at a local beauty parlor, and all students were offered an additional iPod Touch or Makeover, respectively, if they passed the exam at the end of the current year. The grant proposal had specifically listed these actions as being the goal of the proposal. If the iPods and makeovers were purchased, that constituted success.

If true and documentable, I think there's a large section of the Internet which would be very, very interested and very, very loud about this because the males got iPods and the females got makeovers. (And justly so.)

Comment author: roystgnr 09 April 2013 05:01:57PM 3 points [-]

If I hadn't recently seen that "students fighting segregated prom" story from credible news sources, I'd have considered this part of the story to be nearly conclusive evidence of trolling. I should be more charitable than that.

It's still evidence, though. Who could fail to anticipate the devastatingly bad PR from "iPods vs Makeover/mani/pedis"? For that matter, why didn't the devastatingly bad PR occur? Surely the students and their parents weren't under NDA too.

Yet a Google search for 'ipod makeover school -"chic school girls"' doesn't seem to find anything relevant, with or without outraged commentary attached. This random lesswrong page comes up for me in the first couple dozen hits, even on a browser with no Google login or cookies that might trigger personalized rankings.

Nobody ever felt it was worth blogging about how their kids were being given these prizes at school?

Comment author: Eugine_Nier 11 April 2013 03:25:40AM -2 points [-]

Who could fail to anticipate the devastatingly bad PR from "iPods vs Makeover/mani/pedis"? For that matter, why didn't the devastatingly bad PR occur?

Maybe the person applying for the grant was himself black. Then it would be considered acceptable for the same reason rap lyrics are considered acceptable.

Comment author: ThinkOfTheChildren 09 April 2013 06:58:09PM 3 points [-]

The only people aware that the project happened, as far as I know, are myself, my boss, the man in charge, and the 56 students (who were in 6-8th grade at the time, and all from poor black families). The issuer of the grant was the local government, and they issue so many grants that I seriously doubt there's anyone looking at all of them.

Comment author: ChristianKl 11 April 2013 09:26:13PM *  0 points [-]

If a student with poor parents in that age group get's a free iPod, his peer and parents are likely to notice.

The idea that you can give a school a grant worth $800,000 and only one adult in the school knowing about the grant also seems strange.

If a lot of the student got the second iPod makeover, and iPod touches were more expensive at the time the whole thing might have cost $20000. That means $780,000 just disappeared. If that kind of money disappears people are bound to be interested.

Comment author: NancyLebovitz 09 April 2013 06:06:29PM 7 points [-]

If that program exists, it was tiny, which increases the odds that there' would be no public notice.

I suspect we overestimate how much of the world (not just the proportion of people, but the proportion of what's going on) is online.

Comment author: wedrifid 09 April 2013 05:16:05PM 5 points [-]

If I hadn't recently seen that "students fighting segregated prom" story from credible news sources, I'd have considered this part of the story to be nearly conclusive evidence of trolling.

I can imagine some lesswrong users thinking it would be terribly clever for them to create a sock-puppet and "test" how gullible the lesswrong readerbase is via a post like this. If such a case were ever identified I would like to see the user banned by IP rather than rewarded with status and congratulations.

Comment author: Pentashagon 09 April 2013 09:35:28PM 6 points [-]

Banning by IP is useless at best and harmful in most cases (where hapless customers of an ISP get the old IP address of a troll). There is no way to prevent attempts to test how gullible we are; therefore we need to be generally immune to all attempts and not only to specific cases or instigators.

I'd estimate a 30% probability (but with a fairly large variance) that at least some elements of this article are inaccurate and an attempt at trolling. The "best" trolls are 90% truth with one or two outrageous elements using the halo effect to gain belief.

CarlShulman's comment has no satisfactory replies yet. What is the probability that the article's profound accusations are a) completely true, b) otherwise unreported, and c) first reported on lesswrong? It seems more likely that an actual whistle-blower would choose a more widely read media outlet (Wikileaks even?), and probably more than one. b) and c) could just be my inability to find similar information reported elsewhere. It seems to be fairly common knowledge that the education system is broken but not with the specific detail in the article.

Comment author: Eugine_Nier 10 April 2013 06:00:23AM 1 point [-]

It seems more likely that an actual whistle-blower would choose a more widely read media outlet (Wikileaks even?)

The more widely read the media outlet, the higher the probability of the whistle-blower's anonymity getting blown.

Comment author: Pentashagon 10 April 2013 07:48:53PM -1 points [-]

The more widely read the media outlet, the higher the probability of the whistle-blower's anonymity getting blown.

Once it's released anywhere the risk of losing anonymity is basically the same in the end. Either it's a troll and will die here or it's true and will be disseminated everywhere. Such a strategy would only make sense if the original poster thought that this forum would replicate the research and publish it with no mention of the original source, but that seems more like the kind of thing an investigative journalist would do.