= 27d567bc2d48d9f40e9ccc20ea370b15)
nhamann comments on Open Thread June 2010, Part 3 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (606)
The Real Science Gap
ETA: Here's a money quote from near the end of the article:
(Ouch)
I'm not sure I see what the problem is. Capitalism works? It makes it seem like this system is unsustainable or bound to collapse, but I'm not sure I see how two and two fit together. I am particularly confused with this quote:
First of all, how is it a ponzi scheme that is bound to collapse? Also limiting the number of scientists is not going to make the system better, except that maybe individuals will have less competition = more opportunities, which is not a benefit to the whole system, just to the individual.
EDIT: Fixed spelling.
I'm not sure if it meets the Ponzi scheme model, but the problem is this: lots of students are going deeper into debt to get an education that has less and less positive impact on their earning power. So the labor force will be saturated with people having useless skills (given lack of demand, government-driven or otherwise, for people with a standard academic education) and being deep in undischargeable debt.
The inertia of the conventional wisdom ("you've gotta go to college!") is further making the new generation slow to adapt to the reality, not to mention another example of Goodhart's Law.
On top of that, to the extent that people do pick up on this, the sciences will continue to be starved of the people who can bring about advances -- this past generation they were lured away to produce deceptive financial instruments that hid dangerous risk, and which (governments claim) put the global financial system at the brink of collapse.
My take? The system of go-to-college/get-a-job needs to collapse and be replaced, for the most part, by apprenticeships (or "internships" as we fine gentry call them) at a younger age, which will give people significantly more financial security and enhance the economy's productivity. But this will be bad news for academics.
And as for the future of science? The system is broken. Peer review has become pal review, and most working scientists lack serious understanding of rationality and the ability to appropriately analyze their data or know what heavy-duty algorithmic techniques to bring in.
So the slack will have to be picked up by people "outside the system". Yes, they'll be starved for funds and rely on rich people and donations to non-profits, but they'll mostly make up for it by their ability to get much more insight out of much less data: knowing what data-mining techniques to use, spotting parallels across different fields, avoiding the biases that infect academia, and generally automating the kind of inference currently believed to require a human expert to perform.
In short: this, too, shall pass -- the only question is how long we'll have to suffer until the transition is complete.
Sorry, [/rant].
I agree that college as an institution of learning is a waste for most folks - they will "never use this," most disregard the parts of a liberal arts education that they're force-fed, and neither they nor their jobs benefit. Maybe students gain something from networking with each other. But yes, Goodhart's Law applies. Employers appear to use a diploma as an indicator of deligence and intelligence. So long as that's true, students will fritter away four years of their lives and put themselves deep in debt to get a magic sheet of paper.
It's been broken forever, in basically the same way it is now. Most working scientists are trying to prove their idea, because neagtive results don't carry nearly so much prestige as positive results, and the practice of science is mostly about prestige. I'm sure I could find citations for peer review being "pal review" throughout its lifetime. (ooh. I'll try this in a moment.)
To the extent that science has ever worked, it's because the social process of science has worked - scientists are just open-minded enough to, as a whole, let strong evidence change their collective minds. I'm not really convinced that the social process of science has changed significantly over the last decades, and I can imagine these assertions being rooted in generalized nostalgia. Do you have reasons to assert this?
(Are you just blowing off steam about this? I can totally support that, because argh argh argh the publication treadmill in my field headdesk headdesk expletives. But if you have evidence, I'd love to hear it.)
I mainly have evidence for the absolute level, not necessary for the trend (in science getting worse). For the trend, I could point to Goodhart phenomena like having to rely on the publication per unit time metric being gamed, and getting worse as time progresses.
I also think that in this context, the absolute level is evidence of the trend, when you consider that the number of scientists has increased; if the quality of science in general has not increased with more people, it's getting worse per unit person.
For the absolute level, I've noticed scattered pieces of the puzzle that, against my previous strong presumption, support my suspicions. I'm too sleepy to go into detail right now, but briefly:
There's no way that all the different problems being attacked by researchers can be really, fundamentally different: the functionspace is too small for a unique one to exist for each problem, so most should be reducible to a mathematical formalism that can be passed to mathematicians who can tell if it's solvable.
There is evidence that such connections are not being made. The example I use frequently is ecologists and the method of adjacency matrix eigenvectors. That method has been around since the 1960s and forms the basis of Google's PageRank, allowing it to identify crucial sites. Ecologists didn't apply it to the problem of identifying critical ecosystem species until a few years ago.
I've gone into grad school myself and found that existing explanations of concepts is a scattered mess: it's almost like they don't want you to understand papers or break into advanced topics that are the subject of research. Whenever I understand such a topic, I find myself able to explain it in much shorter time than experts in the field in explained it to me. This creates a fog over research, allowing big mistakes to last for years, with no one ever noticing it because too few eyeballs are on it. (This explanation barrier is the topic of my ever-upcoming article "Explain yourself!")
As an example of what a mess it is (and at risk of provoking emotions that aren't relevant to my point), consider climate science. This is an issue where they have to convince LOTS of people, most of whom aren't as smart. You would think that in documenting the evidence supporting their case, scientists would establish a solid walkthrough: a runnable, editable model with every assumption traceable to its source and all inputs traceable to the appropriate databases.
Yet when climate scientists were in the hot seat last fall and wanted to reaffirm the strength of their case, they had no such site to point anyone to. RealClimate.org made a post saying basically, "Um, anyone who's got the links to the public data, it'd be nice if you could post them here..."
To clarify, I'm NOT trying to raise the issue about AGW being a scam, etc. I'm saying that no matter how good the science is, here we have a case where it's of utmost important to explain research to the masses, and so it would have the most thorough documentation and traceability. Yet here, at the top of the hill, no one bothered to trace out the case from start to finish, fully connecting this domain to the rest of collective scientific knowledge.
Er, I'd just expect to see more science being done. I know of no one studying overall mechanisms of science-as-it-is-realized (little-s "science"), and thereby seriously influencing it. Further, that's not something current science is likely to worry about, unless someone can somehow point to irrefutable evidence that science is underperforming.
All of the points you list are real issues; I watch them myself, to constant frustration. I think they have common cause in the incentive structure of science. The following account has been hinted at many times over around Less Wrong, but spelling it out may make it clear how your points follow:
Researchers focus on churning out papers that can actually get accepted at some highly-rated journal or conference, because the quantity of such papers are seen as the main guarantor of being hired as a faculty, making tenure, and getting research grants. This quantity has a strong effect on scientists' individual futures and their reputations. For all but the most well-established or idealistic scientists, this pressure overrides the drive to promote general understanding, increase the world's useful knowledge, or satisfy curiosity[*].
This pressure means that scientists seek the next publication and structure their investigations to yield multiple papers, rather than telling a single coherent story from what might be several least publishable units. Thus, you should expect little synthesis - a least publishable unit is very nearly the author's research minus the current state of knowledge in a specialized subfield. Thus, as you say, existing explanations are a scattered mess.
Since these explanations are scattered and confusing, it's brutally difficult to understand the cutting edge of any particular subfield. Following publication pressure, papers are engineered to garner acceptance from peer reviewers. Those reviewers are part of the same specialized subfield as the author. Thus, if the author fails to use a widely-known concept from outside his subfield to solve a problem in his paper, the reviewers aren't likely to catch it, because it's hard to learn new ideas from other subfields. Thus, the author has no real motivation to investigate subfields outside of his own expertise, and we have a stable situation. Thus, your first and second points.
All this suggests to me that, if we want to make science better, we need to somehow twiddle its incentive structure. But changing longstanding organizational and social trends is, er, outside of my subfield of study.
[*] This demands substantiation, but I have no studies to point to. It's common knowledge, perhaps, and it's true in the research environments I've found myself in. Does it ring true for everyone else reading this, with appropriate experience of academic research?
No, these are recent developments (though the stuff from your first post may be old). For the first 300 years, scientists were amateurs without grants and no one cared about quantity. For evidence of recent changes, look at the age of NIH PIs
-- Bruno Latour, Portait of a Biologist as Wild Capitalist
(ETA: see also.)
I think you've got an example of generalizing from one example, and perhaps the habit of thinking of oneself as typical-- you're unusually good at finding clear explanations, and you think that other people could be about as good if they'd just try a little.
I suspect they'd have to try a lot.
As far as I can tell, most people find it very hard to imagine what it's like to not understand knowledge they've assimilated, which is another example of the same mistake.
Well, I appreciate the compliment, but keep in mind you haven't personally put me to the test on my claim to have that skill at explaining.
But I don't understand why this would be hard -- people make quite a big deal about how "I was little boy/girl like you too one time". Certainly a physics professor would generally remember what it was like to take their first physics class, what confused them, what way of thinking made it clearer, etc.
(I remember one of my professors, later my grad school advisor (bless his heart), was a master at explaining and achieving Level 2 understanding on topics. He was always able to connect it back to related topics, and if students had trouble understanding something, he was always able to identify what the knowledge deficit was and jump in with an explanation of the background info needed.)
To the extent that your assessment is accurate, this problem people have can still be corrected by relatively simple changes in practice. For example, instead of just learning the next class up and moving on, people could make a habit of checking for how it connects to the previous class's knowledge, to related topics, to introductory class knowledge, and to layperson knowledge. It wouldn't help current people, as you have to make it an ongoing effort, but it doesn't sound like it's hard.
Also, is it really that hard for people to ask themselves, "Assume I know nothing. What would I have to be told to be able to do this?"
I remember that it was all pretty straightforward and intuitive. This was not a typical experience, and it also means that I don't really know what average students have trouble with in basic Newtonian physics. Physics professors tend to be people who were unusually good at introductory physics classes. (Meanwhile, I can't seem to find an explanation of standard social skills that doesn't assume a lot of intuitions that I find non-obvious. Fucking small talk, how does it work?!)
Most professors weren't typical students, so why would their recollections be a good guide to what problems typical students have when learning a subject for the first time?
I remember intro physics being straightforward and intuitive, and I had no trouble explaining it to others. In fact, the first day we had a substitute teacher who just told us to read the first chapter, which was just the basics like scientific notation, algebraic manipulation, unit conversion, etc. I ended up just teaching the others when something didn't make sense.
If there was any pattern to it, it was that I was always able to "drop back a level" to any grounding concept. "Wait, do you understand why dividing a variable by itself cancels it out?" "Do you understand what multiplying by a power of 10 does?"
That is, I could trace back to the beginning of what they found confusing. I don't think I was special in having this ability -- it's just something people don't bother to do, or don't themselves possess the understanding to do, whether it's teaching physics or social skills (for which I have the same complaint as you).
Someone who really understands sociality (i.e., level 2, as mentioned above) can fall back to the questions of why people engage in small talk, and what kind of mentality you should have when doing so. But most people either don't bother to do this, or have only an automatic (level 1) understanding.
Do you ever have trouble explaining physics to others? Do you find any commonality to the barriers you encounter?
In mathy fields, how much of it is caused by insufficiently deep understanding and how much of it is caused by taboos against explicitly discussing intuitive ways of thinking that can't be defended as hard results? The common view seems to be that textbooks/lectures are for showing the formal structure of whatever it is you're learning, and to build intuitions you have to spend a lot of time doing exercises. But I've always thought such effort could be partly avoided if instead of playing dignified Zen master, textbooks were full of low-status sentences like "a prisoner's dilemma means two parties both have the opportunity to help the other at a cost that's smaller than the benefit, so it's basically the same thing as trade, where both parties give each other stuff that they value less than the other, so you should imagine trade as people lobbing balls of stuff at each other that grow in the lobbing, and if you zoom out it's like little fountains of stuff coming from nowhere". (ETA: I mean in addition to the math itself, of course.) It's possible that I'm overrating how much such intuitions can be shared between people, maybe because of learning-style issues.
This demonstrates a highly developed theory of mind. In order to do this one needs to both have a good command of material and a good understanding of what people are likely to understand or not understand. This is often very difficult.
For me, a small but significant hack suggested by Anna Salamon was to try to act (and later, to actually be) cheerful and engaged instead of wittily laconic and 'intelligent'. That said, it's rare that I remember to even try. Picking up habits is difficult.
So what is the realistic alternative for those who have no other marketable skills, such as myself? (I specifically don't have a high school diploma, though I suppose it would be trivially easy to nab a GED.)
Until the adjustment happens, there won't be a common way because most people are still in the current inefficient mentality so you don't get scaling effects. Whatever internships friends and family can offer would probably be the best alternative.
In the future, there will probably be some standardized test you'll have to take at age 16-18 to show that you're reasonably competent and your education wasn't a sham. (The SAT tests could probably be used as they stand for this purpose.) Then, most people will go straight to unpaid or low-paid interships in the appropriate field, during which they may have to take classes to get a better theoretical background in their field (like college, but more relevant).
After a relatively short time, they will either prove their mettle and have contacts, experience, and opportunities, or realize it was a bad idea, cut their losses, and try something else. It sounds like a big downside, until you compare it to college today.
I wish I could vote this comment up a hundred times. This insane push toward college without much thought about the quality of the education is extremely harmful. People are more focused on slips of paper that signal status versus the actual ability to do things. Not only that, but people are spending tens of thousands of dollars for degrees that are, let's be honest, mostly worthless. Liberal arts and humanities majors are told that their skill set lies in the ability to "think critically"; this is a necessary but not sufficient skill for success in the modern world. (Aside from the fact that their ability to actually "think critically" is dubious in the first place.) In reality, the entire point is networking, but there has to be a more efficient way of doing this that isn't crippling an entire generation with personal debt.
I would settle for just 10 times if it were in the form of a post. ;)
Evidently the ability to think critically is instilled after the propaganda is spread.
Wow, now that is what I would call fraud. It's something the students should be able to detect right off the bat, given the lack of liberal arts success stories they can point to. It's like they just think, "I like history, so I'll study that", with no consideration of how they'll earn a living in four years (or seven). That can't last.
And I wish I could vote that up a hundred times. I wouldn't mind as much if colleges were more open about "hey, the whole point of being here is networking", but I guess that's something no one can talk about in polite company.
Tell my parents this one.
On the other hand, is 'success' an existentialist concept (in that you have to define it yourself)? I would think it'd be near impossible to come to a consensus as to what is necessary and sufficient for success.
Sure, it's vague. The point is that, for any plausible, conventional definition of success you might be able to come up with, a typical liberal arts degree is definitely insufficient and probably unnecessary to meet that definition's criteria.
Or, it may not pass, and the American educational system may continue to gather detritus until it collapses. Anybody familiar enough with the Chinese Ming dynasty to rationally assess the similarities? I'm not.
Not to be pedantic, but that would be passing. I made no pretense about shortness in the time this will take to pass.
Sounds like we need some heroic ninjas to fill the university water supply with concentrated hallucinogens and blast it with a giant portable microwave.
If you're not happy with what they did in finance, why do you think they would have been useful in science?
They're smart. They're capable of figuring out a creative solution. And the financial instruments they designed were creative, for what they were intended, which was to hide risk and allow banks to offload mortgages to someone else. Someone benefited from the creativity, just not the average worker or consumer.
I recommend The Quants-- those guys weren't just hiding risk in mortages, and it's plausible that they were so hooked on competition and smugness that there was a lot of risk they were failing to see. They weren't just hiding it on purpose.
Yes, capable of figuring out a creative solution to maximizing their goals when faced with the incentive structure of science. You think that the people who remain fail to do science when faced with these incentives, so why do you expect these others be more altruistic?
I suppose thats true, but there is such thing as equilibrium where the factors equal each other out. I do fear that it might be to high, but again, when the price becomes unreasonable, people look for the other options that are cheaper.
Thats kind of sad actually, but no amount of government regulation can fix that. Unfortunatley there is little actual incentive for actual science in a pure capitalist society, though we've been going good so far.
From the article:
I'm not sure how typical this experience is, but assuming it is as common as the article suggests: you don't see a problem with the fact that huge numbers of highly trained (~4 years for a bachelors, 5-7 for a Ph.D) are getting paid very little to work in conditions with almost no long-term job security? You see that as being perfectly fine, and comment that "capitalism works?" I'm not sure what to say. Such job prospects are decidedly unappealing (some might say intolerable), and I think it's reasonable to suggest that such conditions will result in a substantial decrease in the number of smart, dedicated young people interested in becoming scientists. This, to put it bluntly, is a fucking shame.
Maybe that was a little harsh. But the question is, why are "huge numbers of highly trained (~4 years for a bachelors, 5-7 for a Ph.D) [...] getting paid very little to work in conditions with almost no long-term job security?" The article suggests it's because we have a surplus. But if those people weren't so highly trained, would they then get those better jobs? Probably not, people don't discriminate against you because you're "highly trained".
They likely wouldn't, but I doubt that's the point. I think the point is that if they weren't so highly trained, their employment status would be more in line with their qualifications, as opposed to the current situation where Ph.Ds are doing jobs that could be done by less-credentialed people.
That sounds unlikely to me; I've heard the word 'overqualified' used to refer to that kind of discrimination.
And also, the money which is spent on useless "education" could be spent on something more useful, or at least more fun. People with mediocre incomes at least wouldn't lose a lot of flexibility from indebtedness.