Is there any way to do these things without paying a large pricetag? Could you just lurk around campus or something? Only half-joking here.
be sure to first consider the most useful version of grad that you could reliably make for yourself... and then decide whether or not to do it.
Planning fallacy is going to eat you alive if you use this technique.
Grad school is free. At most good PhD programs in the US, if you get in then they will offer you funding which covers tuition and pays you a stipend on the order of $25K per year. In return, you may have to do some work as a TA or in a professor's lab.
The real cost is the ~5 years of your life.
Predictably, Naruto turns out to have inherited all his abilities from his parents, and then improved on them only because he was possessed by the ancient spirit of one of the most powerful beings in existence. And even before that, when the story required him to be the underdog hero, he tended to overcome obstacles using the Kyuubi.
All of the Narutoverse in general is about magic powers (chakra, whatever) passing on from parents to children without much of a change. There's exactly one character in all of Narutoverse who's called out for being powerful due to training, and it isn't Naruto. (A few others are powerful due to research, which is of course always evil.)
Naruto is the opposite of Tsuioku Naritai. It's the story of "everyone had something to protect and practiced like mad, but none of it made a huge difference and most everyone would have been about as powerful anyway." Naruto climbs trees (metaphorically speaking) for many chapters, but keeps being the underdog. Then he starts manifesting powers that make him the most powerful individual in the universe - because he's a shonen hero - and which are entirely due to his parents and outside intervention.
I'm only partway through (the show progresses slowly, even though I'm watching an abridged version that cuts out a bunch of flashbacks and such), but so far the growth mindset & desire to be stronger have been hitting my S1 much more than the "person of destiny" stuff. Basically the reverse of trope #2 in Swimmer963's list - lots of attention on the "practice to become awesome" part while the "inherently awesome" part comes up in passing or off-screen.
It's possible that the story as a whole suffers from the problem that you're pointing out, but that's not the message that my system 1 gets as I watch it.
Naruto. Lots of wanting to be stronger, and training hard in order to become stronger, often as a response to frustration about not being strong enough yet. Tree climbing example.
There are probably many other examples of Japanese fiction with similar themes (Eliezer basically said as much), but Naruto is the one that I'm familiar with.
That is why I looked at all 10 questions in aggregate.
Well, you did not look at calibration, you looked at overconfidence which I don't think is a terribly useful metric -- it ignores the actual calibration (the match between the confidence and the answer) and just smushes everything into two averages.
It reminds me of an old joke about a guy who went hunting with his friend the statistician. They found a deer, the hunter aimed, fired -- and missed. The bullet went six feet to the left of the deer. Amazingly, the deer ignored the shot, so the hunter aimed again, fired, and this time the bullet went six feet to the right of the deer. "You got him, you got him!" yelled the statistician...
So, no, I don't think that overconfidence is a useful metric when we're talking about calibration.
but I also did another analysis which looked at slopes across the range of subjective probabilities
Sorry, ordinary least-squares regression is the wrong tool to use when your response variable is binary. Your slopes are not valid. You need to use logistic regression.
Overconfidence is the main failure of calibration that people tend to make in the published research. If LWers are barely overconfident, then that is pretty interesting.
I used linear regression because perfect calibration is reflected by a linear relationship between subjective probability and correct answers, with a slope of 1.
If you prefer, here is a graph in the same style that Yvain used.
X-axis shows subjective probability, with responses divided into 11 bins (<5, <15, ..., <95, and 95+). Y-axis shows proportion correct in each bin, blue dots show data from all LWers on all calibration questions (after data cleaning), and the line indicates perfect calibration. Dots below the line indicate overconfidence, dots above the line indicate underconfidence. Sample size for the bins ranges from 461 to 2241.
D_Malik's scenario illustrates that it doesn't make sense to partition the questions based on observed difficulty and then measure calibration, because this will induce a selection effect. The correct procedure to partition the questions based on expected difficulty and then measure calibration.
Yes, I agree with that. However it still seems to me that the example with coins is misleading and that the given example of "perfect calibration" is anything but. Let me try to explain.
Since we're talking about calibration, let's not use coin flips but use calibration questions.
Alice gets 100 calibration questions. To each one she provides an answer plus her confidence in her answer expressed as a percentage.
In both yours and D_Malik's example the confidence given is the same for all questions. Let's say it is 80%. That is an important part: Alice gives her confidence for each question as 80%. This means that for her the difficulty of each question is the same -- she cannot distinguish between then on the basis of difficulty.
Let's say the correctness of the answer is binary -- it's either correct or not. It is quite obvious that if we collect all Alice's correct answers in one pile and all her incorrect answers in another pile, she will look to be miscalibrated, both underconfident (for the correct pile) and overconfident (for the incorrect pile).
But now we have the issue that some questions are "easy" and some are "hard". My understanding of these terms is that the test-giver, knowing Alice, can forecast which questions she'll be able to mostly answer correctly (those are the easy ones) and which questions she will not be able to mostly answer correctly (those are the hard ones). If this is so (and assuming the test-giver is right about Alice which is testable by looking at the proportions of easy and hard questions in the correct and incorrect piles), then Alice fails calibration because she cannot distinguish easy and hard questions.
You are suggesting, however, that there is an alternate definition of "easy" and "hard" which is the post-factum assignment of the "easy" label to all questions in the correct pile and of the "hard" label to all questions in the incorrect pile. That makes no sense to me as being an obviously a stupid thing to do, but it may be that the original post argued exactly against this kind of stupidity.
P.S. And, by the way, the original comment which started this subthread quoted Yvain and then D_Malik pronounced Yvain's conclusions suspicious. But Yvain did not condition on the outcomes (correct/incorrect answers), he conditioned on confidence! It's a perfectly valid exercise to create a subset of questions where someone declared, say, 50% confidence, and then see if the proportion of correct answers is around that 50%.
Suppose that I am given a calibration question about a racehorse and I guess "Secretariat" (since that's the only horse I remember) and give a 30% probability (since I figure it's a somewhat plausible answer). If it turns out that Secretariat is the correct answer, then I'll look really underconfident.
But that's just a sample size of one. Giving one question to one LWer is a bad method for testing whether LWers are overconfident or underconfident (or appropriately confident). So, what if we give that same question to 1000 LWers?
That actually doesn't help much. "Secretariat" is a really obvious guess - probably lots of people who know only a little about horseracing will make the same guess, with low to middling probability, and wind up getting it right. On that question, LWers will look horrendously underconfident. The problem with this method is that, in a sense, it still has a sample size of only one, since tests of calibration are sampling both from people and from questions.
The LW survey had better survey design than that, with 10 calibration questions. But Yvain's data analysis had exactly this problem - he analyzed the questions one-by-one, leading (unsurprisingly) to the result that LWers looked wildly underconfident on some questions and wildly overconfident on others. That is why I looked at all 10 questions in aggregate. On average (after some data cleanup) LWers gave a probability of 47.9% and got 44.0% correct. Just 3.9 percentage points of overconfidence. For LWers with 1000+ karma, the average estimate was 49.8% and they got 48.3% correct - just a 1.4 percentage point bias towards overconfidence.
Being well-calibrated does not only mean "not overconfident on average, and not underconfident on average". It also means that your probability estimates track the actual frequencies across the whole range from 0 to 1 - when you say "90%" it happens 90% of the time, when you say "80%" it happens 80% of the time, etc. In D_Malik's hypothetical scenario where you always answer "80%", we aren't getting any data on your calibration for the rest of the range of subjective probabilities. But that scenario could be modified to show calibration across the whole range (e.g., several biased coins, with known biases). My analysis of the LW survey in the previous paragraph also only addresses overconfidence on average, but I also did another analysis which looked at slopes across the range of subjective probabilities and found similar results.
There's been far less writings on improving rationality here on LW during the last few years. Has everything important been said about the subject, or have you just given up on trying to improve your rationality? Are there diminishing returns on improving rationality? Is it related to the fact that it's very hard to get rid off most of cognitive bias, no matter how hard you try to focus on them? Or have people moved talking about these on different forums, or in real life?
Or like Yvain said on 2014 Survey results.
It looks to me like everyone was horrendously underconfident on all the easy questions, and horrendously overconfident on all the hard questions. To give an example of how horrendous, people who were 50% sure of their answers to question 10 got it right only 13% of the time; people who were 100% sure only got it right 44% of the time. Obviously those numbers should be 50% and 100% respectively.
This builds upon results from previous surveys in which your calibration was also horrible. This is not a human universal - people who put even a small amount of training into calibration can become very well calibrated very quickly. This is a sign that most Less Wrongers continue to neglect the very basics of rationality and are incapable of judging how much evidence they have on a given issue. Veterans of the site do no better than newbies on this measure.
I re-analyzed the calibration data, looking at all 10 question averaged together (which I think is a better approach than going question-by-question, for roughly the reasons that D_Malik gives), and found that veterans did better than newbies (and even newbies were pretty well calibrated). I also found similar results for other biases on the 2012 LW survey.
You're safeguarding against the wrong thing. If I needed to fake a prediction that badly, I'd find a security hole in Less Wrong with which to edit all your comments. I wouldn't waste time establishing karma for sockpuppets to post editable hashes to deter others from posting hashes themselves, that would be silly. But as it happens, I'm not planning to edit this hash, and doing that wouldn't have been a viable strategy in the first place.
"Clearly, the way to make our safeguards super-secure is to make yet another comment with the hash."
"Clearly, the way to make my safeguards super-secure is to make yet another Horcrux."
Somehow, you could only see through one of these strategies.
Nice post!
Related: Being proven wrong is like winning the lottery by Phil Birnbaum, How to enjoy being wrong by lincolnquirk
I'm about to start being paid for a job, and I was looking at investment advice from LW. I found this thread from a while back and it seemed good, but it's also 4 years old. Can anyone confirm if the first bullet is still accurate? (get VTSMX or VFINX on vanguard, it doesn't matter too much which one.)
My money is still in VTSMX.
(Actually, half of it is in VTSMX and half is in VGTSX, which is the non-US index fund. But putting it all into VTSMX is fine too.)
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
= 27d567bc2d48d9f40e9ccc20ea370b15)
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
I have an intuition that if we implemented universal basic income, the prices of necessities would rise to the point where people without other sources of income would still be in poverty. I assume there are UBI supporters who've spent more time thinking about that question than I have, and I'm interested in their responses.
(I have some thoughts myself on the general directions responses might take, but I haven't fleshed them out, and I might not care enough to do so.)
If you want information on how increased income due to UBI would affect people's spending on food, you can look at the data that we already have on the relationship between income and spending on food. Three stylized facts:
As income goes up, the proportion of income spent on food goes down.
As income goes up, the total amount of money spent on food goes up.
As income goes up, the proportion of one's food budget spent on restaurants goes up.
These trends generally hold if you are comparing different countries with each other, or if you are comparing different people within a single country, or if you are looking at a single country over time as it gets richer. I don't see any strong reasons to think that they wouldn't also apply to people whose income went up due to receiving a new UBI.
So if a household was making $20,000 per year and spending 20% of it ($4,000) on food, and UBI increases their income to $25,000 per year, then we can predict that they will spend somewhere between $4,000 and $5,000 per year on food, and some of the increased spending will go towards increased quality & convenience (such as eating out). You could probably make more precise predictions if you tried to put numbers on the three stylized facts.
More generally, the model here is: UBI affects the distribution of 'income after taxes & transfers', and the distribution of 'income after taxes & transfers' affects other things like prices & spending habits. So if you want to predict how UBI will affect something like prices, then study how 'income after taxes & transfers' affects prices, and combine that with your estimate of how the UBI will affect the distribution of 'income after taxes & transfers'.