The 102 version of this class would observe that the distribution is unlikely to be really Gaussian on the tails. There's presumably zero probability that you arrive for an important meeting three days late; conversely there's some nonzero probability that you don't show up at all because you were hit by a bus. But obviously the Gaussian is a fine approximation for most practical risk levels.
I'm not so sure it is a fine approximation, though of course it's better than nothing. There's most likely considerable skew to the right -- it's much easier for an unexpected event to delay you a lot than to speed you up a lot -- so the mode (which I think is nearer to what most people think of if they aren't explicitly considering the distribution) is substantially earlier than the mean or median. Empirically, IIRC there's good evidence that asking people to predict best-case completion times for projects gets more or less identical results to asking them to predict typical completion times.
There's presumably zero probability that you arrive for an important meeting three days late
I once showed up for a meeting a week early.
Good post.
A possible difficulty with this is that if you start 65 minutes early as opposed to 45 minutes early, you may end up spending more time on your preparations, since "oh I still have plenty of time". And then the 65 minutes may become equivalent to what was 45 minutes before, since you are doing things slower. So in order to make sure that the math works as you've described, you should remember to not give in to the temptation to take your time, even if you have more of it.
I've never worked out the math if you assume gaussian, to get specific numbers of minutes that function as specific odds of lateness, but I had the same rough insight in terms of accuracy versus calibration. In general, its easier to predict outcomes in advance if you shoot for less than you're capable of, but this can lead to lowered performance if it becomes habitual. With scheduling issues, one thing I've been experimenting with is finding treats to give myself during the "pad time" so that I don't just use up all the minutes on boring stuff. So, if I'm ready early, I get to read a trashy novel or poke around on LW until its really time to go.
The second order optimization is to switch out trashy novels and LW for something more "nutritious" that has similar ability to feel like a reward. Although LW is probably more nutritious than vampires or steampunk... :-)
Edit: Reading more, I see that Alicorn and DuncanS have similar tricks.
As a purely practical measure, for really important occasions, I'll often plan in an activity at second-to-last which is actually unimportant and can be dropped. So, for example, if I have a job interview, my plan will be that, after I've found the entrance to the company office and there is as little left to go wrong as possible, I'll then, as a second-to-last activity, do something like go for a relaxed lunch at a nearby cafe, and then just stroll in at the ideal time.
On the day everything goes to pot, I can use up the time I planned for the second-to-last activity. So I should hopefully still have time to go back for the forgotten briefcase, hire a taxi from my broken-down car, replace my torn shirt, and still get to the interview hungry rather than two hours late.
This is a good plan for anything with a hard start time - weddings, theatre trips, plane trips - add a pleasant activity that you can delete if needed at second-to-last. Of course the length of this optional activity should be at least as long as the number of sigmas you need for the important one. The result is that you arrive on-time and de-stressed (almost) every time.
I upvoted this post because I love how it comprehensibly applies something take-apartable to a real, common problem.
Even though I'm rarely late for things because I have decided to go around thinking of myself as someone who likes to be freakishly early to things. (And brings a book. Sometimes I read for an hour in doctors' waiting rooms before my appointment time.)
I considered putting it in main, but I thought it'd be presumptuous for my first article style article. Also, I'm not exactly sure how to move things between main and discussion. (Also, does moving a top level post wipe out it's comments and stuff?)
If anyone knows how to change it and if it would be appropriate to do so, I'd be all ears.
Technical: Edit the article, change the “Post to” option to LessWrong, and Submit. The comments will not be lost.
Appropriateness: I leave that to others, but the votes on my comment above suggest that seven people may think so.
If I were in your position I'd first ask for someone to do a copyedit/critique of the article just for the little nits of language and flow. (Not saying there's a problem — polishing and second opinions are just a good idea.) And definitely get rid of the extra-wide paragraph spacing — better yet, go into HTML view and strip out any and all superfluous formatting. (I volunteer to do this if desired.)
If you only allocate the exact amount of time to do something, you'll be late 50% of the time. That's the way bell curves work.
Minor nit, since you stated it correctly later, "the exact amount of time" should be "the average amount of time".
Bell curves may be the general case, but for the non-car-owning public-transport-using among us the situation is quite different. If a train runs every 20 minutes then being 1 minute late for the train means being 20 minutes late at the destination. Being 1 minute early has no effect on the time arriving at the destination.
It makes the prep-time discontinuous I guess.
Course, in London everyone expects everyone to often be 20 minutes late coz of the damned trains, so maybe it matter less then, heh.
For this problem, you could make the distribution of the time it takes to get to the train station - you could easily compute the average time it takes for going there, and seeing that by planning to take exactly this amount of time to get there will make you 20 minutes late 50% of the time.
The prep time will only make the "late amount of time" discontinuous, it won't change the probability of being late.
Thanks for taking the time to write this up, it's something I wasn't consciously aware of, but, as you say, extremely obvious in hindsight.
Since it's your first article: you have way too much spacing, both in general (ie. too many short paragraphs) and in the spacing itself (it looks like you have 3 or 4 lines of white space between paragraphs). And your tone is mildly offputting - I know that you're mocking your past-self for acting as though you weren't governed by statistics, but... gah, I'm having trouble qualifying exactly why it bothers me. I want to call it patronising but I'm not sure that's quite the right word.
I know that you're mocking your past-self for acting as though you weren't governed by statistics, but... gah, I'm having trouble qualifying exactly why it bothers me. I want to call it patronising but I'm not sure that's quite the right word.
I was attempting to keep in mind short inferential distances. I tried to make each thought as small a step from the next as possible. Perhaps I went too far in the other direction?
Also, thank you for the feedback.
I actually like the 3-4 empty lines. I didn't even notice them until I read your comment, but I think they make the post easier to read (and skim).
This should be in main.
Also, in the spirit of having enjoyed others' anecdotes about how they have managed to avoid lateness; I got into the habit of setting my alarms two hours before I have to be to class; this gives me a half hour of hitting the snooze button, a half hour to dick around on the internet, and a half hour to have coffee and breakfast if I am okay with arriving five minutes late to class; and these totals are all flexible, allowing five minutes of lateness around 1-2 sigmas (which is fine); I've never been more than 15 minutes late to class since (except when my alarm fails to go off).
Do you really need those 30 minutes of hitting the snooze button? Why not sleep instead and wake up after the first alarm? I tried both and I'm much better off with the latter.
I almost never use an actual alarm (when I expect one to go off, it makes me sleep fitfully) but when I've been woken up and don't have to get up yet, I can fall into a light doze which leaves me pleasantly conscious of being comfortable and relaxed in a way that being outright asleep does not. I don't think I'd take a magic pill that caused me to be instantly alert and energetic on waking if I couldn't have that dozing period for at least a few minutes.
I don't need them, but I often take them if I can and enjoy them, and often find myself feeling significantly better after snoozing.
If you have an Android, try the "gentle alarm" app.
It seems to have roughly the same effect, except you don't have to actually press snooze and you don't have that moment when the blaring alarm goes off and you need to throw it at the wall. Well.. just try it. :)
a) I don't have an android
b) I never feel the need to throw my alarm against the wall
c) I like being able to decide groggily whether I'm ready to get up
So I appreciate your suggestion, which is probably useful for a wide variety of people, but I'm actually quite happy with my current setup and don't feel the need to change things.
There's a useful concept called 6-sigma. If you prepare for deviations of up to 6 sigma, chances are you'll about never be too late in your whole life. The question is whether you think 6 standard deviations is a time you're willing to allocate, but for really important things like job interviews, perhaps one should.
In such statistical problems, a good way of reducing the proportion of times some bad event happens is cutting down the size of the standard deviations. For the problem at hand, this means that even if something disturbs your normal morning, it doesn't have much of an effect.
In practice, regular routines with a low cost that keep your life organized are useful for this - for example, if you always have two bottles of showering gel at hand in the shower and check every day, that can help, or if you always place your car keys in some specific place X and make sure they're there. Also, if you're prepared for bad events, it's possible to avoid cascade effects (such as "couldn't get to the station in time, missed bus"), which also helps.
tl;dr: Shit happens; plan accordingly.
Good post. Nice to have some math to think about my punctuality habits.
Think it is really good using maths like this in the real world to improve skills.
I would presume what assume that arrival time would follow a normal distribution and be slanted further to the right (As there will be a minimum time to get somewhere, but no maximum time taken value that occurs) so not sure whether using the percentages is necessary.
Agree that this post should be on the main page.
Nice article. I wonder though if the the distance to 'on time' is really normally distributed - did you test this?
I'd expect it to be short-tailed on the 'early' end and long-tailed on the 'late' end. For instance, missing a connection can throw you off by half an hour, and there's no symmetric gain available. In parts of a problem that are like this, it's better to look at the frequencies of the various cases than a continuous distribution.
This is an awesome article. But I've always been bothered by people's expectations when it comes to arriving on time for things. In my experience, people are less annoyed at the person who leaves early than the person who arrives late, even if they miss the same amount of the meeting. The usual reasons people give for avoiding being late (missing content, disrupting the meeting) apply just as much to leaving early. Why the double standard? Also, people are generally more understanding if you have to miss something than if you are an hour late, for some reason.
This is all completely anecdotal, obviously.
If a meeting starts with most important things, and progresses to least important ones, missing the beginning is more serious. Also a person leaving early is just missing the latest part, but a person coming late may be missing an important context for things discussed later -- repeating this context for them is a waste of time for others, but not repeating may cause them to ask or suggest irrelevant things.
Nitpick:
people are less annoyed at the person who leaves early than the person who leaves late,
Do you mean comes late?
Aside from that, Viliam Bur has a point, and also people who leave early generally have unavoidable conflicts and are seen as busy instead of lazy/irresponsible. Also, people who come late keep the others waiting, wondering if they should start without them or give them another minute.
Arriving late is usually accompanied with others waiting for me and not being sure when I manage to arrive, which is very annoying. With early leaving this effect is absent. If I let the others know in advance that I will arrive late, the effect would disappear as well, i.e. they will be no more angry at me than they would if I left too early.
People arriving late can be plausibly put down to something that was outside the control of the individual arriving late. Leaving early from a meeting or other activity implies some form of intent on their part.
Everything's time-until-completion can be described this way: project completion times, homework, going to the airport, the duration of foreplay and sex
Minor nitpick: More accurate to say that everything with a lot of independent variables that have about the same level of impact entering into it looks like a bellcurve.That's almost all time-until-completion issues.
I was just learning about bell curves and standard deviations in stat class the other day. Seeing this makes it seem quite a bit more useful, or at least opens my mind to the possibilities of where else I could use it.
I already tend to allot myself more time than I think I need. (Especially when driving.) It still was interesting to read this post, though. I never really considered the math behind my habit.
This algorithm is trivially obvious, and doesn't work in actual meaningfully problematic cases.
It breaks down on large sigmas, where in order to get lateness probability to reasonable levels you will on average sit outside in the rain waiting for 5 hours every day.
It breaks down on large task lengths, where in order to arrive in time, you'd have to start before you got home the previous day.
[/unreasonably pissed of that article didn't have solution to impossible problems]
In hindsight, this post seems incredibly obvious. The meat of it already exists in sayings which we all know we ought to listen to: "Always arrive 10 minutes earlier than you think early is," "If you arrive on time, then you're late," or "Better three hours too soon than one minute too late." Yet even with these sayings, I still never trusted them nor arrived on time. I'd miss deadlines, show up late, and just be generally tardy. The reason is that I never truly understood what it took to arrive on time until I grokked the math of it. So, while this may be remedial reading for most of you, I'm posting this because maybe there's someone out there who missed the same obviousness that I missed.
Statistical Distributions
Everyone here understands that our universe is controlled and explained by math. Math describes how heavenly bodies move. Math describes how our computers run. Math describes how other people act in aggregate. Wait a second, something's not right with that statement... "other people". The way it comes out it's natural to think that math controls the way that other people act, and not myself. Intellectually, I am aware that I am not a special snowflake who is exempt from the laws of math. While I had managed to propagate this thought far enough to crush my belief in libertarian free will, I hadn't propagated it fully through my mind. Specifically, I hadn't realized I could also use math to describe my actions and reap the benefit of understanding them mathematically. I was still late to arrive and missing deadlines, and nothing seemed to help.
But wait, I'm a rationalist! I know all about the planning fallacy; I know to take the outside view! That's enough to save me right? Well, not quite. It seemed I missed one last part of the puzzle... Bell Curves.
When I go to work every day, the time from when I do nothing but getting ready to go to work until the time that I actually arrive there (I'll just call this prep time) usually takes 45 minutes, but sometimes it can take more time or less time. Weirdly and crazily enough, if you plot all the prep times on a graph, the shape would end up looking roughly like a bell. Well that's funny. Math is for other people, but my behavior appears like it can be described statistically. Some days I will have deviations from the normal routine that help me arrive faster while other days will have things that slow me down. Some of them happen more often, some of them happen less often. If I were describable by math, I could almost call these things standard deviations: days where I have almost zero traffic prep time takes 1 standard deviation less, days when I can't find my car keys my prep time takes 1 standard deviation more, days I realize would be late and skip showering take 2 standard deviations less, and days when there is a terrible accident on the freeway end up requiring +2 or +3 standard deviations more in time. To put it in other words, my prep time is a bell curve, and I've got 1-sigma and 2-sigma (and occasionally 3-sigma) events speeding me up and slowing me down.
This holds true for more than just going to work. Everything's time-until-completion can be described this way: project completion times, homework, going to the airport, the duration of foreplay and sex. Everything. It's not always bell curves, but it's a probability distribution with respect to completion times, and that can help give useful insights.
Starting 'On Time' Means You Won't be On Time
What do we gain by understanding that our actions are described by a probability distribution? The first and most important take away is this: If you only allocate the exact amount of time to do something, you'll be late 50% of the time. I'm going to repeat it and italicize because I think it's that important of a point. If you only allocate the exact amount of time to do something, you'll be late 50% of the time. That's the way bell curves work.
I know I've heard jokes about how 90% of the population has above average children, but it wasn't until I really looked at the math of my behavior that I realized I was doing the exact same thing. I'd say "oh it takes me 45 minutes on average to go to work every day, so I'll leave at 7:15." Yet I never realized that I was completely ignoring that half the time would take longer than average. So half the time, I'd end up be pressed for time and have to skip shaving (or something) or I'd end up late. I was terribly unpunctual until I realized I that I had to arrive early to always arrive on time. "If you arrive on time, then you are late." Hmm. You win this one, folk wisdom.
Still, the question remained. How much early would it take to never be late? The answer lay in bell curves.
Acceptable Lateness and Standard deviation
Looking at time requirements as a bell curve implies another thing: One can never completely eliminate all lateness; the only option is to make a choice about what probability of lateness is acceptable. A person must decide what lateness ratio they're willing to take, and then start prepping that many standard deviations beforehand. And, despite what employers say, 0% is not a probability.
If my prep time averages 45 minutes with a standard deviation of 10 minutes then that means...
That's really good risk reduction for a small amount of time spent. (NB, remember that averages are dangerous little things. Taking this to a meta level, consider that being late to work about 3 times a month isn't helpful if you arrive late only once the first month, then get fired the next month when you arrive late 5 times. Hence, "Always arrive 10 minutes earlier than you think early is." God I hate folk wisdom, especially when it's right.)
The risk level you're acceptable with dictates how much time you need for padding. For job interviews, I'm only willing to arrive late to 1 in 1000, so I prepare 3 standard deviations early now. For first dates, I'm willing to miss about 5%. For dinners with the family, I'm okay with being late half the time. It feels similar to the algorithm I used before, which was a sort of ad-hoc thing where I'd prepared earlier for important things. The main difference is that now I can quantify the risk I'm assuming when I procrastinate. It causes each procrastination to become more concrete for me, and drastically reduces the chance that I'll be willing to make those tradeoffs. Instead of being willing to read lesswrong for 10 more minutes in exchange for "oh I might have to rush", I can now see that it would increase my chance of being late from 16% to 50%, which is flatly unacceptable. Viewing procrastination in terms of the latter tradeoff makes it much easier to get myself moving.
The last quote is "Better three hours too soon than one minute too late." I'm glad that at least that one's wrong. I'm sure Umesh would have some stern words for that saying. My key to arriving on time is locating your acceptable risk threshold and making an informed decision about how much risk you are willing to take.
Summary
The time it takes for you to complete any task is (usually) described by a bell curve. How much time you think you'll take is a lie, and not just because of the planning fallacy. Even if you do the sciency-thing and take the outside view, it's still not enough to keep you from getting fired or showing up to your interview late. To consistently show up on time, you must incorporate padding time.
So I've got a new saying, "If you wish to be late only 2.3% of the time, you must start getting ready at least two standard deviations before the average prep time you have needed historically." I wish my mom would have told me this one. It's so much easier to understand than all those other sayings!
(Also my first actual article-thingy, so any comments or suggestions is welcome)