Comment Permalink

John_Maxwell9y200

One thing I've noticed is that framing the same task differently can make me either intrinsically motived to do it or not do it. For example, I have a text file full of blog posts I've identified as high-quality to read at some point. But reading blog posts from this file feels boring and forced in a way that browsing blog posts from Hacker News does not. I'm not sure how to explain this. However, one thing I do try to do is that once I notice myself conceptualizing some task that I endorse in a way that I find intrinsically motivating, I try not to destroy that conceptualization. For example, I don't force myself to do it if I happen not to feel like doing it at some point. I stopped using the Pomodoro Technique because I was afraid it was destroying my intrinsic motivation.

I did some reading of the literature on intrinsic motivation and came to a conclusion I hadn't seen anywhere else, which is that people are intrinsically motivated to complete tasks that raise their status. The reason "extrinsic" rewards don't work: the implicit message of such rewards is that you are lower status than the reward-giver. I don't remember all the evidence that lead me to my conclusion but I do remember that e.g. when the reward-giver hands out praise as a reward for good performance, people perform as well as they do when they are "intrinsically" motivated.

Dr_Manhattan9y10

But reading blog posts from this file feels boring and forced in a way that browsing blog posts from Hacker News does not.

I (and surely many others) have similar problem; one thing that helps a bit is using Pocket app - it's a much nicer reading experience than "this file"; reading things in Notepad is aesthetically aversive. (I also occasionally use VoiceDream, that integrates with pocket and does decent text2speech, so you can listen to the posts when walking etc.)

4btrettel9y

Do you remember any particular articles or whatnot on motivation that helped you come to the conclusion that the opportunity to raise your status motivates? This seems plausible to me and I'd like to read more. Edit: Social status is mentioned on Wikipedia a few times, e.g.

7JonahS9y

Yes, I think that the situation is that people are biologically hardwired to pursue their comparative advantage because doing so was was historically what was most conducive to becoming higher status, so that people's motivation goes way up when they're pursuing their natural comparative advantage (relative to their subjectively perceived communities).

See in context

29 Intrinsic motivation is crucial for overcoming akrasia

by JonahS

17th Jun 2015

4 min read

29

tl;dr: If you struggle with motivational problems, it's likely that the problem is not intrinsic to you, but instead that you haven't yet found work that you find very interesting.

How I discovered how to do great work

Last winter I did something that I had never done before. I spent ~1500 hours working on genuinely original scientific research.

I had done research for my PhD in pure math, but faced squarely, the problems that I worked on were of very little interest to anyone outside of the fields, and I was not very engaged with my research. Pure math is very heavily stacked with talent, and the low hanging fruit has been plucked, so unless you're one of the most talented people in the world, your prospects for doing anything other than derivative work are very poor.

What I did last fall was entirely different. As I trained to be a data scientist, I found that there's far more low hanging fruit in the field than there is in pure math, and found myself working on novel problems that are of broad interest almost immediately.

Having very high intrinsic motivation made a huge difference. I found myself spending all waking hours (~90 hours / week) working on it obsessively, almost involuntarily. Once I emerged, I realized that what I had done over the past ~3.5 months was far more significant than all of the other work that I had done over the span of my entire life combined. I was astonished to find myself having ascended to the pantheon of those who have made major contributions to human knowledge, something that I hadn't imagined possible in my wildest dreams.

The problem isn't "laziness"

Many of the most interesting people who I know are achieving at a level far below their potential. They often have major procrastination problems, and believe this to correspond to them having a character flaw of "laziness". I've become convinced that these people's problems don't come from them being insufficiently disciplined.

Their problems come from them spending their time trying to do work that they find boring. If you find your work boring, it's very likely that you should be doing something else.

References

My position is not unique to me: it's common to extremely high functioning people.

[1] Steve Jobs created Apple, which owns ~0.1%+ of the world's wealth. In his 2005 Stanford commencement address he said:

I'm convinced that the only thing that kept me going was that I loved what I did. You've got to find what you love. And that is as true for your work as it is for your lovers. Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven't found it yet, keep looking. Don't settle. As with all matters of the heart, you'll know when you find it. And, like any great relationship, it just gets better and better as the years roll on. So keep looking until you find it. Don't settle.

[2] Bill Thurston is one of the greatest mathematicians of the 20th century. He formulated the geometrization conjecture, which subsumes the 100 year old Poincare conjecture, considered one of the ~7 most important unsolved problems. He describes his own character as follows:

My attention is more inward than that of most people: it can be resistant to being captured and directed externally. Exercises like these mathematics lessons were excruciatingly boring and painful (whether or not I had "mastered the material"). I used to think my wandering attention and difficulty in completing assignments was a defect, but now I realize my "laziness" is a feature, not a bug. Human society wouldn't function well if everyone were like me, but society is better with everyone not being alike.

[3] Scott Alexander / Yvain is widely regarded as a great writer. Political celebrity Ezra Klein characterized his blog as fantastic. Scott wrote:

On the other hand, I know people who want to get good at writing, and make a mighty resolution to write two hundred words a day every day, and then after the first week they find it’s too annoying and give up. These people think I’m amazing, and why shouldn’t they? I’ve written a few hundred to a few thousand words pretty much every day for the past ten years.

But as I’ve said before, this has taken exactly zero willpower. It’s more that I can’t stop even if I want to. Part of that is probably that when I write, I feel really good about having expressed exactly what it was I meant to say. Lots of people read it, they comment, they praise me, I feel good, I’m encouraged to keep writing, and it’s exactly the same virtuous cycle as my brother got from his piano practice.

[4] Paul Graham is the co-founder of Y-Combinator, a seed funder with a portfolio of combined value exceeding $30 billion (with investees including Dropbox, AirBnB and Stripe). In What You'll Wish You Had Known he wrote

One of the most dangerous illusions you get from school is the idea that doing great things requires a lot of discipline. Most subjects are taught in such a boring way that it's only by discipline that you can flog yourself through them.

Now I know a number of people who do great work, and it's the same with all of them. They have little discipline. They're all terrible procrastinators and find it almost impossible to make themselves do anything they're not interested in. One still hasn't sent out his half of the thank-you notes from his wedding, four years ago. Another has 26,000 emails in her inbox.

I'm not saying you can get away with zero self-discipline. You probably need about the amount you need to go running. [...] But once they get started, interest takes over, and discipline is no longer necessary.

Do you think Shakespeare was gritting his teeth and diligently trying to write Great Literature? Of course not. He was having fun. That's why he's so good.

Motivations

Personal Blog

29

Mentioned in

26Akrasia is confusion about what you want

New Comment

43 comments, sorted by

top scoring

Click to highlight new comments since: Today at 11:49 PM

[-]John_Maxwell9y200

[-]JonahS9y70

I did some reading of the literature on intrinsic motivation and came to a conclusion I hadn't seen anywhere else, which is that people are intrinsically motivated to complete tasks that raise their status.

[-]John_Maxwell9y30

That suggests one way to motivate yourself to do something is to surround yourself with other people who are doing it badly.

[-]Epictetus9y60

You'll never get quality feedback from that kind of environment. If the bar is so low that you need only exert minimal effort to outclass everyone around you, then how will you ever be able to excel?

Worst case, you put yourself in a toxic environment and lose all motivation. If those fine folks around you don't find something important, then why should you? You can pick up bad habits that way. For example, there's one study which found that if your friends are obese, then you have a much higher chance (57%) of becoming obese yourself.

[-]John_Maxwell9y00

“No matter how slow you go, you are still lapping everybody on the couch.” If the choice is between lacking motivation entirely and having some motivation, the second seems better. One possibility is that you could find an environment that motivates you intrinsically, then once intrinsic motivation was acquired, start setting challenges for yourself.

[-]SatvikBeri9y20

I've found a mixed approach helpful: spend some time with people who don't know how to do X, because you can add a lot of value to their lives by showing them how to do it better. Spend some time with people who are much better at X than you, so you consistently improve (and have new things to teach.)

I think most people tend to be far too hesitant to teach things they know, because they know that someone else understands it better. But if that person isn't doing the work to teach, then simply talking about what you know can be incredibly valuable.

[-]Lumifer9y00

In practice I think it would promote laziness and mediocrity much more than motivate you to excel.

[-]btrettel9y40

Edit: Social status is mentioned on Wikipedia a few times, e.g.

[-]John_Maxwell9y50

I think I just searched for info on extrinsic and intrinsic motivation on Google Scholar. Looking at my notes, one of the papers I found was called "Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being" if that helps.

I think another tidbit that persuaded me was seeing autonomy, mastery, and community as all being elements cited as important for intrinsic motivation. These all seemed like facets of social status: autonomy means you are in control (and therefore statusful), mastery means you can build skills that get you status, and community means you have people to be high status relative to.

I was just skimming though; I wouldn't read too much in to my conclusion. If you want to do a literature review and write up your findings that'd probably be pretty valuable. There might be other interesting findings to report on, e.g. IIRC performance on menial tasks doesn't suffer in response to extrinsic rewards.

[-]Dr_Manhattan9y10

But reading blog posts from this file feels boring and forced in a way that browsing blog posts from Hacker News does not.

[-]TeMPOraL9y120

I suffer from severe case of Akrasia that makes me work at 10% of my capacity most of the time; here's something I discovered that made me believe problem is actually in me: my closest friend. I know her for many years, and I never ever saw her working at less than 110% of her capacity. She worked in groceries, online bookshop, sold LED bulbs and furniture, managed people, did customer relations and even social media marketing. She wants to be a writer; she hated almost every one of those jobs, felt they're hindering her development, and yet no matter how tired she was, how annoyed or abused by her bosses, she could always find the strength to focus and do her job at scary efficiency. And given all the people I ever worked with, who never had problems with focus or productivity in the ballpark of my own, I can't conclude otherwise that it's me who is just wired wrongly. I wish I had even 5% of professionalism of my friend, I could do so much more than I am able to do now.

[-]Viliam9y120

This seems to me like a simplification which focuses on one part of the problem, and ignores another part. For some people it will work great (those who have the other part solved), for others it will not work (those who have problems in the other part).

It is not just how interesting or boring is the activity X for you, but also how much do the other circumstances in your life allow you to focus on X. Imagine that you intrinsically love X, but whenever you do it, you get punished by an external force. Gradually your mind will start associating X with unpleasant feelings, and after a while you will find you are no longer attracted to the idea of doing X. A different person starting with the same love towards X, but in a more supportive environment, would have it easier to persevere.

It is not necessarily the whole environment that has an impact on you, but people who matter most. If your parents disapprove of X, but your friends give you admiration and love for your X abilities, it will be relatively easy to emotionally overcome the disapproval of your parents.

Maybe it is my own specific problem -- I have no idea how frequent it is -- but I find it really difficult to find people (in offline world) who would encourage me in doing what I want to do. People around me either don't care about what I do, or don't care about my preferences but have their own ideas about what I should do instead.

Before "you should have fun" is "you should have an environment that supports you in having fun doing X". People are social animals.

[-]Tem429y10

More general: "you should have an environment that is conductive to you doing X". Lots of people thrive in adversity, and a notable minority aren't very social. This may actually be true for most people in limited situations... I was very unlikely to clean up other people's liter and dog poop until I moved into a neighborhood where 1) no one cleans up at all and 2) people thought it was weird to do so. (If instead they had thought that they should clean up, but were too lazy, I would have seen the mess as a fitting punishment for their laziness, and would only have cleaned up that which directly impinged on my immediate environment). Of course, I don't give this as an example of a healthy environment, just as an example where a motivation to do productive work is not dependent on social support.

[-]Lumifer9y90

If you find your work boring, it's very likely that you should be doing something else.

That's true, but that's, um, not the whole truth :-)

I usually express this as saying that your work should be at the intersection of three sets:

Something that you love to do
Something that you're good at
Something that's valuable enough so that other people will pay you for it.

[-][anonymous]9y20

... and if you have dependents, the second two increase in importance. Getting paid matters more and being good at it means getting paid more or the same but with more security.

I have this feeling that I should have tried a career change before we had a child, now it would not be responsible to risk.

I do ERP and frankly I am starting to get really fed up programming the 143. slightly different version of a sales per country per item category report or explaining to accountants that having 0 quantity of goods does not always mean having 0 value of goods.

[-]Vaniver9y50

Steve Jobs created Apple, which owns ~0.1%+ of the world's wealth.

Apple does not own itself. Ownership of Apple may represent a thousandth of the total world wealth, but that's a different thing.

Their problems come from them spending their time trying to do work that they find boring. If you find your work boring, it's very likely that you should be doing something else.

Suppose that 1) I care about my personal satisfaction and fun while working, and 2) I care about the effects that my work has on the world. It's may be that the work you find satisfying--the part you could throw your whole self into and become truly great and exceptional at--is work that no one else cares about, and so won't get you anything in the way of status or sustenance or 'secondary satisfaction.'

If I can applaud the example of Ethan Dickinson, my impression is that the sort of work he finds innately most satisfying, and the sort of thing that he can't not do (as evidenced by his Facebook feed), is related to linguistics. But what he spends most of his time on is programming software, because that pays considerably more than linguistics, and then he can donate that money to MIRI.

I don't know how he feels about programming, but if he were to ask "hey, how do I do something despite not having innate motivation for it?", I would not want to advise him to quit what he does and do what he's innately motivated to do instead. I would point instead to the resources our culture has built up to enhance discipline.

So discipline is important, and indirect satisfaction are important. But I'm not arguing that direct satisfaction and raw enthusiasm are unimportant either; the moderate path is to simultaneously optimize for all four together, and figure out what the right tradeoff between them is.

[-]btrettel9y20

Which resources would you recommend for enhancing discipline?

(You have recommended a few things in particular to me before, but I'm writing this to see what other recommendations you would have, and for others' benefit.)

[-]Vaniver9y130

I know you like to do literature dives, and that's something that I haven't done in this area. The primary name is Angela Lee Duckworth and the modern term is grit.

I think you know many of the systematic tricks, like scheduling your day and using Beeminder, but I'll talk a bit about them for the benefit of others.

Scheduling and task organization are highly connected, so first I'll talk about systems that build on todo lists.

People seem to vary widely in what sort of schedules and todo lists they prefer. The most famous book in this area is Getting Things Done, and many other systems are modifications or reactions to it, of which the one I find most interesting is Zen To Done.

The core insight of Getting Things Done is that when you have external memory systems, you no longer need to use internal memory systems--instead of your mind constantly cycling through things that need to happen in order to make sure that they're remembered, that attention can be freed up for the task at hand.

Another key insight is the Most Important Task, and a related visualization is the Big Rocks (i.e. first block out time and energy for the big important things, and prioritize them over tiny pebbles and sand that you could do afterwards).

There's some generalized motivation advice out there (like this list of 20 tips), but that is a bit more general than this question, since maybe motivation-enhancing techniques deal with changing the goal to a more palatable one, whereas the discipline under discussion here is mostly about manufacturing indirect motivation for goals.

Of these techniques, I think the cluster that this crowd would benefit from the most are the various social ones--surround yourself with people who care about the thing you want to do, who will be disappointed if you don't do that thing, and so on. Beeminder is a way to disappoint yourself if you fail (the part I use it for is mostly to remind myself to do things that don't otherwise fit into a routine nicely), but the way you have it set up where it emails your advisor if you fail is a good example of using social opprobrium as motivation.

Paul Graham claims that startup founders seem way more motivated by fear of failure than by lust for success, and this seems like a primary but ugly component of discipline.

Music seems to have significant effects on mood and motivation. (This may be a less useful recommendation for you because of the tinnitus, but maybe not.) Here's a playlist I came across recently that's optimized for programming, for example.

Being inspired by or emulating others is comparable to the "surround yourself by people you want to be like" technique described earlier. I've found reading biographies useful for tips and motivation. You in particular would like Lives of the Engineers, I think, but in general any historical figure that seemed like a badass probably has a good biography written about them. (Theodore Roosevelt, for example, famously gave a speech after being shot, and I found The Rise of Theodore Roosevelt an engaging read.)

I've found fictional characters useful for this as well; asking questions like "would Zuko give up now?" is often good for an additional rep or two while exercising, say.

It also seems like this may be the sort of thing where poetry / quotations / etc. may be helpful.

On the topic of character in general, check out KIPP's list of seven desirable character traits.

Nate's series of blog posts on why to avoid being motivated by shame or guilt is a strong one, and highly related to avoiding the failure modes of many of the discipline techniques that I'm talking about, as well as positive alternatives.

It's also worth pointing out that most things are marathons, not sprints. (This is a recurring theme of Duckworth's discussion of grit, and a major feature of Nate's most recent blog post.) Being able to tell the difference between depleting renewable resources and nonrenewable resources is a core part of disciplined effort over a long period of time.

[-]FourFire9y20

I would also like to see these resources. A cursory search yields this, however I don't think it is sufficiently indepth.

[-]Vaniver9y50

I responded. Incidentally, Less Wrong has a neat feature where if you click the envelope icon at the bottom right of a comment, you'll get replies to that comment in your inbox as if you had written the comment yourself.

[-]Michael Wiebe9y40

I was astonished to find myself having ascended to the pantheon of those who have made major contributions to human knowledge

Is this your own evaluation of your work?

[-]JonahS9y10

Yes. The basic situation is that I figured out how the methods that Charles Spearman used to discover IQ can be used to shed a lot of light on many different psychology and sociology questions. This is what I was implicitly getting at in my sequence of posts on my Speed Dating Project, though I did a poor job contextualizing the results. IQ is by far the most robust construct to come out of psychology research, so this could in principle revolutionize social science (with a huge amount of work by many talented people).

Some people would say that psychology researchers used the methodology to discover the Big 5 Personality traits among other things, but the constructs that they developed are relatively weak because they haven't utilized the full power of Spearman's framework. See also Sarah Constantin's post A Yardstick for Smell: Thinking in PCA (she preempted me in recognizing the potential of the methods 4.5 years before I did, but I didn't fully understand what she was getting at at the time).

It's difficult to make such claims without coming across as grandiose and/or arrogant, so I should emphasize that I think that many LWers would be capable of doing such research with the right background and style of research, if they took Paul Graham and Steve Jobs' advice. The great mathematician Alexander Grothendieck said:

In our acquisition of knowledge of the Universe ( whether mathematical or otherwise) that which renovates the quest is nothing more nor less than complete innocence. It is in this state of complete innocence that we receive everything from the moment of our birth. Although so often the object of our contempt and of our private fears, it is always in us. It alone can unite humility with boldness so as to allow us to penetrate to the heart of things, or allow things to enter us and taken possession of us.

This unique power is in no way a privilege given to "exceptional talents" - persons of incredible brain power (for example), who are better able to manipulate, with dexterity and ease, an enormous mass of data, ideas and specialized skills. Such gifts are undeniably valuable, and certainly worthy of envy from those who (like myself) were not so endowed at birth," far beyond the ordinary".

Yet it is not these gifts, nor the most determined ambition combined with irresistible will-power, that enables one to surmount the "invisible yet formidable boundaries " that encircle our universe. Only innocence can surmount them, which mere knowledge doesn't even take into account, in those moments when we find ourselves able to listen to things, totally and intensely absorbed in child play.

[...]

In fact, most of these comrades who I gauged to be more brilliant than I have gone on to become distinguished mathematicians. Still, from the perspective of 30 or 35 years, I can state that their imprint upon the mathematics of our time has not been very profound. They've all done things, often beautiful things, in a context that was already set out before them, which they had no inclination to disturb. Without being aware of it, they've remained prisoners of those invisible and despotic circles which delimit the universe of a certain milieu in a given era. To have broken these bounds they would have had to rediscover in themselves that capability which was their birth-right, as it was mine: the capacity to be alone.

[-]ChristianKl9y00

In my bioinformatics studies we did PCA in statistics 102 and I got the impression that it's a commonly used statistical tool. Do you think people generally do PCA wrong, or is your complaint that it's not used enough?

[-]JonahS9y00

I know that many researchers know something about PCA. I do think that it's not applied nearly enough (c.f. Sarah's remarks about Asperger's Syndrome, which was removed from the DSM a few years after she made her post). The main issue to my mind is that when people apply it in psychology they seem to come into it with preconceived notions concerning what they might find, rather than collecting large and diverse datasets, letting it speak for itself, and then trying to interpret what the principal components mean in human terms.

Consider the construct of conscientiousness. It's very suspicious that it maps onto a prexisting notion, and it's just not that predictive. I got lots of C's and D's in school, but worked 90 hours a week for 12 weeks on my speed dating project. Am I conscientious? ;-) As far as I can tell, they came up with questions based on preconceived notions, then did factor analysis, and came up with a construct that some meaning, while being very far from carving reality at its joints.

[-]ChristianKl9y20

The DSM is a mess but I think the problems isn't that there aren't people who understand PCA. There are political reasons inside of the American Psychiatric Association that led it to use definitions that aren't data driven.

It seem to me like to make major contributions to human knowledge you need to do a lot more than say: "Hey PCA is really great". You actually have to understand reasons of why people aren't using it and fixing those reasons.

100 is over a hundred years old, there have been a lot of people thinking that it should be used more. I think I have argued in the past in various time for PCR. The last time was when talking about the design of http://www.omnilibrium.com/ and how it should find factors for political labels via PCR instead of just using the left-right framework. I think I made the same argument for LW census political labels.

It's very suspicious that it maps onto a prexisting notion, and it's just not that predictive. I got lots of C's and D's in school, but worked 90 hours a week for 12 weeks on my speed dating project.

You say that it's not predictive for the preexisting notion. That doesn't mean that various things haven't been predicted with it. Big Five ratings have been predicted via analysing facebook posts.

[-]JonahS9y00

It seem to me like to make major contributions to human knowledge you need to do a lot more than say: "Hey PCA is really great". You actually have to understand reasons of why people aren't using it and fixing those reasons.

Have you read my speed dating project posts? I haven't yet written up the most important one on demographics (I can do that soon, just many conflicting priorities), but the one on individual variation in revealed preferences for attractiveness vs intelligence and sincerity starts to get at what I'm talking about.

My project gives a proof of concept for what I'm talking about in the context of social psychology. I've never seen such an application. So no, it's not just the realization that it could be applied, it's also giving a proof of concept: that's why it took ~1500 hours rather than ~10 hours.

As far as I can tell, the situation is simply that deep knowledge of the technique hasn't yet percolated into the social psychology community, and people who do have the relevant background knowledge haven't actually tried doing social psychology research. All you need is to notice something that's been missed. There are many such things (see Peter Thiel's discussion of how there are still secrets in his book "From Zero To One.")

If I recall correctly, Freeman Dyson has indicated that his demonstration of the equivalence of the two different formulations of quantum electrodynamics isn't as amazing as people believe, but was largely a function of him being one of the first people to learn both formulations! :-)

So I'd strongly encourage you to pursue your ideas more. I've been looking some at the General Social Survey data, where I haven't yet found something highly nontrivial (maybe I'm looking at the data the wrong way, or maybe it's just not a good dataset for this). I'd be happy to share my code with you / a cleaned form of the data, if you're interested in exploring factors for political labels.

[-]ChristianKl9y00

It might be that I have gotten to cynic but if you measure 6 variables it's more likely that one of them get a statistical significant result then if you first turn those 6 variables into 2 variables via PCA.

My project gives a proof of concept for what I'm talking about in the context of social psychology. I've never seen such an application. So no, it's not just the realization that it could be applied, it's also giving a proof of concept: that's why it took ~1500 hours rather than ~10 hours.

That probably where there's something I don't understand. I don't understand why the analysis took ~1500 hours. Spending that much time with a dataset also instinctively triggers "fishing expedition" in my head. I don't know to what extend that's warranted.

I'm not sure that you have shown that it makes more sense to interpret that factor individual preference is about intelligence and sincerity than that it's about the value of fun.

As far as I can see it could also be that fun&physical attractiveness is simply more valued.

So I'd strongly encourage you to pursue your ideas more. I've been looking some at the General Social Survey data, where I haven't yet found something highly nontrivial (maybe I'm looking at the data the wrong way, or maybe it's just not a good dataset for this). I'd be happy to share my code with you / a cleaned form of the data, if you're interested in exploring factors for political labels.

In the case of the spending effort on the GSS I can't envision what success looks like. It's straightforward to find PCR factors but I don't know how to put them to good use.

A more interesting project would be to explore LW's ideological landscape. It would be very interested in how various rationalist beliefs interact with each other. Does seeing yourself as an "aspiring rationalist" correlates to beliefs on UFAI risk?

Having a project that searches where the main dimensions of disagreement in this community would be valuable. Maybe 300 questions that are answered on a Likert scale. Maybe 150 rationality questions, 100 big 5 questions and 50 autism questions.

[-]JonahS9y00

It might be that I have gotten to cynic but if you measure 6 variables it's more likely that one of them get a statistical significant result then if you first turn those 6 variables into 2 variables via PCA.

Yes, this is the point :-)

[-]JonahS9y00

That probably where there's something I don't understand. I don't understand why the analysis took ~1500 hours. Spending that much time with a dataset also instinctively triggers "fishing expedition" in my head. I don't know to what extend that's warranted.

The issue of multiple hypothesis testing is precisely why it took 1500 hours :-). I was dealing with the general question "how can you find the most interesting generalizable patterns in a human interpretable data set?" It'll take me a long time to externalize what I learned.

For now I'll just remark that dimensionality reduction reduces concerns around multiple hypothesis testing. If you have a cluster of variables A and a cluster of features B and you suspect that there's some relationship between the variables A and the variables B, you can do PCA on the two clusters separately, then look at correlations between the first few principal components rather than looking at all pairwise correlations between variables in A and variables in B.

A more interesting project would be to explore LW's ideological landscape. It would be very interested in how various rationalist beliefs interact with each other. Does seeing yourself as an "aspiring rationalist" correlates to beliefs on UFAI risk?

There is the 2014 LW survey data, which is interesting, even if less substantive than what you have in mind. I have an unfinished project that I'm doing with it (got bogged down in cleaning it to make it nicely readable).

[-]Nornagest9y00

Consider the construct of conscientiousness. It's very suspicious that it maps onto a prexisting notion...

Is it? We've been modeling each other as long as language has existed. Conscientiousness might not correspond to a single well-defined causal system in the brain, but it would be no surprise to me at all to find common words in most languages for close empirical clusters in personality-space. And the Big 5 factors are very much empirical constructs, not causal.

[-]JonahS9y00

Ok, I guess what I mean is that it's suspicious that it maps onto a preexisting notion held by the general population, in the same way that it would be suspicious for psychology research to apparently show the existence of demon possession (which humans have in fact believed in). I wouldn't find it suspicious if it mapped onto a notion of someone with demonstrated exceptional ability to read and connect with people (e.g. Bill Clinton).

The way scientific progress occurs is by developing progressively more refined understandings of what's going on: for example, passing from the Ptolemaic model of the stars and planets to the Copernican model to the Newtonian model to Einstein's theory of general relativity. One can't hope to understand reality if one isn't flexible enough to recognize that things might be very different from how they initially appear.

[-]Vladimir_Golovin9y30

I fully agree with this. Here's a good quote I found on the web:

If you really deeply care about something, you will do it. You will do it without needing a list or a system or a reminder.

(Source)

[-][anonymous]9y20

Their problems come from them spending their time trying to do work that they find boring. If you find your work boring, it's very likely that you should be doing something else.

Because the market is marvelously set up so that for every single thing any human being may want to do, there are buyers. Allow me to stay skeptical here.

Finding likeable AND marketable things to do is an art in itself.

Here is how it tends to actually happen (esp. if you live in the more credentialist type of cultures): you study something because it looks like a good idea at that time. You are lucky and manage to get hired without experience, and gain experience. From that on, nobody will ever hire you for anything else, ever, because you are mentally boxed into that category.

For example there is this woman who went to gastro school and worked for 12 years as an industrial or breakfast cook, waitress, these kinds of gastro jobs and got really fed up with it. She always like plants, and she was the kind of rural girl who grew up gardening/farming so she figured she could work at that kind of company, raising flowers, it is just a generic skill every rural person can do. I even helped lied a bit on her CV - renamed her experience at her parents rural garden/farm as experience at a local flower and vegetable raising business. Still nobody hired her for a year, people just looked at the resume and did not find an agriculture school degree + they were like what does this cook/waitress want here after 12 years of doing that kind of job? There were not even answer after about 40 e-mailed applications (there weren't more of these business around) nor from the city (applying to take care of flowers in public parks) so she sighed, gave up the dream and went back to greasy world of industrial kitchens.

This is how it usually works. In most industries there are more than any capable people with proper degrees and even relevant experience. They will not hire a profession-hopper.

I mean, OK I would hire half of LW into an ERP software job even if their background is something like medicine because I know they are smart and would learn it in 3 months, but that is different, the difference is knowing they are smart, having this semi-personal knowledge. But the typical job application process does not convey it. It is just a name and credentials and list of experiences.

[-]somervta9y20

Formatting note: something seems to have deleted a couple of your time units:

I spent ~1500 working on genuinely original scientific research.

and

which subsumes the 100 old Poincare conjecture,

[-]JonahS9y20

Thanks, fixed.

[-]Shmi9y10

This matches my personal experience pretty well. Unfortunately, finding this low-hanging fruit in the area you are interested in, and make it pay bills, is a non-trivial problem. If there was someone really good in solving this particular issue for people, at a modest fee or commision, they would make a killer startup. Certainly this seems like a (meta-)low-hanging fruit.

[-]Lumifer9y20

I don't know if it's low-hanging. In my experience finding what you are REALLY interested in involves a lot of trial and error, while aptitude tests are worthless beyond the pretty obvious division into techies and humanities people.

[-]FourFire9y00

I am of the opinion that much better tests can exist, not sure if they do, which can divide people into further categories.

[-]Vaniver9y20

Certainly this seems like a (meta-)low-hanging fruit.

There seem to be fairly effective interest / aptitude tests, that I remember taking in middle school. The level of detail is insufficient to identify low-hanging fruits ("you should go work in this particular research group") but is still narrow enough to be useful ("these are the jobs that the 0.1% of the population most like you finds most satisfying"). This can raise to attention jobs or fields that are high impact or high paying that someone might not be consciously aware of, or realize that it suits them well. (The canonical example here is MRI sales; someone has to sell them, but a person with strong ability to remember facts and figures and understand technical things, and also to deal with people, may not have that come to mind as an intersection of their interests and aptitudes.)

[-]FourFire9y00

Are any of these open/free to take, because I have use for such a test.

[-]Tem429y10

I don't know of any that are free, but What Color Is Your Parachute? is generally recognized as a good one, and is likely to be available at most used book stores -- though would probably have to settle for an older edition.

Edit: and any library will have a copy.

[-]btrettel9y10

In my view, a lot of the lack of "low hanging fruit" many people see is working in the wrong field. Some fields (for example, computer science) are extremely saturated with very bright people. As Lumifer suggested, sampling different potential interests is a reasonable strategy. I've been surprised by the random things that turned out to be fascinating to me.

Many opportunities exist in fields where certain knowledge is useful but rare. For example, there are many fields where math and coding knowledge could make a big impact. I know someone who got a PhD in math and then started working on problems related to fire protection. The specific application is not so critical, but the rarity of their ability in that field is. They seem to love their job, and I think this is a generally useful strategy. You just need to find fields in the intersection of problems of sufficient interest to you and problems your skills and knowledge can help solve. Some other examples: math and programming knowledge could be super useful in psychology and medicine.

Part of the problem here is a hidden constraint that I see from many people. Many smart people seem to believe many fields are below them, and they won't consider them because of this. You should see the fields that other people are avoiding for this reason as opportunities.

In terms of finding these sorts of opportunities, I don't think there's a good general procedure other than seeing how your current knowledge could fit into any field you encounter, and taking action when you see opportunity.

[-]SatvikBeri9y00

I found my motivation to frequently depend much more on external factors than the task itself. For example, I worked on Math for essentially every waking hour at Berkeley (and completed all the Master's courses when I was 19 as a result), and I worked on programming/data science tasks 80 hours/week at one job. But I've had very similar types of tasks in different environments and found it quite difficult to put it anything near the same number of hours.

I've found that my motivation seems to be constraint-based: if I feel like I'm getting enough sleep, enough socialization, enough food etc. then I find it very easy to work a lot. But if any one of these is lacking then my motivation plummets. In particular, all the environments where I worked exceptionally hard were ones where I was surrounded by people who felt like part of "my tribe": simply being around people whom I like isn't enough.

Moderation Log