Request for advice:

Like many people on lesswrong, I probably lie towards the smart end of the bell curve in terms of intelligence, but I'm starting to suspect that I lie somewhere below the mean in terms of ability to focus, concentrate, and direct my attention.

I only recently became concerned about this because it wasn't much of a problem when I was in school. There, I was able to do acceptably well overall by doing well in the subjects that came easily to me without working hard (science, maths... you know the score) and mediocrely in those that didn't. Ditto my undergrad/bachelor's degree.

But I'm currently struggling rather with the thesis project for my master's degree in computer science. The specifics of the thesis itself don't matter, other than that it's a piece of empirical/numerical research involving a lot of coding and a prose write-up. None of the technical aspects of it are beyond me, and yet I feel like in some way it's the first very difficult thing I've ever tried, really tried, at. The hard part is sustaining interest over the whole length of the thing, planning and organizing the overarching, erm, arc of the project as a whole, and forming a 'narrative' out of all the hard-won bits and pieces of data. (I suppose the fact that I feel fairly sure that the project is likely to find a negative result (i.e. that the method under inspection doesn't offer any gains over simpler methods) also doesn't help my motivation.) Luckily, I did well enough in the taught part of my course that I only need to get a mediocre mark in this part in order to get a 'merit' overall.

But I'm also concerned about how this bodes for my future career. I'd like to do well in work, but I'm beginning to wonder whether I'm deficient in a skill which would allow me to do much better.

To convey what I'm talking about: often when I'm trying to work at home I flit between coding for work, reading, coding for fun, listening to music, etc., etc., etc., and consequent don't engage with any of them very deeply, or get much done. Also, I have almost always taken a very long time to get to sleep, often an hour or more, because I find it hard to 'switch off' my brain when I'm in bed and have decided it's time to go to sleep. (I've recently been making the paradoxical attempt to try very hard to switch my brain off and stop thinking in bed, with, surprisingly, some limited success.)

I feel like I lack the five-second level skill to suppress (or at least, to decline to pursue) any old interesting thought which appears while I'm doing something else.

Things I've attempted:

• Meditation. It seems plausible that meditating could help to 'train' deliberate attention direction in other aspects of life. Does anyone have any experience with this? I tried checking the literature, and found only one weak-ish study supporting such a hypothesis, but I'd be open to anecdotal evidence. I've tried to meditate a few times (less than ten), for about half an hour each time. The first couple of times I became weirdly aggravated and agitated at how bad at it I was: I was frustrated by the realization that something as simple as focusing on one thing and avoiding other thoughts was beyond me. After the first couple of times, I no longer find it aggravating, but I have yet to find it rewarding, either. I haven't yet managed to obtain the focused, quiet state which I understand is the aim, at least for more than a handful of seconds a couple of times. (Is this normal beginner-level performance?)
• Pomodoros. I've had some success with doing pomodoros of work, including beeminding them, but I find that they're best suited to well-defined, discrete tasks. Tasks which are more nebulous seem less suited to it. Also, I find it hard to do pomodoros unless I'm feeling high-willpower, but perhaps this is fixable with, erm, the application of more willpower.

Things I've considered but haven't attempted:

• Medication, self-. Is this the sort of thing which would be amenable to a course of Modafinil, or some other nootropic? I could be open to trying this, if it were likely to work.
• Medication, other. I could try seeing a doctor to see if what I'm talking about warrants a diagnosis of ADD, and a prescription of Ritalin or a similar drug. I have no idea whether what I'm describing would be considered drastic enough to warrant either of those, though.

Any experience with any of the above, speculation on which of them might bear fruit, or suggestions of completely different ideas welcome.

Is this the sort of thing that can be 'trained' through willpower? It seems like a fairly 'deep', even a fundamental, aspect of brain function, so I wonder how plastic such a thing is. Any thoughts on this welcome also.

Finally, am I just worrying too much about this? I was recently heartened to come across this Nassim Nicholas Taleb quote:

If you get easily bored, it means that your BS detector is functioning properly; if you forget (some) things, it means that your mind knows how to filter; and if you feel sadness, it means that you are human.

Perhaps I just have a very stringent bullshit detector. Evidence in favour of this proposal: I think I am able to focus extremely well on personal projects (typically things that I code for fun and find intrinsically rewarding). In fact, when I stop those, it's less often from boredom and more by guiltily tearing myself away in order to get back to my "real" work. (On the other hand, perhaps there's such a thing as a too-stringent bullshit detector - one so stringent as to give false positives.)

Summary: I'm concerned that my focus/concentration skills are significantly worse than average, and that this could be detrimental to my outcomes in life. How can I improve them?