Self-Study Questions Thread

TylerJay

LESSWRONG
LW

17 Self-Study Questions Thread

29th Jan 2014

1 min read

17

There are a lot of autodidacts on LessWrong and many of us hold scholarship to be a virtue. Learning things on your own can be tricky for a number of reasons, but there is a lot of good material on LessWrong on how to do it efficiently. We know that it's usually best to build small skills in the right order, but sometimes it's difficult to figure out just what that order is, especially across disciplines. University programs have strict prerequisites that are supposed to ensure that you can handle a course before you take it, but taking a college course catalog's prerequisites to be absolute is an easy way to get demotivated before you learn anything you're actually interested in.

I've seen a lot of people run into trouble with self-study when they really want to study topic x, but know that they have to study topic y first as a prerequisite. (For example, I once saw someone on LW who wanted to start learning programming, but thought he couldn't get started without learning a bunch of math first.) For progress to be by accumulation and not by random walk, it is sometimes necessary to do this, but it can be hard to tell.

I wanted to create a thread where would-be autodidacts with limited knowledge of their target domain can ask these kinds of questions so that they can better plan their courses of study. Whether it's questions about specific texts, recommended prerequisites, MOOCs, course overlap, whatever.

Any general discussion about self-study is welcome too.

Personal Blog

17

Mentioned in

11Proposal: LW courses

New Comment

66 comments, sorted by

top scoring

Click to highlight new comments since: Today at 3:23 AM

[-]solipsist11y120

Life tip: If you want to study X but aren't sure if you need to study Y first, just start studying X.

If you read a textbook is too hard, you will quickly discover and correct your mistake. If you read a textbook is too easy, you risk wasting months on material below your capabilities, and you won't find out until you move on to the textbook you should have started with in the first place.

[-]Shmi11y50

If you read a textbook is too hard, you will quickly discover and correct your mistake.

Not in my experience. It's pretty hard to figure out what you don't know.

[-]solipsist11y20

Could you give a specific example of what that failure mode looks like? I've either never experienced it or never noticed myself experiencing it.

EDIT: Would you agree with my advice when reading textbooks with exercises? That is, where your understanding of the material can be falsified with incorrect answers?

[-]Shmi11y80

Ideally the learning process should follow the Prerequisites from Abstruse Goose linked elsewhere in the comments (hopefully except for the last panel). In reality, however, there is an overwhelming temptation to just look up the unfamiliar concept on Wikipedia and plow on. As a result, the incomplete understanding quickly compounds and it becomes impossible to make any meaningful progress.

As for exercises, yes, one has to do all the exercises in textbooks, and then some. Inability to solve some of the chapter problems or taking too long to do it is a good indication of missing some prerequisites.

So, to modify your advice, "If you want to study X but aren't sure if you need to study Y first", get a Y textbook and make sure you can solve or at least set up all or most of the exercises in it.

[-]TylerJay11y40

This disagreement was the point I was trying to make. That sometimes, the right advice is to just get started, such as the guy who thought he couldn't start programming without learning a bunch of math first that I mentioned in the OP. I actually advised him to just start and backfill later. However, in other cases, it definitely seems right to learn things in order.

I suspect at least one of the failure modes related to not learning things in the right order is slower, more arduous learning that isn't fun. If you struggle with the whole thing and look up the requisite material as you go, you don't get a sense for which parts of what you're learning are interesting results of that topic as opposed to just requisite knowledge gaps. You don't have the experience of thinking "that doesn't sound right..." and then having your confusion cleared up as you keep reading and it is explained. Instead, everything sounds foreign and you don't appreciate which parts of it are the interesting results of the topic at hand. This leads to poor retention and a low ability to apply what you've learned, even if you manage to make it through and "feel like you understand it."

[-]ThrustVectoring11y00

I suspect at least one of the failure modes related to not learning things in the right order is slower, more arduous learning that isn't fun.

And if you want everything to make sense anyway, you're going to learn the prerequisites anyways. So not only are you having more difficulties now, but you're spending time getting a tenuous grasp on a subject that you'll want to revisit later anyways once you can put things into a coherent framework.

[-]A1987dM11y00

Could you give a specific example of what that failure mode looks like? I've either never experienced it or never noticed myself experiencing it.

See the post “Double Illusion of Transparency”.

[-]Shmi11y70

Autodidacts beware.

[-]Vladimir_Nesov11y100

Original link (at Abstruse Goose).

[-][anonymous]11y00

[-]newerspeak11y00

Newton gave out reading lists like that too. Geniuses are always the worst teachers.

[-]Daniel_Burfoot11y50

As a strategic note, I am increasingly skeptical of the idea of trying to learn a bunch of skills that you may or may not use, especially advanced math skills. The key problem with learning is that skill ability decays over time unless you invest a lot of time on maintenance. So if you invest X hours learning a skill, you will have an asset whose value decays over time, while if you spend the same amount of effort making money, you will have an asset that grows exponentially in value over time.

[-]NoSignalNoNoise11y20

Real interest rates are negative, so currently, money doesn't grow in value over time. A better investment would probably be to learn skills that you do plan on using.

[-]Dias11y10

The expected real return on the stock market is positive. You shouldn't hold large cash balances.

[-]ChristianKl11y00

The expected real return on the stock market is positive.

The return that you expect might be positive but in general different people can have different beliefs about what they expect the SAP500 to do in the next ten years.

[-]Dias11y20

The expected return implicit in futures rates is positive. If you disagree you should short futures and earn a positive return that way.

[-]ChristianKl11y-10

No. Conservative investment strategies aren't about buying SAP500 and buying shorts on it.

They are about buying treasury bonds. Some rational players in the market do buy bonds. If you want at the expected return of the market you have to look at treasury rates given inflation adjustment.

Quick googling reveal 2.65% for treasury bonds. Official inflation rates are 1.5%.

That means expected return on your money is 1.15%. You could however say that the official inflation numbers are messaged and the market is generally too optimistic. The difference from 1.15% to negative expectations isn't that high.

If you think the market is off by 1.2% you don't gain a high return by betting on corresponding futures.

[-]TylerJay11y50

A few questions that I've had recently:

Do you really need multivariable calculus for probability theory?
Would taking a traditional statistics course be a good place to start learning statistics and probability theory, or do they tend to teach a "bad frequentist mindset" that later has to be unlearned?
Is Cormen et al. suitable as an introductory algorithms text?

I've seen differing opinions on this one, so I wanted to make it a poll:

Is Jaynes suitable as an introductory probability theory textbook? [pollid:594]

[-]Qiaochu_Yuan11y50

"Probability theory" is an extremely broad term. You don't need any calculus to understand discrete random variables (and if your motivation is directed towards MIRI research, this is mostly the kind of probability that's relevant). You'll want to know some one-variable calculus to understand continuous random variables (so you can manipulate probability density functions and maybe characteristic functions). It would be helpful to be comfortable with some multivariable integral calculus for the purpose of understanding collections of continuous random variables, but something like Stokes' theorem won't be particularly relevant.
My impression is that most traditional statistics courses are very bad (not even because of anything to do with frequentism, they're just very bad). Don't take one.
Can you clarify what you mean by this?

[-]TylerJay11y10

Thanks, that's really helpful.

For #3, I was basically looking for some feedback on this text. It was recommended in the best textbooks on every subject thread, but I've heard mixed reviews about it. The top rated review on amazon claims

Students will need a very strong mathematical background and a strong arm to even think about picking up this book because the it is heavy (both physically and metaphorically). Mastery of discrete math is a must, graph theory, programming, and, combinatorics will also help.

and another review by a Comp Sci professor calls it "Magisterial, and Impenetrable".

These are not things you see written about something like Sedgewick's book. Interestingly enough, the reviews on this one all similarly praise Sedgewick's book over Cormen.

So I guess my question was "Is Cormen et al. actually a good introductory text on algorithms for someone who has not taken an algorithms course before?"

[-]Anatoly_Vorobey11y50

The answer is "probably not". Cormen is too comprehensive and dry for self-study; it's best used as the textbook to back an algorithms course or as a reference to consult later on.

A very good book is Skiena, The Algorithm Design Manual. I usually recommend it to people who want to brush up on their algorithms before programming interviews, but I think it's accessible enough to a novice as well. Its strengths are an intelligent selection of topics and an emphasis on teaching how to select an algorithm in a real-life situation.

[-]gjm11y20

How much of a mathematician are you? I would recommend Cormen et al on its own only to people with a strong mathematical background and plenty of talent. I like the idea of Cormen et al + Skiena together (and I'm pretty sure I said so in that thread) but don't have actual evidence on how well that works in practice.

[-]Qiaochu_Yuan11y20

Ah, sorry, I was referring to your question about Jaynes (that question was meant to be numbered with a 4). What are you looking to get out of an introductory probability textbook?

[-]ThrustVectoring11y00

My impression is that most traditional statistics courses are very bad (not even because of anything to do with frequentism, they're just very bad). Don't take one.

That's my impression as well. I've had to take a statistics course, and I wound up memorizing procedures enough to vomit it up on tests on homework. There wasn't really a coherent framework, so the least work method of passing the course was memorizing the teacher's passwords.

If you have to take a traditional statistics course, don't expect to get a solid understanding of statistics out of it. Perhaps you'll know how to use certain orthodox analysis methods.

[-]solipsist11y40

Traditional statistics classes teach procedures, but not the math needed to understand them. Being told to use maths without being told how the maths work is infuriating to certain sorts of people, and those sorts of people are abnormally concentrated on LessWrong. Jaynes builds up procedures from first principles, which appeals to some but is pointlessly abstract for others.

Different minds learn differently. The answer to this poll entirely depends on who you are.

[-]Stabilizer11y30

No.
I don't know.
Yes.

[-]NoSignalNoNoise11y00

It is worth pointing out that probability and statistics are generally considered to be separate, although closely related topics. Because statistics is built on probability, you are probably better off learning probability first, and then statistics.

[-]James_Miller11y40

I'm looking for a non-language specific book on proper computer programming techniques.

[-]protest_boy11y60

I highly recommend the book Concepts, Techniques, and Models of Computer Programming (http://www.amazon.com/Concepts-Techniques-Models-Computer-Programming/dp/0262220695) which is the closest I've seen to distilling programming to its essence. It's language agnostic in the sense that you start with a small "kernel language" and build it up incorporating different concepts as needed.

[-]Dr_Manhattan11y20

The book's author is running a MOOC on EdX https://www.edx.org/course/louvainx/louvainx-louv1-01x-paradigms-computer-1203

[-]Lumifer11y40

You'll have to specify your needs a bit better.

Are you interested in algorithms (in which case you'd want to take a look at Knuth)? In the proper way to architect or organize software (in which case Design Patterns might be useful)? In how to write clear, understandable, efficient code (in which case language will have to enter into this)?

[-]Shmi11y30

I generally recommend Code Complete by Steve McConnel for procedural and OO programming, less so for rapid scripting, even less suitable if you are into functional programming (where SICP rules supreme). For ways to deal with the common issues on a software development team, see his other books, Software Project Survival Guide. Rapid Development is also good.

[-]Antiochus11y20

Completely language-agnostic programming book is a bit of a contradiction. You're better off finding books on the subject that use the language you're most comfortable with. I'd look at books on object oriented programming and algorithm design, and when you're completely comfortable with those topics, move on to design patterns.

[-]TylerJay11y00

What level are you at with programming and what kinds of techniques? Algorithms and data structures? Design patterns?

[-]James_Miller11y20

Relatively basic, at a level just above codecademy.com. Mostly I'm trying to help my young son become a better programmer and I don't have the language to do it when, for example, he makes things much more complex than necessary.

[-]solipsist11y20

Code: The Hidden Language of Computer Hardware and Software is a good book for a precocious ~12 year old.

[-]James_Miller11y00

Thanks, I just ordered it.

[-]adbge11y40

Does anyone have an idea of the prerequisites necessary for Information Theory, Inference, and Learning Algorithms or Introduction to Economic Analysis?

[-]Stabilizer11y30

Skimming through these two textbooks, it seems that:

1) Information Theory... would go down a lot easier if you are already comfortable with probability theory. Nothing fancy, but being able to solve basic "balls and urns" problems, which require some knowledge of combinatorics. Also, knowledge of linear algebra would help a lot.

2) ...Economic Analysis seems to require comfort with calculus. Comfort with reading, manipulating and understanding graphs---similar to the kind you encounter in Newtonian kinematics---would also go a long way.

[-]Dahlen11y30

I am an aspiring autodidact in math who's having motivation issues and would like to identify their cause. I've been, embarrassingly, stuck at basically the same high-school, introductory level for a few years now in all of the material I've begun studying so far, and this is because I've been studying with a frequency ranging from one week per month, to once every three months or something. My chosen study methods are slow and thorough.

I often feel frustrated about the material I'm studying, and find it rather tedious. I suspect that one reason for this is the fact that I'm a freshman undergrad in a STEM field, so I've already gone through all of what I'm studying now and more (albeit more superficially), have been acquainted with more difficult math and would really like to get into the meat of things -- however, I'm aware that I don't have a solid grasp on the basics and need to practice even the easy exercises until it becomes second nature to me.

I often wonder whether my struggles with finding the motivation to go through my textbooks are an ordinary part of the life of people who eventually overcome them (through better study plans, or increased conscientiousness or whatever) and go on to become good mathematicians, or whether it just means that I'm simply naturally disinterested in math, and will always be so, and consequently I'll never raise above the level of a mediocre STEM undergrad. I mean, if I were cut out for math, then I'd be naturally curious and enthusiastic about studying it, right? Or at least I'd have the conscientiousness / work ethic to study diligently even when it doesn't feel exciting. So far none of these conditions apply; does it mean that they never will? (I expect this feeling of tediousness to go away once I start to become good at college-level math. But maybe there's a selection effect at play here: only people who are naturally good at math master the undergrad level; people who naturally suck at math give up by the end of high school.)

I've also noticed that my lack of motivation is not domain-specific: I fail to habitually study every other field I've ever meant to study. In fact, I've been comparatively successful in studying math (and, to a lesser extent, physics and programming). This leads me to thinking the issue is a lack of study habits. The funny thing is that I constantly -- habitually, even -- fret about lacking study habits, yet so far (3 years or so) this has never translated into the instillation of said study habits.

Have any LessWrongers had similar issues and managed to overcome them?

[-]iarwain111y30

For introductory math, I've enjoyed the Art of Problem Solving series. I too prefer the slow and thorough method of study, and these books are really comprehensive.

For study motivation, try the Akrasia Tactics thread, as well as these posts.

For myself, I've found that the best study motivator is to attend a class on the subject, even if that means paying for it. In fact, paying for it is an extra motivator to do the necessary work.

I'm actually in a very similar situation to you. Several of the links above were provided by Jonah Sinick over at Cognito Mentoring. I'm in the middle of an email conversation with him and he's been very helpful.

[-]marchdown11y00

Yes, I have had similar issues and I can't say that I did manage to overcome them successfully, but I'm committed to continue, and I'm in the middle of reorganizing my life so that I could direct more resources there.

I'm not offering any specific advice for now, beside the obvious: http://www.sparringmind.com/changing-habits/, http://www.sparringmind.com/productivity-science/, but I'm responding here to start the dialogue and to nudge us both in the right direction.

So yeah, I wish for you to untangle your motivations and follow through.

[-]Dahlen11y00

Thanks for the reply. I've tried just about any productivity technique except for Beeminder, which I'm fairly sure just isn't the right tool for me for a number of (financial as well as psychological) reasons. I've read The Willpower Instinct and a few other books on the topic. I'm currently looking into nootropics and neurochemistry as a last resort, more precisely dopaminergics, more precisely selegiline (I seem to fit the profile of someone who's generally low on dopamine, but I don't know for sure); modafinil too, for focus reasons.

The results have led me to think that I have positively affected plenty of areas of my life, except for the one I originally set out to improve, i.e. studying. Maybe I'm doing that wrong; maybe copying down entire textbooks is tedious as well as disheartening.

How does your case sound like?

[-]Dagon11y30

People vary quite a bit, so this may only apply to a subset of readers. It definitely applies to me. I learn by working backwards much better than by finding a path from basics to an end result.

Also, for a whole lot of topics, "learn X" or even "study X" is not a well-formed intent. There are often an incredibly wide range of activities you could undertake that seem to be leading to your goal(s), but are actually part of a completely different result that happens to share the same name "know X".

Don't attempt to study machine learning. Pick an outcome you want (perhaps building a LessWrong comment classifier), and study enough to get that done. Then pick another outcome, and do that.

Don't attempt to study quantum mechanics. Instead, try to figure out why you should care about the many-worlds interpretation.

[-][anonymous]11y100

Instead, try to figure out why you should care about the many-worlds interpretation.

This strikes me as terrible advice. You're advising people to exclusively read filtered evidence on a topic they know nothing about.

[This comment is no longer endorsed by its author]Reply

[-]solipsist11y40

Don't attempt to study machine learning. Pick an outcome you want ... and study enough to get that done.

For the record, machine learning is a topic where (1) it's easy to fool yourself with over-optimistic test result (2) it's easy to waste months improving parts of a project which do not matter in the end . These problems are both mitigated (somewhat) by learning the foundations.

Not to say you wouldn't learn a lot by just diving in. Diving in, then going back and figuring out what you were doing seems like it could be a good thing for many people.

[-]Petruchio11y10

My newest interest is to become a winning poker player. I have recently recented advice doing so professionally isn't quite feasible, since the sanity waterline has risen in the online poker world. However, I am not fully discouraged and I wish to pursue the skills needed to become an above average player, if not a professional one. Also, I am considering brick and mortar poker games in addition to online games.

I have been having some trouble calculating pot odds and implied odds. I grasp what they are conceptually, but actually calculating them has been a bust for me. These skills are needed for value betting, and what I have found online so far has been difficult to understand.

[-]Jayson_Virissimo11y10

As I mentioned here, I am engaged in self-study to become a software engineer. I now have a (tentative) 3 year (roughly 2^10 hour) plan for learning the relavent skills. You can take a look at what I'm doing and track my progress here.

Currently, it includes learning touch-typing, math, logic, computer science, Java, C, Ruby, Haskell, SQL, HTML/CSS, Unix, and getting a degree in computer science with a concentration in software engineering.

The idea is to read and do all the exercises in the texts listed under each goal and move onto small personal and open source projects for the remainder of the time.

I'm open to and would appreciate constructive criticism of the plan, book suggestions, relevant anti-akrasia techniques, etc... Also, before I get any comments about the "slow" pace, keep in mind I own my own business, have an almost 1 year old son, and am going to school full-time.

[-][anonymous]11y40

I think your plan is good, and I like the idea of having it visible (and on github as well - a requirement for software engineers these days).

You have 32 hours for HTML and CSS but you could also include some time prior to that on core design principles / Human Computer interfaces. It is one of those skills that while not directly related to building the software (interface is often done by a design team), will set you apart as a developer.

[-]Jayson_Virissimo11y10

How would one go about efficiently learning core design principles/human computer interfaces? Do you recommend any texts or online courses?

[-][anonymous]11y10

I recommend this book DESIGN creation of artifacts in society as this covers the design process more so than technical interfaces. It is very useful. The book is written by Karl T. Ulrich and follows the free course, which is very good in itself, but I wouldn't recommend the time commitment required for it.

If the course Human-Computer Interaction runs again I would recommend doing it. A fair time commitment but you go through the process of building web interfaces and peer reviewing them against the various heuristics.

In terms of the nitty gritty details like layout, font selection, colours, spacings, etc I used blogs, wikipedia and other ineffective ways of learning, but I would love to see a good textbook recommendation on the subject.

[-]Jayson_Virissimo11y00

Thanks.

[-]BartMan11y00

Hey. Your plan looks interesting and I'm wishing you all the best with your project on self-studying. If you want to learn about logic take a look at the online courses on coursera from the University of Melbourne : https://www.coursera.org/course/logic1 https://www.coursera.org/course/logic2

[-]CAE_Jones11y10

Somehow, the way my education jumped about, I got through high school, acing advanced topics in physics, astrophysics, and scoring high enough on the advanced placement calculus exam to skip some college courses, all without learning the cross or dot products.

I didn't do so well in college physics.

(I tried researching these online, but everything I found was either impenetrably technical or used an inaccessible format for mathematic notation. I finally got a hold of javascript functions, but I still wouldn't know when to use them.)

[-]pragmatist11y10

Div, Grad, Curl is a great introduction to vector calculus.

[-]TylerJay11y00

If you're still interested in learning, I found that physics 1 on MIT OCW did a pretty good job of explaining it.

To just learn the operations, this was a good reference. Dot product and cross product is explained about halfway through. here is the accompanying video where Walter Lewin (a charismatic and energetic physics prof) walks through actually doing it.

[-]NoSignalNoNoise11y10

What background material is prerequisite for Artificial Intelligence: A Modern Approach?

[-]solipsist11y50

That was the first CS I read, and was what convinced me to study Computer Science. There are no prerequisites.

EDIT: When I read that book, I had known Scheme) and calculus for several years. I don't recall either of them being required, but (outside view) they might have been. If you don't know how to program at all in any language, you may get lost. But in general that book does a very good job of exposition.

[-]Dr_Manhattan11y00

I'd claim that basic programming background, and high school math. Any math in AIMA that goes beyond high school is easily learnable on demand. If you don't know programming things will be difficult.

[-]somervta11y00

It would be helpful to be able to program, especially as most of the exercises require it, but it's not neccessary.. A basic grasp of math, particularly logic is also helpful, although the book does introduce them when neccessary.

[-]BartMan11y00

I'm having the exact problem that you're describing. I want to study computer science in a university here in Greece but to do that I have to give exams in subjects that are irrelevant to a computer science curriculum (anatomy,biology etc.) because in High School I was in a "medical" department. So now because I don't want to spend a year of my life studying those courses I can't study computer science in a university and I became autodidact , which is very challenging so far...

[-][anonymous]11y00

I am interested in recommendations on Maths textbooks with the aim on working in the AGI field.

I am on a focused learning drive at the moment and found these MOOC's to be particularly good - Calculus One, Calculus Two: Sequences and Series , Coding the Matrix: Linear Algebra through Computer Science Applications and Linear and Integer Programming .

So I have a basic understanding of Calculus and Linear Algebra and am not sure what area to focus on next.

[-]Qiaochu_Yuan11y30

Can you clarify what you mean by working in AGI? Is your aim, for example, to contribute to MIRI research? In that case, at least at the moment it seems like a good idea to learn some mathematical logic (enough to understand the incompleteness theorem, say) and some basic probability. On the other hand, if your aim is to contribute to creating AGI, then I'm honestly not sure if it's even a good idea on net to offer advice in this direction.

[-]Lumifer11y20

On the other hand, if your aim is to contribute to creating AGI, then I'm honestly not sure if it's even a good idea on net to offer advice in this direction.

Translation: if we tell you, we may have to kill you later

:-D

[-][anonymous]11y00

Thanks, my aim is to contribute to MIRI research in some way so I am skilling up in Maths in order to comprehend the various research papers I have come across.

On the other hand, if your aim is to contribute to creating AGI, then I'm honestly not sure if it's even a good idea on net to offer advice in this direction.

This seems to be a common response. Are AGI researchers encouraged to not talk about their work, or is it the UFAI risk that makes people hesitant to discuss specifics?

[-]Qiaochu_Yuan11y20

It's certainly the AI risk that makes me hesitant to discuss specifics (to the extent that I have any specifics to discuss). I don't know anything about the broader AGI community (to the extent that there is such a thing) other than the small subset of it I'm aware of through parts of the AI risk community, so I wouldn't be able to tell you what their norms are.

Moderation Log

66Comments