Cheers, thanks for writing. I was very anti-math high school student, almost got expelled for throwing a temper tantrum at my algebra 2 teacher cuz I thought it wasn't fair they were making me sit through it. That was 10th grade and they didn't make me take any other math courses. 7 or 8 years later I took the placement exam at a community college, placed into precalc I, retreated to khanacademy and retook the exam a few months later placing into calc I, took that and discrete and all of their sequels, ended up getting straight As and tutoring every single math course there. I feel like I owe a lot to khanacademy, a tailored user experience really adds a ton of value over throwing myself at textbooks (and I did eventually figure out how to throw myself at textbooks, but also failed at doing so many times).
The purpose of my comment is to register for anyone intimidated by comments they've seen that imply people in the movement were doing math at such n such level in grade school that we're out here, we exist, and we're doing stuff; we who had to put effort into precalc in our 20s.
That is hard to believe, you seem so smart at the UoB discord and your podcast :), thanks for sharing
Congratulations on the accomplishments! I did something similar a few years ago, using Spivak's Calculus rather than Apostol's (though I did consider using Apostol's).
I used to think similarly to you, that I needn't artificially reinforce concepts because those I actually need with be naturally reinforced in the wild, while those I don't won't, leaving me with only the skills I actually need.
However, this argument is subtly flawed. It assumes that concepts which are reinforced in the short-medium term are important, and concepts which are important are reinforced in the short-medium term. In fact, I found that often this is not necessarily the case, or at least not the case a sufficiently high proportion of times that I became annoyed and started looking for a solution. Many times, concepts which were only briefly touched on in the text, which I thought of as minor footnotes at most, were in fact the key to solving a problem I was stuck on. Similarly, often I would know a concept was talked about in a book, but not remember the details (since it had been so long since I studied the material), so need to go in and refresh my memory on the relevant section, re-reading >10 min worth of material.
Thus I recommend using Anki or some other spaced-repetition program to remember the concepts you learn. My proficiency in subjects I learned after using Anki is tremendously higher than my proficiency in subjects I learned before using Anki, because it allows me to have an immediate and incredibly fast register of everything I've learned about in the relevant field for basically no work (<30 min a day).
A genuine congratulations for learning the rare skill of spotting and writing valid proofs.
Graham’s Number I see as ridiculous, apparently one of the answers to his original problem could be as low as a single digit number, why have power towers on power towers then?
Graham's number is an upper bound on the exact solution to a Ramsey-type problem. Ramsey numbers and related generalizations are notorious for being very easy to define and yet very expensive to compute with brute-force search, and many of the most significant results in Ramsey theory are proofs of extraordinarily large upper bounds on Ramsey numbers. Graham would have proved a smaller bound if he could.
(In fact, as I understand it, the popular Graham's number is slightly larger than the published result, but the published result is only slightly smaller in relative terms, for a lot more work.)
Just wanted to say thanks, especially for the very helpful guide. You've convinced me to make this my 2022 growth project.
About your difficulty to see the application of theoretical math, often times even the people developing the theories have no clue. GH Hardy, one of the most prominent figures of Number Theory in the 20th century, was famously proud of its pureness and lack of applicability in the outside world. He was a strong pacifist living in the beginning of WWII, so he was particularly proud that it would have no military application whatsoever.
Fast forward 80 years, and Number Theory is being used literally billions of times per second in our world: it is the basis of public key cryptography. As you can expect based on that, it has military applications. But the funniest thing is that it only took a few years for Number Theory to be fundamental in cracking the Nazi Enigma codes during WWII. See more on that in Wikipedia: https://en.wikipedia.org/wiki/A_Mathematician%27s_Apology
I would say that most of what drives research in mathematics (and sometimes in other sciences as well) is pure curiosity and desire to know, with no consideration about the aplicability of any of it. But then it sometimes turns out that even the most esoteric parts of math end up being useful in describing the real world.
The University of Bayes Discord (UoB) has study groups for Bayesian statistics which might be relevant to you. The newest study group is doing Statistical Rethinking 2022 as the lectures get posted to YouTube. It requires less math than you have demonstrated in your post.
If you want a slightly more rigors path to Bayesian statistics, then I would advice to read Lambert or Gelman See here for more info.
If you want to take the mathematician approach and lean probability theory first, then the book Probability 110 by Blitzstein is pretty good, the study group at UoB is as of writing half way trough that book.
I really admire your patience to re-learn math entirely from the extremely fundamental levels on-wards. I've had a similar situation with Computer Science for the longest time where I would have a large breadth of understanding of Comp Sci topics, but I didn't feel as if I had a deep, intuitive understanding of all the topics and how they related to each other. All the online courses I found online seemed disjunct and separate from each other, and I would often start them and stop halfway through when I felt as if they were going nowhere. It's even worse when you try to start from scratch but get bored out of your mind re-learing concepts you learned the week prior to it.
Interestingly though, when I got into game development and game design, that was how you were expected to learn - you pick up a bunch of topics/algorithms/design patterns superficially, and they eventually fit together as you interact with them more often.
Perhaps running through a bunch of books through on your study guide will be how I learn Python and AI development properly this time :)
Cross-posted from my substack
Summary:
This goes over a year of self teaching, the topics of math I was able to learn, how I view the best way to learn math and what my general view of math as a whole is with my limited current knowledge on the subject. At the end is a guide I made for anybody else that wishes to teach themselves math, no matter the skill level.
The Wild World of Online Learning:
When you first start out to do any project it's usually best to have instructions and a goal of what a ‘finished’ state looks like in mind. Most consumer bought appliances that require assembly come with a very hand holdy instruction sheet and all the parts required, the same can not be said for trying to learn an area of knowledge. I would argue a good chuck, >60% of the year, was just spent on figuring out what I needed to learn and where the next logical step from that was.
I will fully admit that GPT-3 and other AI advances got me interested in joining the industry. I will also fully admit that I immediately jumped into Coursera’s Intro to Machine Learning and crashed hard when I realized I wasn't getting anything out of it because I barely knew high school algebra and couldn't understand what I was looking at, because I wasn't interested in anything math related until now. Even reading about what partial derivatives were, there was too much scaffolding of previous calculus math needed to really set me up for understanding. The kicker was I could still pass the coding assignments and it made my “grade” look good, but that’s mostly because of how the course was set up. I complained about online learning because of it here, but I’m going to be a bit more dispassionate in this review.
This cycle repeated in a few more online courses. I would spend time going through the course, get by doing the assignments, start to get frustrated because I didn’t know why what I was learning was going to be important down the road, then eventually drop the course and start looking for another one that I felt could actually help me. It was about half way through the year that I decided to drop online learning due to the resentment of feeling like I wasted my time on the courses. The problem was I could do all the practice problems of the online courses fine, but I still felt like I had only islands of unconnected knowledge and I hadn't learned anything actually useful. It was like college all over again, just doing enough to get a grade then moving on. Many courses seem designed to give a myopic view of a topic, a few practice problems, and then moving onto another topic. I’m more okay with colleges doing it since you get a degree and usually classes build off each other, but knowing I wouldn't get any recognition and most courses dead stopped after it felt bitter. But I do have my share of the blame of course.
I didn’t know what to look for, and so what I found probably wasn't that good. I won’t say perfect learning materials exist, or that most online learning courses are not just a trade of money for a sense of accomplishment. But knowing what I was supposed to learn and would have helped me beforehand would have helped me filter courses much better. So what happens is what I’m calling the autodidact's paradox: you need to already have the knowledge about what you want to learn about to get the best learning resources for that subject. This is of course easily solved by mentors or professors giving guidance in the field, but I did this on my own and was struggling.
Even though I wouldn't rate my experience with online courses very high, it wasn't pointless to me because it helped me learn what I wanted to avoid in an online course. And the insular knowledge tidbits sometimes did come back and help make learning things easier in the long run. It’s just during the course that it felt pointless, and honestly a good book could probably cover the topics faster and more in depth.
Knowing where my faults were from trying and failing many math and programming courses, and feeling burnt from online courses, I decided to go back to the basics and this time just to follow along with books. So the second half of the year, starting around July, I went on Kahn academy and started up high school algebra 1 again with the intent to really learn it this time.
Learning Math For Real.
This wasn't going to be like the first half of the year. I had a solid list of resources I could trust, I had a plan to move through them, and all I needed to do was do it. And I did.
I did all the way through Precalculus on Khan Academy. I thought the videos were good but took a while, so I only watched them if I was hard stuck on a section. Most of the time what I did was I would instantly go to the practice problems and continuously do them. Moving on when I got ¾ correct. It took about two solid months of three to four hours per day to get through it. Math just takes a good amount of time, especially learning and thinking through the problems.
After that I decided to read How to Prove It by Velleman, Calculus Vol.1 by Apostol, and Probability by Degroot. Probability I got through the first 2 chapters or so until it started throwing out integration which went over my head at the time. My biggest math faux pas is I got through the first third of Proofs by Velleman before dropping it, not because it was bad but cause Calc and Prob also started with an intro to proofs along with set theory, and their introduction was good enough to work off that I went ahead. I will most likely go back to finish it… eventually… maybe. Apostol's Calculus book I loved and read all the way through, although it was much more rigorous than anything I had encountered thus far.
The book's exercises being as rigorous as they are, I could only do a couple of the easier problems of the section at most. Some sections I couldn't do any, looking specifically at first and second order differential equations here. After reading for a while I realized I didn’t really like doing the problems in the book, mainly because they didn’t hold your hand as much. I was thinking back on how the online problems usually were involved and offered greater explanation on why you got something wrong, and helped to retrace steps and correct it. One day I was looking through EDx again and realized something that made me embarrassed about not thinking of it before. Just do both: learn from two resources, use the book as a guide through the concepts and the area you wanted to learn, then do practice problems online where you can get greater feedback. Learning from two resources also allows coverage of concepts from two viewpoints and leads to better understanding. Learning from multiple resources I heard about in this post by LessWrong user TurnTrout, which has other useful tips. So I found some equivalent online learning courses that I audited that I could go alongside with my math books, basically reinventing the college system of having readings, then lectures, then curated problems. Just from more diversified sources.
Apostol’s book covers what is typically called in the college programs Calc 1, Calc 2, and a bit of Linear Algebra. So from derivatives to integration and beyond, leaving nothing out using the axiom then proofs then exercises typical math book formula. Even though it was way over my head at the start, and some of it still is, I am way more comfortable now reading more advanced math texts then I was before. I also enjoy the little history lessons in each section and talks of how all the areas in the book tie together and what some of the areas are used for in future / different courses as well. Every complaint I had about online courses was solved by how good this book is.
How Does One Learn Math?
Over the year this is something I asked myself more and more, how does one learn math? What does learning actually look like and what is a good way to learn? Learning math is deceptively simple, just read the section about it and do some practice problems, then you have “learned” that section. But that is a facile representation of the hard part of teaching yourself. How many problems do you have to do to feel like you learned something? How comfortable with the topic and the surrounding extensions to it do you have to be to feel comfortable in that subject as a whole? How much time can you afford to spend ensuring your knowledge in one area that could have been better spent just moving on and getting rough knowledge in another? There are always constraints and trade offs. We live in a finite world with finite time, and these questions bothered me a lot.
My idea of optimal math learning essentially boiled down to this over the year: breadth first exposure to math topics is better than the school like depth first exposure to math topics.
Taking an outside look at how schools seemed to try and teach mathematics it seemed like they would do a brief (sometimes feeling non-existent in my case) intro into a topic then drill as many practice problems as they can onto a student. The students presumably with many other classes will probably look up how to solve the problem quickly to save time, do all the assignments, take the test and then wait for the next topic to repeat the cycle. For non math majors this is a necessary evil we must do to get to what we actually want. For math majors maybe it's different, and they can see the connections between subjects we outsiders can not. How well this is done depends on school quality, but this is what I would call front-loading math skills. Doing many exercises to drill in precise instructions to a topic that will hopefully last long term.
I think this way of learning is mostly a waste of time. Specifically because knowing that skills will decay when they are not used, if one doesn’t use many of the topics learned in their math courses then spending countless hours drilling problems is a waste of time that could be better allocated into learning how those problems arise and when to look for them. Schools seem to try to use a shotgun method of “we don’t know what this student will be doing in the future so we should try to get them all proficient in everything”, it has to be that way in our current system and that’s fine, it is what it is, most students I’ve talked to don’t know what they want to do in the future but have a blurry life outline that what they are majoring in now is correct. But if you choose to self learn with a clear end goal of a certain field there's benefits to be had by not over drilling exercises initially and putting them off till later.
My thinking came to be over the year that skills I will need will naturally be strengthened over time just due to the fact that I will be using them more, skills I don’t need will decay due to lack of use, but it will be useful to recognize at the very least what the problem is so I can look up how to solve it if need be. So what I do is when I read a section now I try to recognize where this math problem might arise, what it’s general structure looks like, and what related fields it might show up in. I do about 3-4 practice problems and I move on. I don’t get too deep into edge cases of the topic where most college students go, and I will fully admit it causes me to be less skilled. But I can move on faster and reinforce my prior learning while learning new material. As I go farther, what I need to know strengthens itself and what I don’t decays. Over time I get to the proficiency level of the college student who front-loaded all the problems, and over time they probably decay to my level using what's needed. But I will have covered more ground.
So to me learning math is just problem recognition. I know I won’t remember every way of how to solve a problem, or be able to re-derive a solution easily. But if I can at least recognize what I am looking at I can find what I need to solve it. This is what I am aiming for as I go through these lower levels of math of Algebra, Calc, etc. That’s not to say math isn’t difficult when it is time to learn it; it does still take a lot of brain power. But it's the best way I’ve found to learn it so far.
The nice thing about problem recognition in learning math is that all previous topics learned get abstracted easily, that's why symbols are so nice. When I see the integration symbol I don’t have to go over every little aspect of step functions or least upper bound set theory in my head to eventually come to a re-derivation of the formula used for polynomials. I can just use the formula to solve the problem. If I had to go over the number line to the Cartesian plane to all the types of lines all just to do a simple derivative I would probably go insane. While many people initially complain about the alphabet in math problems, I’ve come to appreciate the abstraction so we don’t have to reinvent the wheel every problem.
Once I get more into my field of choice doing Artificial Intelligence research I will be forced by necessity to deal with edge cases and tough problems, but I look forward to that. I think anybody trying to do research in their field is dealing with really tough problems. While reading Apostol’s book I was thinking about how some people spend their life on one math formula that gets a paragraph dedicated to it if they are lucky. Only to be learned and abstracted by college students in about a week or two. I could, if I had different values, probably spend a good chunk or even my whole life diving into one specific topic out of the whole book like the mathematicians of the past. It’s a weird feeling knowing I get to read quickly what took lifetimes to accumulate.
My General View of Mathematics After One Year.
These are my rough thoughts on what I think mathematicians are doing, what they are trying to accomplish, and some thoughts on the problems I’ve seen them try to solve with a layman's view.
While reading Apostol’s I quickly picked up on a common pattern. The topic would introduce the simplest case of the problem with a single variable (if possible), then one with multiple variables, then as many variables as possible, maybe infinite. While I found math problems with the potential for so many variables interesting, I wondered how that would apply to the physical world which we live in. I have since seen linear equations using hundreds of dimensions of input but it did help me get an idea of how to categorize math into the areas of theory and practice.
In my current view, theorists see how far they can take their specific topic of math and contend with interesting and weird edge cases of the topic. If some of the proven theories seem useful for a project that could be made, that specific part of the theory is used by applied mathematicians to make the project work. Applied mathematicians working in our world usually seem to get different titles however; calling themselves physicists, finance managers, and computer scientists rather than just mathematicians. While I enjoy the outcomes of the applied mathematicians, some of the theorists work I see as incomprehensible. But I think that’s just my lack of knowledge showing.
I don’t know if I will ever venture down the road of mathematics far enough to understand why the Monster group is important in topology, or why Conway’s knot is important either. Graham’s Number I see as ridiculous, apparently one of the answers to his original problem could be as low as a single digit number, why have power towers on power towers then? I don’t understand spending so much time on such a problem, or why to try to solve Fermat's Last Theorem. In my current understanding there is no practical application to these things even if they were solved; they are just fun puzzles. I can relatively understand what a computer science theorist is trying to solve, even if I don’t understand it I can see its application. In pure math theory I can’t understand it nor see its application right now.
I think it's incredible what I’ve seen so far of how theory and application has come together with Calculus and Algebra to build the modern world. How the mathematicians of old were learning how to describe the physical world around them in precise language. As I go further into math I intuitively think the relationship between math and the physical world will get more blurry, not as clearly defined as tracking how things change or accumulate over time. I’m excited to learn and battle my confusion about what is happening. Hopefully in another year's time I can call the person writing this clueless.
Concluding Thoughts:
While getting to the place I am required many small and sometimes large changes in my thinking and learning materials. I feel much more comfortable in my ability to learn things and my current plan going forward for learning mathematics. I don’t feel like I am good at mathematics, I’m not sure I will ever feel that way, but I know I have gotten better and that’s a good feeling I am going to continue to pursue. I will no doubt change my mind on what resources to use and get lost and confused while learning, but that's just part of the process. Thinking back to the beginning of the year, I certainly think I am in a better place than I was both in my knowledge of math, my position in life, and my perception of how I will fare in the future.
The study guide I’ve made and have been following is here.