Read through the sequence, but every time there is a many worlds assertion, stop and think whether adopting it lets you do anything more than feel superior to single-worlders. Take notes of the sort "without MWI, the following argument advanced by EY would not work: ...", then try to see if someone interpretation-agnostic would still be able to make the same argument. Simply learning the EY-path through QM is little better than memorizing scriptures (= guessing teacher's password). Feel free to post your progress and questions in the appropriate thread.

I think the expository part of the QM sequence can be skipped without significant loss of understanding of the broad philosophical point EY is attempting to make. However, if you are interested in QM for its own sake, I would recommend reading a quick non-pop-sci introduction to the theory before reading the sequence. Otherwise I think you will emerge from the sequence with only the illusion of understanding. In my experience on this site, it is far too common that people whose main exposure to QM is the sequence misunderstand not just the mathematics of the theory but also the conceptual structure of MWI.

If you want a really good, really short introduction to the mathematics of QM, this book is excellent. It's written in a very engaging, non-textbooky way, so don't worry about it being a dry read. It does presume some mathematical sophistication though. If you're not comfortable with differential equations and linear algebra, I would advise against reading it.

If you don't have that sort of mathematical background, read this instead. It actually is an introductory book, presuming very little knowledge on the reader's part. I recommend it over standard intro textbooks (such as Griffiths) for someone with your interests, because it focuses on developing a deep conceptual understanding of the theory rather than on doing calculations.

Finally, you could try David Albert's idiosyncratic but compelling book on the philosophy of QM. Again, the book is completely introductory. It won't teach you much about the math of the theory (for that, read Hughes), but it will give you a short overview of the main extant interpretations of QM, evaluating their strengths and weaknesses. The author is skeptical about MWI (in fact, he seems skeptical about pretty much every interpretation), but he's a very sharp philosopher, and his arguments are worth considering seriously.

I think being epistemically rational entails learning the mathematical part of QM first and reading through the QM sequence afterwards. Before you can seriously attack the problem of interpretation of QM, you needn't necessarily know the experiments and hard data and evidence for QM, but you must know the mathematical structure of the theory, because that's the thing what you are going to interpret!

Be sure you operationally understand QM under the collapse interpretation, that is, you should know how to calculate probability distribution of observed results in series of subsequent measurements of different observables and you should understand the standard jargon. You will probably have to learn Hamiltonian mechanics in the course (if you don't know it already); it is not strictly necessary for the collapse-related questions, but most textbooks assume familiarity with it from the beginning, and besides, broader and more general understanding of QM is probably a better goal than understanding only the aspects which EY had decided to write about. I suggest starting with the collapse interpretation because it is the easiest one to understand for a person accustomed only to classical mechanics. The downside is obvious., but, on the other hand, from what you've written I don't get the impression that you could be easily biased in favour of quantum collapse theories.

Your attitude towards the problem is apparently healthy, so let me just point out few (some of them perhaps obvious) things:

• Interpretation of a particular scientific theory is important for professionals working on the cutting-edge research (mainly because although different interpretations may be precisely equivalent in their experimental predictions, they may be profoundly different in their ability to inspire new ideas). For the non-professionals learning such things may serve to satisfy their intellectual curiosity, but it is certainly not the hallmark of general rationality. (I recall Yudkowsky has written that MWI is actually not an interpretation but rather a full theory distinct from the Copenhagen interpretation, but the arguments for that were, as far as I remember, mostly semantical. Even if we agreed upon MWI being a theory it would still not change the fact that one's opinion about it isn't that much correlated with one's rationality.)
• In comparison to the other sequences, there aren't many rationality-related insights in the QM sequence.
• The sequence doesn't give a faithful account of the alternatives to MWI. The matter is presented as if there was practically one alternative to MWI - wave-function realism with objective collapse (i.e. "there is an ontologically fundamental observer-independent thing, the wave function, which discontinuously changes upon measurement"). This is even not a true description of the Copenhagen interpretation, which is more of a positivist "here is the mathematical formalism you need for your calculations and who cares about ontology". Of course, even under other sequence posts people regularly accuse Yudkowsky of distorting the conventional philosophical positions to make his own look better, but nowhere this is as apparent as in the QM sequence. (This may be my own bias, though, because I am originally a physicist and therefore, when reading the sequences, I had been more informed about QM than, say, about philosophical zombies, and thus had been better equipped to spot the problems).
• The sequence lacks the level of formality I would consider suitable. When speaking about Many Worlds, one should for example try to define precisely what constitutes a "world" in the technical sense. I don't remember seeing such definitions in the sequence (although it's quite a long time since I have read it and my memories may be faulty; I suggest you judge for yourself).
• Quite surprisingly the approach taken in the QM sequence seems to contradict some of the EY's positions expressed elsewhere. Namely there is a recognisable realist stance without enough emphasise on the map/territory distinction. Although "probability is in the mind" is a central thesis to many EY's posts, "wave function is in the mind" is not considered a viable alternative. Somewhat similar maxim "shut up and calculate", again defended elsewhere in the Sequences, is rejected for the quantum-mechanical context in a quite rantish post. But again, I may be missing some subtleties in EY's positions.
• Another thing which is not properly addressed is the fact that in among all physical observables time plays a special role in quantum theory (being the only observable which can't be represented by an operator) while it is perfectly on par with position coordinates in general relativity. There are attempts to strip time of its unique role, mostly pursued by people trying to quantise general relativity, but it is not clear how MWI could survive such a change. On a related note, since quantum measurement is apparently not time-reversal symmetric, contrary all other microscopic dynamical laws, there is the possibility that quantum measurement is in fact an essentially macroscopic emergent process whose irreversibility has the same origin as the irreversibility of thermodynamical evolution. I don't remember seeing a discussion about this in the QM sequence; granted, this is a hard question, but at least the possibility of the non-fundamental nature of collapse (or world-splitting, if you prefer) should be noted.
• I remember few people commenting about the QM sequence as being the worst of the Sequences, an opinion I agree with, mostly for the reasons given above.

How should I go about being epistemically rational about this, what is the strong attack (and response) on what SEP claims is MWI's weakest defense (essentially, that MWI acts as a curiosity-stopper in the face of theoretical "collapse", if I understand correctly), and should I proceed with my plans of reading through the Quantum Physics sequence?

As prase and David Gerard have said, to understand QM, first learn QM. The actual mathematics and physics of it, not any popularisation. When you can pass a finals exam in the subject, with distinction, making minimal use of reference materials, then you will have reached a position from which you may be able to begin to participate in useful discussions about QM. How many similarly qualified people there are on LessWrong to have such a discussion with, or who they might be, I do not know.

I hasten to add that by that standard, I am certainly not one of them. I do not know QM. I hope I have never pontificated on the subject here or anywhere else, but I certainly do not do so now.

Anyone wanting a quick self-test, and with access to a university library, might entertain themselves with this volume, a book of exercises in QM, with worked solutions. If you can do even one of the problems at sight, and get it right, then you know more QM than you will ever discover from popularisations. When you can do all of them, you can claim to know QM.

Are you reading the Sequence in order to learn about QM, or in order to complete the Less Wrong course in rationality? The most sober way to think about QM is as a recipe for prediction, for which there are several competing explanations, and all of which might be wrong. The Sequence aims to rise above the disputes of physicists and rationally pick the winner. It doesn't really rise above, but neither does it sink below. An argument by a physicist in favor of a specific interpretation usually contains a mix of good points and blindspots, and Eliezer's argument is at that level. Understanding why QM works is a problem of the first order, and you won't learn the answer by reading the Sequence or even by reading a hundred textbooks. But you will learn more about the problem and about what we do know; and if you read the comments, you may even avoid acquiring false knowledge.

As for studying rationality, it's even easier to avoid going astray - just study the arguments bearing in mind that the specific conclusions about physics may be wrong, and ask how substantively that would affect the higher-level lessons that you're asked to draw.

That SEP article is the article on MWI. The other SEP articles on QM don't say that.