You're preaching to the choir, here...
But here's what you said in the 2013 strategy post:
But it’s not clear that additional expository work is of high value after (1) the expository work MIRI and others have done so far, (2) Sotala & Yampolskiy’s forthcoming survey article on proposals for handling AI risk, and (3) Bostrom’s forthcoming book on machine superintelligence. Thus, we decided to not invest much in expository research in 2013.
Seems a bit inconsistent?
Perhaps there are specific parts of my case that you don't find compelling?
Yes, again quoting from that strategy post:
Valuable strategic research on AI risk reduction is difficult to purchase. Very few people have the degree of domain knowledge and analytic ability to contribute. Moreover, it’s difficult for others to “catch up,” because most of the analysis that has been done hasn’t been written up clearly. (Bostrom’s book should help with that, though.)
My point is that publicly available valuable strategic research on AI risk reduction isn't that difficult to purchase, and that's what we need. All that information locked up in Eliezer and Carl's heads isn't doing much good as far as contributing to the building of a vibrant research community on Singularity strategies. (I would argue it's not even very good for guiding MIRI's own strategy since it's not available for external review/vetting.) To create new publicly available strategic research, we don't need people to catch up to their level, just to catch up to whatever is publicly available now. (Note that you're wrongly discouraging people from doing strategy research by saying that they need to catch up to insiders' unpublished knowledge when they really don't.) The fact that you've tried many different things and failed to get them to write stuff down faster argues more strongly for this, since it means we can't expect them to write much stuff down in the foreseeable future.
Math research can get academic “traction” more easily than strategic research can.
I don't see a compelling argument that getting academic traction for FAI math research is of net positive impact and of similar magnitude compared to getting academic traction for strategic research, so the fact that it's easier to do isn't a compelling argument for preferring it over strategic research.
Seems a bit inconsistent?
Ah. Yes.
What I should have written is: "it's not clear that additional expository work, of the kind we can easily purchase, is of high value..." (I've changed the text now.) What I had in mind, there, is the very basic stuff that is relatively easy to purchase because 20+ people can write it, and some of them are available and willing to help us out on the cheap. But like I say in the post, I'm not sure that additional exposition on the super-basics is of high value after the stuff we've already done and Bostrom's book...
One possible answer to the argument "attempting to build FAI based on Eliezer's ideas seems infeasible and increases the risk of UFAI without helping much to increase the probability of a good outcome, and therefore we should try to achieve a positive Singularity by other means" is that it's too early to decide this. Even if our best current estimate is that trying to build such an FAI increases risk, there is still a reasonable chance that this estimate will turn out to be wrong after further investigation. Therefore, the counter-argument goes, we ought to mount a serious investigation into the feasibility and safety of Eliezer's design (as well as other possible FAI approaches), before deciding to either move forward or give up.
(I've been given to understand that this is a standard belief within SI, except possibly for Eliezer, which makes me wonder why nobody gave this counter-argument in response to my post linked above. ETA: Carl Shulman did subsequently give me a version of this argument here.)
This answer makes sense to me, except for the concern that even seriously investigating the feasibility of FAI is risky, if the team doing so isn't fully rational. For example they may be overconfident about their abilities and thereby overestimate the feasibility and safety, or commit sunken cost fallacy once they have developed lots of FAI-relevant theory in the attempt to study feasibility, or become too attached to their status and identity as FAI researchers, or some team members may disagree with a consensus of "give up" and leave to form their own AGI teams and take the dangerous knowledge developed with them.
So the question comes down to, how rational is such an FAI feasibility team likely to be, and is that enough for the benefits to exceed the costs? I don't have a lot of good ideas about how to answer this, but the question seems really important to bring up. I'm hoping this post this will trigger SI people to tell us their thoughts, and maybe other LWers have ideas they can share.