I like to imagine that eventually we will be able to boil the counter-intuitive parts of quantum physics away into something more elegant. I keep coming back to the idea that every current interaction could theoretically be modeled as the interactions of variously polarized electromagnetic waves. Such as mass being caused by rotational acceleration of light, and charge being emergent from the cross-interactions of polarized photons. I doubt the idea really carves reality at the joints, but I think it's probably closer to accurate than the standard model, which is functional but patchworked, much like the predictive models used by astrologers prior to the acceptance of heliocentrism.

"Superhuman AI" as the term is generally used is a fixed reference standard, i.e. your average rationalist computer scientist circa 2013. This particular definition has meaning because if we posit that human beings are able to create an AGI, then a first generation superhuman AGI would be able to understand and modify its own source code, thereby starting the FOOM process. If human beings are not smart enough to write an AGI then this is a moot point. But if we are, then we can be sure that once that self-modifying AGI also reaches human-level capability, it will quickly surpass us in a singularity event.

So the point of whether IA advances humans faster or slower than AGI is a rather uninteresting point. All that matters is when a self-modifying AGI becomes more capable than its creators at the time of its inception.

As to your very last point, it is probably because the timescales for AI are much closer than IA. AI is basically a solvable software problem, and there are many supercompute clusters in the world that could are probably capable of running a superhuman AGI at real time speeds, if such a software existed. Significant IA, on the other hand, requires fundamental breakthroughs in hardware...

I have been mulling around a rough and mostly unformed idea in my head regarding AI-first vs IA-first strategies, but I was loathe to try and put it into words until I saw this post, and noticed that one of the scenarios that I consider highly probable was completely absent.

On the basis that subhuman AGI poses minimal risk to humanity, and that IA increases the level of optimization ability required of an AI to be considered human level or above, it seems that there is a substantial probability that an IA-first strategy could lead to a scenario in which no superhuman AGI can be developed because it is economically infeasible to research that field as opposed to optimizing accelerating returns from IA creation and implementation. Development of AI whether friendly or not would certainly occur at a faster pace, but if IA proves to simply be easier than AI, which given our poor ability estimate the difficulty of both approaches may be true, development in that field would continue to outpace it. It could certainly instigate either a fast or slow takeoff event from our current perspective, but from the perspective of enhanced humans it would be simply an extension of existing trends.

A similar argument could be made in regard to Hanson's WBEM based scenarios, through the implication that given the ability to store a mind to some hardware system, it would be more economically efficient to emulate that mind at a faster pace than to parallel process multiple copies of that mind in the same hardware space, and likewise hardware design would trend toward rapid emulation of single workers rather than multiple instances in order to reduce costs accrued by redundancy and increase gains in efficiency accrued by experience. This would imply that mind enhancement of a few high efficiency minds would occur much earlier and that exceptional numbers of emulated workers would be unlikely to be created, but rather that a few high value workers would occupy a large majority of relevant hardware very soon after the creation of such technology.

An IA field with greater pace than AI does of course present its own problems, and I'm not trying to endorse moving towards an IA-first approach with my ramblings. I suppose I'm simply trying to express the belief that discussion of IA as an alternative to AI rather than an instrument toward AI is rather lacking in this forum and I find myself confused as to why.

you should consider reading more about the issue so that you don't come off as naive

(Not being mistaken is a better purpose than appearing sophisticated.)

Thank you both for the feedback - it is always useful. Yes I realise this is a hard job with no likely consensus, but what would the alternative be?

At some stage we need to get the AI to understand human values so it knows if it is being unfriendly, and at the very least if we have no measurable way of identifying friendliness how will progress be tracked?

While personal identification with a label can be constraining, I find that the use of labels for signalling are tremendous. Not only does a label work in the same way as jargon, expressing a complex data set with a simple phrase, but because most labels carry tribal consequences it acts as a somewhat costly signal in terms of identifying alliances. Admittedly, one could develop a habit of using labels which becomes a personal identification, but being aware of such risk is the best way to combat the effects thereof.

Taboo "altruism" and "egoism". Those words in their original meaning are merely strawmen. Everyone cares about other people (except for psychopaths, but psychopaths are also bad at optimizing for their own long-term utility). Everyone cares about their own utility (even Mother Theresa was happy to get a lot of prestige for herself, and to promote her favorite religion). In real life, speaking about altruists and egoists is probably just speaking about signalling... who spends a lot of time announcing that they care about other people (regardless of what they really do for them), and who neglects this part of signalling (regardless of what they really do). Or sometimes it is merely about whom we like and whom we don't.

While it may not be the point of the exercise, from examining the situation, it appears that the best course of action would be to attempt to convert as many people in Rationalistland as possible to altruism. The reason I find this interesting is because it mirrors real world behavior of many rationalists. There are a bevy of resources for effective altruism, discussions on optimizing the world based on an altruistic view, and numerous discussions which simply assume altruism in their construction. Very little exists in the way of openly discussing optimal egoism. As an egoist myself, I find this to be a perfectly acceptable situation, as it is beneficial for me if more people who aren't me or are in groups that do not include me become altruistic or become more effective at being altruistic.

I haven't compiled any data relating rudeness to karma, and thus only have my imperfect recollection of prior comments to draw on, but I can certainly see your point here. I doubt, however, that an unpopular opinion or argument would benefit from rudeness if the post is initially well formed. I would expect rudeness to amplify polarization, thereby benefiting popular arguments and high status posters, and politeness to mitigate it. Would you be willing to provide me with some examples for or against this expectation from your observations?

Yet if it is about "10 good ways to prepare for the job interview" I usually don't read this kind of objections. On the contrary it is assumed that when going for an interview candidates will dress as well as they can, have polished their CVs and often waded through lists of common questions/problems and their solutions(speaking as a computer programmer here). Not doing so would be considered sloppy. It is rare to hear: "People, just go to the interview and present yourself as you are, if the company likes you it will take you."

While most of this post seems weakly designed and poorly edited (I will assume due purely to excessive haste), this statement brings up a point worth discussing. In truth, misrepresenting oneself in a job interview is a poor choice for one who desires stable and fruitful employment. Certainly one should strive to display their positive qualities while minimizing their negative qualities, but such a tactic is certainly not deceitful, as it is assumed by your interviewer that you will be performing such an optimization of your facade and will adjust their expectations accordingly. Likewise I believe that a critical difference between the "PUA" culture that is being discussed here and the central essence of optimizing one's ability to attract a mate is in the level of misrepresentation applied to an altered goal set.

A person not interested in keeping a job for any significant duration would have no motivation to be honest during an interview, as actually being effective is no longer a concern. A person attempting to attract a mate with no intention of producing offspring or maintaining a relationship that includes emotional investment is also lacking the motivation for honesty. One need not be in any way sexist for such a duplicitous mode of operation to be effective, it is merely the circumstance that our current culture expects male initiation of courtship rituals toward females. Refining the technique of initiating and succeeding in such social interactions is in and of itself a neutral goal, like any tool or technique, but when applying such an "art", there can certainly exist distasteful methods. The difference between a shrewd businessman and a con-man often lies primarily in the level of respect for the other member of their transactions, and the same can be said of this mating technique.