If we have many generations of rapid improvement of the algorithms this will be much easier if one doesn't need to make new hard-ware each time.

Yes, and this tradeoff exists today with some rough mix between general processors and more specialized ASICs.

I think this will hold true for a while, but it is important to point out a few subpoints:

1. If moore's law slows down this will shift the balance farther towards specialized processors.

2. Even most 'general' processors today are actually a mix of CISC and vector processing, with more and more performance coming from the less-general vector portion of the chip.

3. For most complex real world problems algorithms eventually tend to have much less room for improvement than hardware - even if algorithmic improvements intially dominate. After a while algorithmic improvements end within the best complexity class and then further improvements are just constants and are swamped by hardware improvement.

Modern GPUs for example have 16 or more vector processors for every general logic processor.

The brain is like a very slow processor with massively wide dedicated statistical inference circuitry.

As a result of all this (and the point at the end of my last post) I expect that future AGIs will be built out of a heterogeneous mix of processors but with the bulk being something like a wide-vector processor with alot of very specialized statistical inference circuitry.

This type of design will still have huge flexibility by having program-ability at the network architecture level - it could for example simulate humanish and various types of mammalian brains as well as a whole range of radically different mind architectures all built out of the same building blocks.

The cortex doesn't have a specialized vision circuit - there appears to be just one general purpose circuit it uses for everything. The visual regions become visual regions on account of . . processing visual input data.

That seems plausibly correct but we don't actually know that.

We have pretty good maps of the low-level circuitry in the cortex at this point and it's clearly built out of a highly repetitive base circuit pattern, similar to how everything is built out of cells at a lower level. I don't have a single good introductory link, but it's called the laminar cortical pattern.

Given how much humans rely on vision it isn't at all implausible that there have been subtle genetic tweaks that make our visual regions more effective in processing visual data (I don't know the literature in this area at all).

Yes, there are slight variations, but slight is the keyword. The cortex is highly general - the 'visual' region develops very differently in deaf people, for example, creating a entirely different audio processing networks much more powerful than what most people have.

The flexibility is remarkable - if you hook up electrodes to the tongue that send a rough visual signal from a camera, in time the cortical regions connected to the tongue start becoming rough visual regions and limited tongue based vision is the result.

Incorrect, the best factoring algorithms are subexponential.

I stand corrected on prime factorization - I saw the exp(....) part and assumed exponential before reading into it more.

Vision is a good example of this (we have some of the best vision of any mammal.) The rest are tasks which no other entities can do very well, and we don't have any good reason to think humans are anywhere near good at them in an absolute sense.

This is a good point, but note the huge difference between the abilities or efficiency of an entire human mind vs the efficiency of the brain's architecture or the efficiency of the lower level components from which it is built - such as the laminar cortical circuit.

I think this discussion started concerning your original point:

It isn't at all obvious that the brain is even highly efficient at solving AI-type problems, especially given that humans have only needed to solve much of what we consider standard problems for a very short span of evolutionary history (and note that general mammal brain architecture looks very similar to ours

The cortical algorithm appears to be a pretty powerful and efficient low level building block. In evolutionary terms it has been around for much longer than human brains and naturally we can expect that it is much closer to optimality in the design configuration space in terms of the components it is built from.

As we go up a level to higher level brain architectures that are more recent in evolutionary terms we should expect there to be more room for improvement.

A lot of the tasks that humans have specialized in are not generally bottlenecks for useful computation.

The mammalian cortex is not specialized for particular tasks - this is the primary advantage of it's architecture over it's predecessors (at the cost of a much larger size than more specialized circuitry).