Violating the von Neumann architecture is the general trend that silicon computer design is going, yes. However, the requirements of backwards compatibility are fierce, and it may even be fundamentally harder to program multicore or "distributed" architectures.

Think of the separation of CPU and memory as a lie - "spatial geometry does not exist, everything is one pointer reference away from everything else" - and perhaps it makes sense why programmers find single-core easier to write code for. The lie is a useful abstraction.

One of the major differences between evolved and human-designed implementations is that evolved implementations take an amount of time and effort roughly proportional to the size of the solution, but human-designed implementations usually take time and effort proportional to the size of the argument that the solution is correct. (Of course, it's entirely possible to find edge cases that blur this distinction, but you understand what I'm gesturing at.)

That means that evolution (or other weak solution-space search methods) may have an advantage at finding solutions that are hard to explain why they work. (At least, until humans develop a well-known and general theory that they can reference in their designs.)

Depends on what organizational principles you are talking about. At the very generic level, the brain's extremely distributed architecture is already the direction computers are steadily moving towards out of necessity (with supercomputers farther ahead).

As for memristors, they will probably have many generic uses. They just also happen to be very powerful for cortex-like AGI applications.

The cortical model for an analog AGI circuit would be incredibly fast but it would be specific for AGI applications (which of course is still quite broad in scope). For regular computation you'd still use digital programmable chips.