How Feasible Is the Rapid Development of Artificial Superintelligence?

Kaj_Sotala

Well so what if they can predict the future better? That's certainly one possible advantage of AI, but it's far from the only one. My greatest fear/hope of AI is that it will be able to design technology much better than humans.

The way I think of it, designing technology is a special case of prediction. E.g. to design a steam engine, you need to be able to predict how steam behaves in different conditions and whether, given some candidate design, the pressure from the steam will be transformed into useful work or not.

You can design things based on a priori prediction, but you don't have to in many cases...you can also use trial and error instead.

8gjm10y

It's possible to be very good at prediction but still rather bad at design. Suppose you have a black box that does physics simulations with perfect accuracy. Then you can predict exactly what will happen if you build any given thing, by asking the black box. But it won't, of itself, give you ideas about what things to ask it about, or understanding of why it produces the results it does beyond "that's how the physics works out". (To be good at design you do, I think, need to be pretty good at prediction.)

14

How Feasible Is the Rapid Development of Artificial Superintelligence?

14

14

14

How Feasible Is the Rapid Development of Artificial Superintelligence?

14

14