I'm broadly interested in the question, what physical limits if any, will a superintelligence face? What problems will it have to solve and which ones will it struggle with?
Eliezer Yudkowsky has made the claim "A Bayesian superintelligence, hooked up to a webcam, would invent General Relativity as a hypothesis—perhaps not the dominant hypothesis, compared to Newtonian mechanics, but still a hypothesis under direct consideration—by the time it had seen the third frame of a falling apple. It might guess it from the first frame, if it saw the statics of a bent blade of grass.”
I can't see how this is true. It isn't obvious to me that one could conclude anything from a video like that without a substantial prior knowledge of mathematical physics. Seeing a red, vaguely circular object, move across a screen tells me nothing unless I already know an enormous amount.
We can put absolute physical limits on the energy cost of a computation, at least in classical physics. How many computations would we expect an AI to need in order to do X or Y. Can we effectively box an AI by only giving it a 50W power supply?
I think there are some interesting questions at the intersection of information theory/physics/computer science that seem like they would be relevant for the AI discussion that I haven't seen addressed anywhere. There's a lot of hand-waving, and arguments about things that seem true, but "seem true" is a pretty terrible argument. Unlike math, "seem true" pretty reliably yields whatever you wanted to believe in the first place.
I'm making slow progress on some of these questions, and I'll eventually write it up, but encouragement, suggestions, etc. would be pretty welcome, because it's a lot of work and it's pretty difficult to justify the time/effort expenditure.
I can't see how this is true. It isn't obvious to me that one could conclude anything from a video like that without a substantial prior knowledge of mathematical physics. Seeing a red, vaguely circular object, move across a screen tells me nothing unless I already know an enormous amount.
This DeepMind paper describes their neural network learning from an emulated Atari 2600 display as its only input and eventually learning to directly use its output to the emulated Atari controls to do very well at several games. The neural network was not built with ...
I suggested recently that part of the problem with with LW was a lock of discussion posts which was caused by people not thinking of much to post about.
When I ask myself "what might be a good topic for a post?", my mind goes blank, but surely not everything that's worth saying that's related to rationality has been said.
So, is there something at the back of your mind which might be interesting? A topic which got some discussion in an open thread that could be worth pursuing?
If you've found anything which helps you generate useable ideas, please comment about it-- or possibly write a post on the subject.