query comments on FAI Research Constraints and AGI Side Effects - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (58)
Oh yes, it sounds like I did misunderstand you. I thought you were saying you didn't understand how such a thing could happen in principle, not that you were skeptical of the currently popular models. The classes U and F above, should something like that ever come to pass, need not be AIXI-like (nor need they involve utility functions).
I think I'm hearing that you're very skeptical about the validity of current toy mathematical models. I think it's common for people to motte and bailey between the mathematics and the phenomena they're hoping to model, and it's an easy mistake for most people to make. In a good discussion, you should separate out the "math wank" (which I like to just call math) from the transfer of that wank to reality that you hope to model.
Sometimes toy models are helpful and some times they are distractions that lead nowhere or embody a mistaken preconception. I see you as claiming these models are distractions, not that no model is possible. Accurate?
I offered the transform as an example how things can mathematically factor, so like I said, that may not be what the solution looks like. My feeling is that it's too soon to throw out anything that might look like that pattern though.