the way you decide it's tasty is by guessing it's tasty, and guessing some other things, and criticizing those guesses, and "it's tasty" survives criticism while its rivals don't.
lots of this is done at an unconscious level.
it has to be this way b/c it's the only known way of creating knowledge that could actually work. if you find it awkward or burdensome, that doesn't make it impossible – which puts it ahead of its rivals.
The word you're looking for is "testing". I test whether that thing is tasty.
Testing is not the same thing as evolution.
it has to be this way b/c it's the only known way of creating knowledge that could actually work
That's an entirely circular argument.
Why expect AGIs to be better at thinking than human beings? Is there some argument that human thinking problems are primarily due to hardware constraints? Has anyone here put much thought into parenting/educating AGIs?