Is it that unproductive to read a lot of fiction? I read extremely quickly and still retain a lot of what I read, and this seems to be quite a useful skill, and I've always assumed that one of the reasons I have this skill is because I read so much as a child.

I have the same experience. It could be that I wouldn't have read nonfiction as voraciously, and less practice would result in less reading skill now. But I suspect that I read far past the point of diminishing returns.

I also do seem to have been interested in learning things, and so if they were presented at a faster speed I think I would have learned more. If I had been spending an hour a day on various required subjects like civics and history and English, which would have been enough, and four hours on self-paced math, I suspect I would be in a much better academic position than I am now. (This is what I'm thinking of when I said strongly suboptimal- if I were educating a child, I don't think I would put 8 of their hours into fiction reading, though I might put 4.)