In a possible world where I irrationally believe the sky is blue, "I think the sky is blue" is true but "It's rational to believe the sky is blue" is false.

But in that possible world, you could just as well say "I think the sky is blue" and "It's rational to believe the sky is blue" and "The sky is blue". You shouldn't find yourself in the situation of saying "I believe that the sky is blue, but the sky is not actually blue".