I think floods would only be one type of problem from climate change. Other would be extreme weather, such as hurricanes, tornadoes, etc. These would be quite destabilizing for a number of governments, and contribute to social unrest, which has unanticipated consequences. Even worse, at some point, we can face abrupt climate change.

Now, this is all probabilistic, and I'm not saying it will necessarily happen, but this is a super-short version of why I consider climate change an X-risk.