There are indeed downsides, but my feeling is that if you got a job and moved out, you would afterwards ask yourself "Why the hell didn't I do this sooner?!"

One concrete point in favor of being self-supporting is that even if your first job is crappy, your next one will be marginally better, and so on. Right after high school I worked at a fast food restaurant. Then I worked in a bookstore, then doing telephone tech support, and now I'm a computer programmer. (I don't have any degree.)

By the way, I worked in that fast food job for four years. That was too long, but at the time I was too lazy and too scared to look for something else.