I don't know enough about gardening to have a reasonable opinion on that, but here are some possibilities:

• It takes resources in the form of studying fashion, or hiring someone else to do it for you, in order to know which flowers are in, and they're not the ones you expect. Compare to fashion in clothing; I'm probably unfashionable, and this correctly signals that I don't have enough time to keep up with trends or enough hip friends to advise me on them.

• The high-status flowers are harder/more expensive to grow than the low status flowers. Compare to his discussion on gravel driveways being higher-status than concrete because they require more maintenance.

• The high-status flowers are considered ugly to most untrained people, so they're a net loss unless you know you're associating with people who have been trained in taste. Compare to modern academic music, which will sound unpleasant to an untrained ear; therefore if you like it it signals not only that you yourself are trained, but that you expect the people who judge you based on your musical tastes to be trained. Hence the "any flower too vividly red is prole" comment.

• The high-status flowers would look pretentious to low-status people (I realize this is partly an explanatory regress). For example, anyone can give their kid a name that sounds upper-class, but if the kid is poor they risk looking stupid.