I don't know where Fussell got this gardening thing. Coming from a family of WASPy upper-middle class gardeners, I've never heard any mention of flowers being "tacky" (which would have been code for "lower-class"). And none of the flowers he names as upper-class are especially high maintenance except for roses, which could also be lower-class depending on color. Things I would consider upper-class are Japanese maples ($400 for a sapling and they take ages to grow), espaliered fruit trees, arbors and gazebos, hedges or topiaries that need a lot of clipping, and any layout that looks artistic enough to have come from a landscape designer. One thing that makes a garden look arriviste to me is a sea of mulch with annuals plopped in them at regimented intervals.

I tried to find pictures of gardens that could be considered upper class - does the Jaqueline Kennedy garden at the White House seem good enough? Because it's full of topiary but also geraniums, salvia, chrysanthemums, and something I think are begonias (all identified as lower class by the article).