We'd occasionally see defections from one lifestyle to the other, but historically they don't seem very common.

Or, at least, they end up becoming irrelevant for the same reasons that the agriculturalists won in the first place. If Roanoke disappeared because all of the settlers decided to ditch the farm and live as Indians, there were still way more Europeans coming than the few Europeans that defected, and the new colonists could support a much higher population density than the ones that went native.