I think not plucking things out of the causal lattice should work both ways.

It's unlikely that a missing right pinky finger would make it really impossible for someone not to be a piano player professionally. There are a million other things, both personal conscious choices and circumstances, that come into play.

But also, Einstein probably would have discovered relativity even if he had taken one extra day off to do something frivolous.

This isn't to say you don't have to make the right choices when opportunities or inspirations present themselves, but even the ambitious 14 year old might have to wait a year before finding the right opportunity.