The human mind is very sensitive to small modification in it's constituent parts. Up to a third of USA inmates have some sort of neurological condition that undermines the functioning of their frontal cortexes or amigdalas.

It is utmost important to realize how much easier an unpredictable modification would be in a virtual world - both because the simulation may be imperfect, and it's imperfections have cumulative effects, or because the augmentation itself changes the Markov network structures in such a way that brain-tumoresque behavior emerges.