Your body does influence you somewhat, especially the parts that secrete hormones, but then, everything that isn't your body does too.

We are connected with the world. Most trivially, if we wouldn't have food, oxygen, and temperature in a given range, we would die. Less trivially, we are influenced by friends, television, amount of light during the day, etc. Similarly we are influenced by our internal organs: pain, digestion, ebriety, fatigue.

You remove any of this, and the person changes, more or less. Conventionally, very big change is usually fatal, we say the person has died. Small changes in environment (external or internal) often cause temporary changes, we call this mood -- a person with a different mood is considered the same person; their functions are a bit different, but with a high probability sooner or later their mood will change again. So it seems like a clear line between a "macrochange" (death) and a "microchange" (mood).

During longer time, microchanges can accumulate to something that no longer feels like a microchange, because we cannot expect it to change back. We say that people grow and their personality changes. This can sometimes feel discomforting, but it's considered normal.

In the right environment, the microchanges can accumulate faster; it can be called brainwashing (when intentionally caused by other people) or a sudden change of personality (when caused by drugs, illness, brain damage), and this idea is very discomforting for other people.

What does this all mean? The boundary between "me" and "not me" is not completely clear. There is a body, which under normal conditions cannot split or merge with other bodies, so this is a conventional base for identity (other base of identity is the memory). But the mind inside the body is changing. By uploading we are trying to preserve the mind and discard the body -- trying to discard the most obvious conventional source of identity, and preserve the more fluid ones (memory, personality).

Every part removed means change; generally, more removed parts mean greater change. With different inputs, your outputs will also become different. It starts with "I cannot fart" and continues with "I never feel happiness or sadness", unless you have a farting interface or a hormone-generating interface. If you cannot fart anymore, you have changed a bit. If you cannot feel happiness, desire, love, curiosity, you have changed more.

Perhaps we could try to estimate how much various factors contribute to our personality, so we could aim to preserve, say, 80% of the personality. We could decide to sacrifice farting, but preserve mood changes. Anyway, it would be just the first iteration. In the second iteration, the trans-human individual could decide to remove some unpleasant moods and add more pleasant ones, etc. If it happens in many iterations, slowly enough, we will feel that it is normal. If it happens faster, it will be discomforting for other people. After long enough time, there will be only a shared path (not even a single shared identity, if copying and merging ems becomes possible).

But this is not an argument against uploading. Even if my mind is destined to disappear -- either by quick change at the moment of death, or by accumulated slow changes after uploading -- I would prefer the slow way, if I had a choice, because that idea feels less painful. It's just a warning that any form of uploading will change the personality, or at least enable future changes, which may feel OK for the person changing, but may be shocking to outside observers.