Yes, I think "is this me" is an ethical question, and I also think ethics is purely physical (or more specifically a concept which only affects reality via physical representations of itself).
Well, let's take for granted that there's nothing that's super-natural or anything like that. But I guess I'd still caution against looking for certain kinds of physical explanations when they might not be appropriate to the subject matter. Let me explain by way of a couple of clear cases, so we can try to figure out where 'I' stand relative to each.
So if we want an explanation of hydrogen, I think we'd do well to look into a physical explanation. For every case of hydrogen, we can observe the relevant physical system and its properties, and these physical observations will directly inform and explanation (even, a complete explanation) of hydrogen in general. Hydrogen is an ideal case of physical explanation.
But what about the 'rook' in the game of Chess? Every rook and every chess game is a physical system. Indeed, we could go about and look for cases of a rook, and we will always find some physical object.
But we won't learn much about chess rooks that way. For one thing, we won't see that much in common in how rooks are physically instantiated. Some rooks will be little plastic castles, others will be made of wood or stone. Some will be computer code, others will be just neurological. And even if we did come up with a complete list of the physical instances of chess rooks, that wouldn't do much to explain them: in principle, I can use anything as a rook: a penny, a wad of paper, a patch of colored light, a vintage Porsche 959, anything so long as I can move it around a chess board. The rook has to have some physical properties, but observations about these properties just aren't very interesting. We can do a physics of chess rooks, but we won't get very much out of it.
I think ethics, politics, economics, etc. are all more like chess rooks than they are like hydrogen. There's nothing supernatural about the ethical, but that doesn't mean physics, or even biology, is a good place to go looking for an explanation.
Okay, I think I see where you're coming from. Let me sum it up to see if I'm getting this right:
...The important aspects of some categories of objects (aka concepts) which humans recognize is not easily reducible to constituent parts (e.g. disassemble a plastic Rook, a wooden Rook, and a memory of a Rook and there's nothing "Rooklike" to link them).
Even not-easily reducible concepts are technically reducible (they are still physical), but looking at the smallest structure is a hilariously inefficient and ineffective way to approach understanding t
aka The Fuzzy Pattern Theory of Identity
Background reading: Timeless Identity, The Anthropic Trilemma
Identity is not based on continuity of physical material.
Identity is not based on causal links to previous/future selves.
Identity is not usefully defined as a single point in thingspace. An "I" which only exists for an instant (i.e. zero continuity of identity) does not even remotely correspond to what we're trying to express by the word "I" in general use, and refers instead to a single snapshot. Consider the choice between putting yourself in stasis for eternity against living normally; a definition of "I" which prefers self-preservation by literally preserving a snapshot of one instant is massively unintuitive and uninformative compared to a definition which leads us to preserve "I" by allowing it to keep living even if that includes change.
Identity is not the current isolated frame.
So if none of those are what "I"/Identity is based on, what is?
Some configurations of matter I would consider to be definitely me, and some definitely not me. Between the two extremes there are plenty of border cases wherever you try to draw a line. As an exercise: five minutes in the past ete, 30 years in the future ete, alternate branch ete brought up by different parents, ete's identical twin, ete with different genetics/body but a mindstate near-identical to current ete, sibling raised in same environment with many shared memories, random human, monkey, mouse, bacteria, rock. With sufficiently advanced technology, it would be possible to change me between those configurations one atom at a time. Without appeals to physical or causal continuity, there's no way to cleanly draw a hard binary line without violating what we mean by "I" in some important way or allowing, at some point, a change vastly below perceptible levels to flip a configuration from "me" to "not-me" all at once.
Or, put another way, identity is not binary, it is fuzzy like everything else in human conceptspace.
It's interesting to note that examining common language use shows that in some sense this is widely known. When someone's changed by an experience or acting in a way unfitting with your model of them it's common to say something along the lines of "he's like a different person" or "she's not acting like herself", and the qualifier!person nomenclature that is becoming a bit more frequent, all hint at different versions of a person having only partially the same identity.
Why do we have a sense of identity?
For something as universal as the feeling of having an identity there's likely to be some evolutionary purpose. Luckily, it's fairly straightforward to see why it would increase fitness. The brain's learning is based on reward/punishment and connecting behaviours which are helpful/harmful to them, which is great for some things but could struggle with long term goals since the reward for making the right/punishment for wrong decision comes very distantly from the choice, so relatively weakly connected and reinforced. Creatures which can easily identify future/past continuations using an "I" concept of their own presence have a ready-built way to handle delayed gratification situations. Evolution needs to connect up "doing this will make "I" concept future be expected to get reward" to some reward in order to encourage the creature to think longer term, rather than specifically connecting each possible long term beneficial reward to each behaviour. Kaj_Sotala's attempt to dissolve subjective expectation and personal identity contains another approach to understanding why we have a sense of identity, as well as many other interesting thoughts.
So what is it?
If you took yourself from right now and changed your entire body into a hippopotamus, or uploaded yourself into a computer, but still retained full memories/consciousness/responses to situations, you would likely consider yourself a more central example of the fuzzy "I" concept than if you made the physically relatively small change of removing your personality and memories. General physical structure is not a core feature of "I", though it is a relatively minor part.
Your "I"/identity is a concept (in the conceptspace/thingspace sense), centred on current you, with configurations of matter being considered more central to the "I" cluster the more similar they are to current you in the ways which current you values.
To give some concrete examples: Most people consider their memories to be very important to them, so any configuration without a similar set of memories is going to be distant. Many people consider some political/social/family group/belief system to be extremely important to them, so an alternate version of themselves in a different group would be considered moderately distant. An Olympic athlete or model may put an unusually large amount of importance on their body, so changes to it would move a configuration away from their idea of self quicker than for most.
This fits very nicely with intuition about changing core beliefs or things you care about (e.g. athlete becomes disabled, large change in personal circumstances) making you in at least some sense a different person, and as far as I can tell does not fall apart/prove useless in similar ways to alternative definitions.
What consequences does this theory have for common issues with identity?
Why does this matter?
Flawed assumptions and confusion about identity seem to underlie several notable difficulties in decision theory, anthropic issues, and less directly problems understanding what morality is, as I hope to explore in future posts.
Thanks to Skeptityke for reading through this and giving useful suggestions, as well as writing this which meant there was a lot less background I needed to explain.