Mitchell, you are on to an important point: Observers must be well-defined.

Worlds are not well-defined, and there is no definite number of worlds (given standard physics).

You may be interested in my proposed Many Computations Interpretation, in which observers are identified not with so-called 'worlds' but with implementations of computations: http://arxiv.org/abs/0709.0544

See my blog for further discussion: http://onqm.blogspot.com/