Speed of light latency puts limits on the maximum size of a brain. Or, rather, it enforces a relationship between the speed of operation of a brain and its size. At a certain point, making a sphere of computronium bigger no longer makes it more effective, since it needs to talk to working components that are physically distant. Granted, it's not a small limit, especially if people want a bunch of copies of themselves.