However, even if the multiverse is infinite, or not even a set, I as a finite mind can only look at finite pieces of it. My caring function looks at a small piece of the multiverse, because it cannot comprehend the whole thing.

Your limitation does not inherently prohibit you from having a caring function that looks at the entire multiverse. The constraint is on how complex the pattern of evaluation of the features of the multiverse can be.