In a Big World, though, there is no one person who generated those memories

If we define a "Big World" to be, say Tegmark Level I infinite universe, then it is still the case that one particular space/time localized stable pattern realized in organic molecules did create the memories. There are other "copies" of that same pattern 10^118 meters away, but they are not here. I am unconvinced of your somewhat radical statement here.