Caledonian, you are completely wrong. Even if having an experience requires more than a single instant, it surely does not require more than a second: and a brain could arise and persist even for a full second, with sufficient luck, within a heat dead universe (or in other chaotic circumstances.) So you can look at a sequence of brains: brain 1, having experience A; brain 2, remembering experience A and having experience B; brain 3, remembering experience A & B and having experience C; and so on. There will be many, many, more sequences of such brains that remember incredibly disordered events than sequences that remember ordered events.

The fact that we do not have such experiences shows that the dust theory is false, whether you like it or not, and whether or not you think its premises are incontrovertible.