After all, Joe is a deterministic physical system; his current state (together with the state of his future self's past light-cone) fully determines what Joe's future action will be. There is no Physically Irreducible Moment of Choice, where this same Joe, with his own exact actual past, "can" go one way or the other.

You sound to me as though you don't believe in free will.

“You sound to me as though you don’t believe in free will,” said Billy Pilgrim.

“If I hadn’t spent so much time studying Earthlings,” said the Tralfamadorian, “I wouldn’t have any idea what was meant by free will. I’ve visited thirty-one inhabited planets in the universe, and I have studied reports on one hundred more. Only on Earth is there any talk of free will.”

-- Kurt Vonnegut, Slaughterhouse–Five