Looking at that map of representations of the DNQ agent playing Space Invaders I can't help thinking if it really has learned any concepts that are similar to what a human would learn.

The DNQ agent has a much simpler visual system that is suitable to the low complexity graphics of the Atari world. It also learns through supervised backprop on the RL signals, whereas the human cortex appears to learn through some more complex mix of RL and UL (more UL the closer one gets to the sensory stream, more RL as one moves up closer to the reward cirucits)

The more complex vision ANNs trained on natural images do produce visual concepts (features) that are reasonably close to those found in various stages of the human visual system. It all depends on the training data and the model architecture.