Why do you think we have been converging to something? 

What is the point of science, otherwise? Better prediction of observations? But you can't explain what an observantion is.

If the territory theory is able to explain the purpose of science, and the no-territory theory is not , the territory theory is better.

What you think you see is a result of multiple layers of processing. What you get is observations, not the unfettered access to this territory thing.

..according to a map which has "inputs from the territory" marked on it.

seem to recall someone (EY?) defining "reality" as "that which generates our observations". Which seems like a fairly natural definition to me.It is not a definition, it's a hypothesis.

At least in the way Eliezer uses it. I make no assumptions about the source of observations, if any.

Well, you need to. If the territory theory can explain the very existence of observations, and the no-territory theory cannot, the territory theory is better,

You construct progressively more accurate models to explain past and predict future inputs. In the process, you gain access to new and more elaborate inputs.

Inputs from where?

The difference is that [if] the assumption of the territory implies that we have a chance to learn everything there is to learn some day, construct the absolutely accurate map of the territory

No it doesn't. "The territory exists, but is not perfectly mappable" is a coherent assumption, particularly in view if the definition of the territory as the source of observations.