I think I disagree, but it would depend on implementation details.

One possibility I can see is that you keep around a copy of the LaTeX distribution to parse these easily embedded LaTeX fragments, something like LuaTeX might someday turn out to be. In that case, you're still stuck supporting LaTeX's monstrosity of a toolchain. In that case, there's still e.g. no LaTeX on the iPad.

Another possibility is that you rewrite the LaTeX engine ... ah, nevermind, this isn't a possibility.