And it wouldn't really work, it is simply not possible to escape Goedel's incompleteness theorem if you have something even slightly powerful and non-finite, it will get you one way or another.

This is slightly exaggerated. The theory of real numbers is non-finite and quite powerful, but it has a complete axiomatization.