Both countries share a cultural inheritance from the native Taino people, but how much of that was able to escape post-contact ethnocide is a matter of debate. The two countires were united under first Spanish and then French colonial rule, but following the end of colonialism, the mainly Spanish-speaking east engaged in a hard fought war of independance from the French-speaking west. The "awful straight" border you noticed was the end result of this war.