Not only would I decline the invitation, I would be extremely suspicious of the fact that very few have defected, and also extremely suspicious of those who have. What you're describing goes beyond telepathy. It's effectively one mind with many personalities. I could never trust any guarantee of safe passage through such a place. It would be trivial for a collective mind to rob a single mind of choice, then convince it that it made that choice. It would also be slightly less trivial but still plausible for a collective to convince that mind - and fool other independent minds - into believing that it chose to defect when it actually didn't.

On the other hand, if after observing the political landscape of the time period I came to realize that this entity is clearly taking over, then I would jump on board as a self-preserving strategy, knowing that at some point the non-connected independent minds would become marginalized enough to feel threatened and lash out violently, at which point the faster-thinking collective mind wins the fight. Being caught in a collective is less horrible than being caught in the crossfire.