Letting women vote had downsides for certain people too. That didn't make it less right.

As for stability, you need to consider the benefits of having the best people from around the world in the jobs where their comparative advantage is greatest.