Given most countries tax their residents as opposed to their citizens, I say let them leave and they forfeit their right to be residents.
In the case of the US, where they tax both their citizens and their residents. If you want to leave and not pay, you forfeit the residency and the citizenship.
How do you mean? He's obviously a resident in Belgium otherwise he'd be paying French taxes? If he moves back to France he'd become a French resident again or if he in practice spends more days of the year in France than in Belgium, then he'll be treated as if he's a resident in France.
Yes. He had to move to Belgium to remove the obligation to pay tax in france. Should he decide to move to the south of france instead, he would come back under the taxaction of france.
In the case of the US, where they tax both their citizens and their residents. If you want to leave and not pay, you forfeit the residency and the citizenship.