Because the USA is a free and open society which should not punish someone just because they gave up their citizenship?

Honestly, your position is something I would've easily attributed to the Soviets at some point in time, and never thought an American would feel that way ...

Why should it permanently ban them?

For the same reason they entertain other people getting visitor's visas.