Exceptions vs. goto-based cleanup is purely a matter of personal programming taste and habits. Gotos exist because there are people who find them convenient, and not because gotos are "legitimate" in the context of that particular programming language.

Making the language minimalistic is never a goal of a language design unless it's a Brainfuck. If it were a goal, there would've not been three loop constructs in any modern languages.