-Werror isn't a "debugging switch". It should be enabled at all times, for development builds and production builds.

If compilation warnings are given, then the responsible thing to do is to assume the program is invalid, at least in certain situations. Fix the code, avoid the warnings completely, and that's that.

There are remarkably few cases where compilation warnings can or should be ignored.

Since the compiler didn't succeed at compilation, no program was generated at all. Removing asserts is really compiling a different program.