But we _are_ talking about developers working on critical path software, like medical devices, who would do exactly this.

(There have also been some reports of doctors making diagnosis decisions based on ChatGPT results.)