And how exactly are you going to prove derivative work?

OpenAI is basically betting the existence of the company on that.

Meta is betting the existence of their Llama models on it.

I don't think that's true. When chatgpt generates something that infringes (even on something not in the training data) it is still infringement and the output cannot be used by the user for anything they couldn't use the original for.

Luckily it doesn't do that often under normal use

But that's the point he tries to make. When you "teach" LLM with some knowledge, you teach it a set of patterns. It won't necessarily drop the code that infringes copyright. Say you load Gumroad code into Gemini Pro context and say something like: "Check this app. Analyze the implementation of feature XY... I need you to help me implement feature XY... but in Go". Then, you can recreate an entire platform that will look nothing like the original but will have the same features and open source it.

Except they have billions of dollars to make that bet.