GPT4 is a multi-modal model. They haven't exposed a way to use the image embeddings, so consumers can't utilize it, but the model accepts image input and it was trained on images and text. Yes, there are other modals that can be incorporated, and reinforcement learning is still pretty nascent/very much unsolved.

The fact that it's 1) multi-modal and 2) we're approaching text exhaustion does not imply 3) that other modalities are exhausted

Yeah I just like pointing out that GPT-4 is multimodal because most people don't seem to realize this, as they never read the GPT-4 papers.