It's like the new crud but instead of designing the database tables and columns you're tweaking the prompts.

I would try to fine-tune GPT such that I don't need to repeat the first part for every query. Since OpenAI is billed by the token (a thousand tokens are about 750 words), it makes sense to fine-tune once then only submit the changing part.

Caveat emptor: I didn't try this out yet. No idea whether this would work.

I know its crazy, there are also some other promts that I have not edited from LangChain that acutally run in the background