My experience with them is limited, but I’m having issues with the LLMs ignoring the skills content. I guess it makes sense, it’s like any other piece of context.
But it’s put a damper in my dream of constraining them with well crafted skills, and producing high quality output.
Yeah, I'm still trying to figure out why some skills are used every day, while others are constantly ignored. I suspect partially overlapping skill areas might confuse it.
I've added a UserPromptSubmit hook that does basic regex matches on requests, and tries to interject a tool suggestion, but it's still not foolproof.
Yeah, the context window is a blunt instrument, everything competes for attention. I get better luck with shorter, more opinionated skills that front-load the key constraints vs. comprehensive docs that get diluted. Also explicitly invoking them (use the X skill) seems to help vs hoping they get picked up automatically
Yes, unfortunately the most reliable way is to inject them into the user prompt at a fresh session. My guess is that biasing towards checking for the tools availability too much affects performance, which might explain why it is quite rarer that I see it just choose to use a skill without previous prompting.
But it’s put a damper in my dream of constraining them with well crafted skills, and producing high quality output.