Someone attempted to argue with me that vanilla ice cream is "spicy" because vanilla is a spice. While pedantically true, that's not a person I'd ever want to eat dinner with.
But it is an annoying ambiguity in English. You would probably say that curry paste is spicy even it had no capsaicin, which does happen. People also mean "has spices" when they say "spicy". I have heard Russians use the word Russian word "bitter" to describe the pain sensation of capsaicin too.
I prefer the Spanish word "picante" - stingy. It stings, it causes pain. It's not a flavour like a spice would be.
The problem is that there just isn't a good English word to distinguish it, I have to explain every time that I don't mean flavour, I mean pain. (Russian also has this problem, ambiguity is in every natural language.)
I have heard older Soviet Russians use the word горко, but this was about ten years ago. They were trying to express that the food tasted bad and they really just called it bitter when they meant it had capsaicin.
I've found this distinction to be dialect-dependent. That is Indians tend to make a sharp distinction between spicy and hot, and Americans tend to treat spicy and hot as synonyms.
Spicy, as in "using chili peppers," meaning it has capsaicin gives a bit of a lie to this. You literally just add capsaicin to food to make it spicy in many places. Is how they can give a number score to how spicy to make it. Leaving out the capsaicin does not change the base flavor profile at all.
Now, it is confusing, as spiced food is typically not spiced with capsaicin. Rather, spiced food has traditional spices and is often not spicy. (Think spiced rum. Not "hot" at all.)
Spicy IS the flavor, you're confusing Spicy food with Hot food.