Is tzatziki the only word that both starts and contains, but doesn’t end, with ts sound?

tsetse. https://en.wikipedia.org/wiki/Tsetse_fly

Tzatziki, Tsetse and Tzar are not English words; OP said there was just one English word (which he identified as "pizza", which isn't an English word).

It has foreign roots but it's surely an English word by any workable definition.

And "tsetse" has been used in English for nearly 100 years longer than pizza - the latter only really introduced because of Allied soldiers sent to in Italy.

Google Ngram says it wasn't until the 1970s when "pizza" was more commonly used than "tsetse." https://books.google.com/ngrams/graph?content=tsetse%2Cpizza... . Or from BoingBoing today, "North Americans feared and misunderstood pizza in the 1950s" - https://boingboing.net/2022/11/21/north-americans-feared-and... .