"By definition" is a poor justification. I almost referenced the exact link you've given here. It gives poor justification for primality of one - resting on authority and lacking justification. The closest it comes (not very) is, "If 1 were admitted as a prime, these two presentations would be considered different factorizations of 15 into prime numbers, so the statement of that theorem would have to be modified." So what?
"The closest it comes (not very) is, "If 1 were admitted as a prime, these two presentations would be considered different factorizations of 15 into prime numbers, so the statement of that theorem would have to be modified." So what?"
Many proofs depend on the unique factorization of primes, many directly and many many more indirectly. It is a trivial mechanical modification to them to deal with the non-uniqueness of a prime factorization if you admit 1 as a prime number by explicitly taking that case and saying it doesn't affect this case. So in that sense, no, it's not important.
Except... you've now taken numerous proofs and made them longer... and for what? There's no didactic advantage. There's no proof that is made easier by letting 1 be prime. What's the advantage of adding all these special cases? None.
And that's the real reason. In the end, "by definition" is the only justification, and the reason we choose the definition is that it works the best. Unlike 0 to the power of 0 where there's at least a bit of argument to be had (though the overwhelming preponderance is for it to be 1), there's no reason to put 1 in the set of prime numbers. Even if you don't personally consider it a "lot" of evidence, it's still entirely one sided.
I don't believe it is a poor justification. It is a name chosen for this particular set of numbers. In fact, every name we've put on things in mathematics is "by definition". Real numbers, pi, odd numbers, perfect numbers. Someone found that particular definition useful, and others continued to use it.
In the case of prime numbers I believe you cited one of the best reasons of leaving 1 out of the set. If someone needs a name for "the number one and all prime numbers", they can define it.
"By definition" is the only possible justification, as for any other axiom in mathematics. Your intuition that such a thing is 'poor' and that there is a 'better' (why, more natural?, ordained by a divine source?) way to do it, is wrong.
