This is why these measures have to be taken with a grain of salt (but are still useful).

Probability is subjective, in this case because it's dependant on the design of the experiment / quality of the analysis of that experiment to determine a p-value of a given result.

The book "Bayesian analysis in high energy physics" is a short and sweet introduction. If I got the title wrong I'll update it later.