In physics, we do experiments because we don't know the rules of the natural world. That's the only way we can learn them.
In CS/programming, if we're doing any experiments, they're done because we're too stupid to infer the right answer beforehand. Insofar as you're dealing with the theoretical part of the problem, you shouldn't need any experiments, because the results are purely mathematic and should be perfectly predictable. And if you're doing hardware experiments, it's not computer science either - that's electrical engineering or applied physics. Whatever other tests you may be doing to save some time are still the engineering type.
The point is, computer science doesn't have any area in which it could use experiments for discovery. It's fair to say then, that it's not really a science. (It's also fair to point out that the line between science and engineering is getting blurry nowadays.)
> In CS/programming, if we're doing any experiments, they're
> done because we're too stupid to infer the right answer
That's the same as for physics. In theory, there is no difference between theory and practice. In practice, there is.
More specifically, for CS: what you're describing is an alternate universe where the halting problem[1] doesn't exist, or rather has been solved. In short, there is no way to predict, in the general case, whether a program will terminate or not, other than running it (= doing the experiment).
More generally, if you have a mechanism that predicts with perfect accuracy what the output of my program is, more quickly than running the program, then I will just call that new mechanism "the program", and use that instead of the original program. And then you're back to running the program to find out what it does.
UPDATE: To me, this is a (if not THE) fundamental results of theoretical computer science: a precise description of its limits. And since this is an article by Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it"[2]
In physics, we do experiments because we don't know the rules of the natural world. That's the only way we can learn them.
In CS/programming, if we're doing any experiments, they're done because we're too stupid to infer the right answer beforehand. Insofar as you're dealing with the theoretical part of the problem, you shouldn't need any experiments, because the results are purely mathematic and should be perfectly predictable. And if you're doing hardware experiments, it's not computer science either - that's electrical engineering or applied physics. Whatever other tests you may be doing to save some time are still the engineering type.
The point is, computer science doesn't have any area in which it could use experiments for discovery. It's fair to say then, that it's not really a science. (It's also fair to point out that the line between science and engineering is getting blurry nowadays.)