I am not yet entirely sure what the compiler does and how one could actually use it. Would it make sense to recompile source code, even third party source code, that is used on a multicore webserver using this compiler in order to gain some performance improvements?

Is your multicore webserver embedded? If not, I'm not sure this is for you.

  The embedded processor chip in mobile phones is small and
  not very powerful, so machine learning is more applicable
  to use in these devices in order to make the chip more
  energy efficient and powerful. (http://www.milepost.eu/resources.html)

More explanation is available here: http://ctuning.org/wiki/index.php/CTools:MilepostFramework. The chart should give you a basic idea of their architecture: they dynamically instrument runs of your programs under different optimization to find optimal settings for your release build.

yes, I read that, but wouldn't the same approach make sense for non-embedded processors? Or are the usual compilers already so good that there is not much to be gained by intelliently optimising the compiling process? What stroke me was the phrase "more applicable to", I thought this might mean: "but it works also on other processors...somewhat". I have not much idea about how the compiler actually works, or why the compilers for embedded processors are so bad that just these need this kind of optimisation while others do not. I guess I'll have to read through the stuff to find out...

Compilers for embedded processors aren't bad at all. But architectures vary heavily, and the subject of this article is a system that tunes optimizations automatically, instead of having engineers tune the optimizations themselves.

Also, note that for embedded systems low-level performance is typically a more important concern than for non-embedded systems. You typically don't care whether a function in your web or desktop application ends up getting inlined or not. It can make a significant difference in an embedded system.