Just to be clear, because this confused me at first:
- GCC recently switched from building their own code as C, to building as C++. At the moment they only use the subset of C++ which is C, but this affects compiler behaviour.
- People complained that C++ programs couldn't be compiled as efficiently as C programs, and this would make things slower.
- This post shows that compiling GCC as C, versus C++, does not result in an appreciable performance change. He benchmarks this by building the kernel using both versions of GCC.
Most of C++ devs think that C is part of C++. Even in this document it is stated something like: C subset of C++ vs C. Why? Because current C is not subset of C++, C++ is compatible with C89 or C90.
This can look like nitpicking, but it's really annoying to again and again hear such things from mouth of C++ gurus.
You just demonstrated another difference in C vs C++ mindset.
C doesn't need so many changes like C++, that is the fact and advantage. C++ introduces huge bunch of new stuff, you know add, add, add, then fix, fix, fix. At the end you get overengineered buerocratic 'give me my 5%' language.
So, in C99 some stuff are very useful like variable length arrays, restrict keywords, variadic macros... these are not cosmetic. And btw, evenfor that you don't need C99 to c89 converter, if you want c89 compatibility use macros. But, hey, macros are ugly, right? They are ugly but they don't make my eyes bleed as some stuff in C++.
So yes, i'm twisting your arm, sorry about that, but it needs to be twisted ;-)
P.S. Your compiler is your choice, just don't force others to use it.
I was under the impression that people were complaining about compilation speed AND the optimizations available to a C++ compiler (even if it was targeting the C subset). I wish the article gave some information about the runtime difference.
The article is about the runtime difference. In this case, the "run" is using gcc's compiled in both languages (via gcc and g++) to build the Linux kernel. Both gcc's show about the same runtime performance when executing this task.
I can see how this is a little confusing but it shouldn't be that hard. And yet there have been hours of comments on this story that miss this point.
"AND the optimizations available to a C++ compiler (even if it was targeting the C subset). I wish the article gave some information about the runtime difference."
To be absolutely explicit, because I realize my phrase wasn't clear, I was under the impression that C++ optimizations were expensive (say, under -O3 or -Os), so I wish that the same test were run under -O3 or -Os (comparing both the speed of compilation AND the optimizations)
> I was under the impression that C++ optimizations were expensive (say, under -O3 or -Os), so I wish that the same test were run under -O3 or -Os (comparing both the speed of compilation AND the optimizations)
What optimizations are available to the C subset of C++ that are not also available to C? The only thing I can think of offhand which would make compilation of C-with-C++ slower is the difference in symbol naming.
From what I understand of GCC, once the compilation has progressed to the optimization stage the code has already been converted to an intermediate language so the source frontend (gcc or g++) should not necessarily make available extra optimization passes. Again, this is speaking strictly of the subset of C which is common to both C and C++.
> What optimizations are available to the C subset of C++ that are not also available to C?
I thought C++ had some stricter rules about aliasing, allowing more room for optimization of the same code, but I'm having trouble finding out exactly what the difference was.
I actually haven't done a comprehensive test yet, but with debugging symbols turned off, are the same set of optimizations performed in gcc and in g++ compiling C code ?
The answer to that question probably depends on what optimization passes are in the language frontend (as opposed to the language-agnostic backend).
Of course, it's possible that some backend optimizations require proper annotations to be placed by the frontend. Either way though the GCC guys have been pretty good with their C support so I'd be very surprised if the C frontend didn't have the superior optimizer (if they are actually different on their support for the C subset supported by C and C++).
> Using stats –trim-outliers, which throws away best and worse
people make this mistake over and over again. there's no point in taking anything but the best time when talking about performance in a preemptive multiprocessing environment. the best time isn't an outlier, every time other than it is!
The best time is irrelevant in practice if it only happens very infrequently, under a very specific set of circumstances.
Benchmarks are only useful if they're indicative of what the likely outcome in a real-world scenario will be.
What you propose is much akin to those completely unrealistic "The Computer Language Benchmarks Game"-style benchmarks that show JavaScript (or Java, or PHP, or whichever slower language you prefer to use instead) as being as fast as C or C++. Perhaps that's true, but only when you discard reality almost completely. JavaScript only ends up performing decently because the benchmark is trivial, and the implementation has been highly tuned, often into a form that we'd never actually see in any real-world product. And predictably, JavaScript's performance ends up being much worse than C's or C++'s when it comes to real software, contrary to what the flawed benchmark suggests.
> The best time is irrelevant in practice if it only happens very infrequently, under a very specific set of circumstances.
This isn't a practical benchmark, though. This is an experimental performance comparison. So to ensure that your results aren't influenced by other variables, it's a good idea to attempt to produce the ideal environment for best performance across both tests. So binding cores(to eliminate as much variance from Linux's thread scheduler) and running nothing else on the machine are actually good decisions to make here.
Comparisons to the Computer Language Benchmarks Game is somewhat irrelevant here, because the comparison being run is actually extremely simple: Using a GCC executable compiled as C versus a GCC executable compiled as C++. The traditional failings of the game aren't here(implementation issues, language-specific tuning issues, etc.) because the code that GCC is compiled from is identical, the compiler options should be identical(except for C vs C++ compile setting), and the test setup should be run on the same machine, with the same compiler options and tuning choices.
>> that show JavaScript (or Java, or PHP, or whichever slower language you prefer to use instead) as being as fast as C or C++ <<
I've learned that it's unrealistic to expect someone to have actually looked at "The Computer Language Benchmarks Game" before pointing to it as an example for whatever point they wish to make ;-)
The only mystery here is why you think we wouldn't notice that "the implementation has been highly tuned" etc applies just as much to the C and C++ programs as to the JavaScript programs.
real-world is not preditcable. i can make any program run arbitrarily slow by making it swap or nice it into oblivion. if i'm measuring microbenchmark performance, i want to know how fast can it possibly go. measuring real-world averages makes sense only in a real-world deployment setting - this obviously isn't the case here.
No, there are other things which affect timing besides switching to other processes. For example, virt:physical mapping of the process memory affects cache performance unpredictably.
Yes. We can blame the ipchains/iptables syntax on him ;)
Also, he used to poke fun at C++ (many older C devs did that) but more and more of them are coming around to C++. The last automobile systems programmers I worked with were switching to C++ as well. In ten years, C++ will be the new C.
The results are what I expected and not that interesting. Compiling C as C++ usually yields the exact same native code as compiling it as C, except for some very minor differences.
What would be interesting is how long it now takes to compile GCC itself. For code that does not use C++ features, I also expect a very minor difference.
Compilation of PETSc, which is written in the intersection of C and C++, takes twice as long using a C++ compiler as using a C compiler. I'd be interested in the time difference for compiling GCC.
Interesting aside: There are some things that are faster in C++ than C, for example the standard library sort functions are massively faster because the compiler can in-line the comparison.
Well, yes you are correct. However if you need the fastest sorting you're probably gonna inline the comparison regardless of language. If qsort is slow due to the function pointer comparison just move it inline manually.
yes but the point was the title is ambiguous because almost all people will care about the speed of the binary it outputs not it itself. Indeed that is something people spend a lot of time on
It's measuring the speed of the GCC-compiled-as-C compiler against the GCC-compiled-as-C++ compiler by measuring how long each of those takes to compile the Linux kernel. It's not measuring how long it took to compile GCC itself.
Yeah, people seem confused by this. You can build the last release of gcc either with a C++ compiler or with a C compiler. I did both, and benchmarked the results.
Sorry, I missed the sentence: "With this in mind, and Ian Taylor’s bold assertion that "The C subset of C++ is as efficient as C", I wanted to test what had changed with some actual measurements."
To test a compiler's performances, you use it to compile stuff don't you? That's what TFA does: it builds GCC 4.7 using both a C compiler and a C++ compiler, then looks how the results perform, by compiling stuff with them.
TFA did not bench how long the original compilation (whether through C or through C++) took, that would have been pointless and irrelevant.
> So would not both gcc versions be expected to be very similar on instruction level?
It's going to go through the g++ frontend, parsing and compiling it using C++ semantics. There's no guarantee that the C subset of C++ will yield the exact same result as that of an actual C compiler. And a decade or two ago, as TFA notes, it most definitely did not.
I guess I just assumed that considering that both compilers are part of the same compiler collection that their output, while not guaranteed to be the same, can be expected to be somewhat similar. And that the divergences should not have all that much of an impact on execution speed. But I guess I did jump to a conclusion there.
The backend should do the same thing, but if the C++ frontend gives complete garbage to the backend, I guess there's only so much the backend can do to fix the mess.
I don't think having a single statistics point is enough to make a judgement here. There are many different programs of which compilers are a very small and relatively rarely used subset.
So 0.3% is a measurable difference and 0.1% is noise. That doesn't sound overly pragmatic to me. Benchmarking is an art, and from his setup 0.3% seems like noise as well.
They did 40 runs of compilations (10 each, twice). At the very least, a scatter or bar graph would be helpful, as it would let you see the data distribution (those numbers are good as uninterpretable on their own).
Saying that, 40 isn't very many times to draw statistical conclusions so you'd probably need more data.
I think a histogram or boxplot might be more appropriate to show the distribution. A scatter plot isn't appropriate for data whose independent variable is categorical (unless you jitter), and a bar plot (unless a histogram) would just show the means, wouldn't it?
We don't know, because GCC does not execute as a Python program.
This wasn't testing the speed of compiling a program as C++ versus compiling it as C; it was testing the speed of compiling the Linux kernel using GCC-compiled-as-C++ as the compiler versus using GCC-compiled-as-C as the compiler.
(And it concluded that the difference was insignificant anyway!)
People were complaining that the binaries would be slower? All the complaints I heard were "now it will take 10 times longer to compile gcc", not "now gcc will run slower".
It's certainly not IO-bound. Think about the Linux kernel. It's compiling on the order of 100MB of source code into something of the same order of magnitude of binary objects, and it takes minutes. That is a not close to saturating any kind of disk device. Not to mention that the intermediate files will be cached in RAM by the file cache and can be used without them hitting the disk.
My kernel directory is 13GB, most of that is intermediate build products. Linus has written somewhere on how much faster kernel compilation is with an SSD.
My dual-SSD striped setup with humongous throughput (600MB/s) made a kernel compilation 1.4% faster compared to using a 5400rpm HD. But then I have 16GB RAM, and RAM is a huge bottleneck in its own right, as swapping or pushing out the file cache will kill compilation times.
> I don't think compilation is CPU-bound - it's probably closer to 50/50 between CPU & I/O, if not outright I/O-bound. So, is compiling the kernel really a good benchmark for what you're trying to measure?
> The result is in line with what I'd expect, though :)
On first run without an SSD it certainly would be (linking the modules of a distro kernel, particularly so). The rest depends on rusty's setup, and in 14 minutes, it looks like it was a trimmed down build that could fit in memory.
- GCC recently switched from building their own code as C, to building as C++. At the moment they only use the subset of C++ which is C, but this affects compiler behaviour.
- People complained that C++ programs couldn't be compiled as efficiently as C programs, and this would make things slower.
- This post shows that compiling GCC as C, versus C++, does not result in an appreciable performance change. He benchmarks this by building the kernel using both versions of GCC.