"This is what software crackers and antivirus writers do on a regular basis. For an enlightening write-up, see the Symantec paper(s) on Stuxnet."

Well, not really.

I was both software cracker and virus writer myself(just for fun, we never distributed our virus or our cracks, keygens and stuff outside).

I was part of a larger group doing it. I had contacts with people that became famous for breaking several important protections, specially for games.

You have no idea how big software is. A million lines of code is impossible to read by any human being in his entire life. Yes, there are automatic tools and great disassemblers like IDA pro, but even if you were to see the entire source code, it is very easy to hide some flaw into the code.

If you add undocumented hardware into the equation, then it is very hard to disassemble without significant resources.

This hardware took dozens of millions of dollars to develop, hundreds of very smart people to design, with all the documentation. It will take at least an order of magnitude more to decode not having that info.

I don't know when you were a virus writer (the heyday of the virus writers I knew was the mid-1990s, and the virus writers I was aware of post-2000 had more knowledge of WinAPI than of x86), but: the state of the art for disassembly and reverse engineering has progressed dramatically in the past 10 years.

In particular, reverse engineering is no longer the specialty of virus authors the way it used to be; a totally mainstream application of reverse engineering, practiced by most major security companies, is reverse engineering patches to find the underlying security flaws they fix so they can be weaponized.

It is easy to hide flaws in any code. Assembly doesn't make it much easier.

Can you expand on or point to a write up on reverse engineering patches?

Do you mean that reversers' try to locate the, say, buffer overflow that was fixed and try to find another way to exploit it? Why would major companies want to do this?

The earliest talk I ever saw on automated binary reverse engineering was this one: http://www.blackhat.com/presentations/win-usa-04/bh-win-04-f...

I actually didn't see the BH talk, but I saw a similar talk that Halvar gave at CanSecWest shortly after.

The gist of it is, imagine that you have a binary that you are looking to find vulnerabilities in to exploit. You can go through all the trouble of discovering a vulnerability, and then hope it doesn't get patched; or you can sit and wait for a patch for said binary. There's reams of data out there about how long it takes for systems to apply patches, but in general, you can find vulnerable versions of patched software long after the patch has been released.

Using binary differential analysis you are basically zeroing in on the parts that were changed (which you can imagine is a much smaller subset of the overall binary) and find the vulnerability much more quickly.

There are tools (there is/was a product called Bindiff that I don't know if Zynamics still sells after they got bought by Google), which help you do this in a more automated fashion.

That means that with much less work, you can write up a working exploit that will still work on some decent percentage of the install base for the application (until everyone patches it).

Additionally, you can imagine that a lot of times when vulnerabilities get fixed, they aren't necessarily fixed with the utmost rigor. There's a lot of cases where an individual vulnerability might be fixed, but if you look at what was done, you can find other parts of the binary that are vulnerable to the same underlying flaw. Knowing what gets changed in the patch can tell you a lot about underlying issues.

So they can build tools that test for those vulnerabilities, or detect attempts to exploit them and block them on the network.

Think of Windows. You and I both know that not all machines running windows are up-to-date with their security patches. Reverse engineering a patch for a 0-day exploit could give an attacker an idea on how to compromise un-patched machines. With all the un-patched windows xp machines in the world, you could probably build your own bot net if you're smart enough :)

Right, but it's harder to detect subtle manipulation. It only takes a one bit error to reduce the key space by half.

It is hard to detect subtle manipulation period. But assembly language is an extremely concrete representation of a program; it is not an especially great way to write underhanded code.

It's possible to write obfuscated assembly, of course, but it sticks out as obfuscated.

To further get your head around the lack of cover compiled code would give a backdoor, it's worth knowing that Hopper will give you serviceable C-ish code from x86_64 for... let me check here... fifty nine bucks.

(It's possible that my only message with that last graf is "Hopper is great").

How easy is to detect off-by-one errors looking at assembly?

You are talking about a wildly different threat model than this little section of the thread is talking about.

Meanwhile: if you can't trust the hardware, you can't trust C code either.