The clients determined their own position, yes. It was not a peer to peer game and the server ran sanity checks on what the client was telling it (is their velocity acceptable for the car they have? did they accelerate too fast? are they in an area of the world that should not be accessible? etc) which worked pretty well. We deemed it was too resource intensive and too slow to run a physics engine on the server (it was a "massively multiplayer" game).
It doesn't sound like that at all. The person you're replying to just said that they were sufficiently frustrated with the physics engine's treatment (or lack thereof) of network phenomena that they bypassed it completely, telling the physics engine "this car is just a static object at this location."
That doesn't mean that there wasn't some other process which determined how that number was generated before it was handed to the physics engine -- either via a central server or something more distributed.
Speaking of the latter, I wonder whether consensus algorithms can be pulled off with a small enough bandwidth that you could incorporate them into a high-intensity game. Your consensus bandwidth is not the slowest-peer bandwidth but rather the median bandwidth, which can be much higher -- the question I'm asking is whether there's a good way for a slow peer to say, "hey, to improve my performance by reducing my bandwidth/latency, can one of you guys act as my server?"
Possibly, but what about collusion? This would be a problem especially with team games, one team has an extra player so 51% so they come to the consensus that everyone on the other team is always dead.
Well, I don't think it's a huge problem. Maybe I'm wrong.
I feel like it's core to the idea of "let's go play a game" that we largely agree on its rules -- and if more than half of us deny those rules, they are going to go off and play a different game; we won't want to play with them. So one response is for the reference client to simply say "if I see too many totally crazy things I'm just going to disconnect and ignore those peers who were saying that for a while."
But is it really a problem? You've got to imagine that we've got this red-vs-blue team game with two clients: ref and hax. The only way Red will be able to do this is if the red+hax population is greater than 50% among the server-population, because all of the ref clients will reject bad physics.
So suppose we've got a game of 21 people. 15 of them use the hax client (~70% participation), and we'll just assume there are no low-latency peers for the moment. The red team gets 11 consistently; the blue team gets 10. Then assuming team assignments are totally random, there's still only an 0.12% chance in any given game of the red team actually having 11 hax nodes and dominating the game. In the vast majority of the games they'll have to play honestly. And that's with 70% of the peers trying to game the system. (It gets a little worse if we include low-latency peers. So let's assume that there are 4 and they get distributed unevenly, 1 on red, 3 on blue. The consensus threshold is now 9. Assuming we lost 3 hax clients in the process, 16% of such games will be vulnerable to your attack. That's enough to make things frustrating.)
Meanwhile, the hax-client may make things unplayable unless it behaves like the ref-client when it's not in the majority.
The basic point is that the red-vs-blue partition makes those attacks not a concern. But when that partition doesn't exist, then there's a bigger problem. So the more concerning thing for me is denial of service. I don't think I'll get 70% of the legitimate customers to try to hack the game, but I do think that IPv6 support could lead to one person having a block of 10,000 IP addresses being able to take the majority of peers in all of my games. Okay: they may not have enough red peers to win the games for red or enough blue peers to win the game for blue, but suppose that their goal isn't to win, but just to shut down the system. Suppose their modified client, instead of saying "everyone on blue suddenly dies", says simply "everyone suddenly dies". Now 99% of games become unplayable, all of my legitimate users rage-quit, and I'm totally screwed.
So the problem is that sign-ups must be relatively closed and everyone needs to be able to validate that independently. I'm not sure how to solve that in a distributed way without some web-of-trust thing going on.
Sounds like you never played Red Faction. I love that game but watching people fly around the map, clip through walls and shoot 1000 rockets a second (much to the dismay of my graphics card) wasn't the most pleasant experience.