It would have to be one really fast network... zstd compresses and decompresses at 5+ GB (bytes, not bits) per second.

I just tested on a ramdisk:

  tool  cspeed    size  dspeed
  zstd  361 MB/s  16%   1321 MB/s
  lzop  512 MB/s  29%    539 MB/s
  lz4   555 MB/s  29%   1190 MB/s

If working from files on disk that happen not to be cached, the speed differences are likely to disappear, even on many NVMe disks.

(It just so happens that the concatenation of all text-looking .tar files I happen to have on this machine is roughly a gigabyte (though I did the math for the actual size)).

Looks like it depends heavily on choice of file, but I see good performance on both compressible and uncompressible files. Small files tend to perform (relatively) bad though. Here is a sample of 3 large files with different compression ratios:

  zstd -b1 --fast -i10 some-rpi-os-image-idk.img
  -1#-os-image-idk.img :2097152000 -> 226798302 (x9.247), 6765.0 MB/s, 5897.3 MB/s

  zstd -b1 --fast -i10 jdk-11.0.8+10.tar
  -1#jdk-11.0.8+10.tar : 688844800 -> 142114709 (x4.847), 2660.7 MB/s, 2506.8 MB/s

  zstd -b1 --fast -i10 archlinux-2025.04.01-x86_64.iso
  -1#.04.01-x86_64.iso :1236303872 ->1221271223 (x1.012), 3643.5 MB/s, 7836.6 MB/s

Ain't no way zstd compresses at 5+, even at -1. That's the sort of throughputs you see on lz4 running on a bunch of core (either half a dozen very fast, or 12~16 merely fast).

Where are you getting this performance? On the average computer this is by far not the speed.

Valve tends to take a different view...

Valve has different needs then most. Their files are rarely change so they only need to do expensive compression once and they save a ton in bandwidth/storage along with fact that their users are more tolerant of download responsiveness.

Is the network only doing an rsync? Then you are probably right.

For every other network, you should compress as you are likely dealing with multiple tenants that would all like a piece of your 40Gbps bandwidth.

In your logic, you should not compress as multiple tenants would all like a piece of your CPU.

This will always be something you have to determine for your own situation. At least at my work, CPU cores are plentiful, IO isn't. We rarely have apps that need more than a fraction of the CPU cores (barring garbage collection). Yet we are often serving fairly large chunks of data from those same apps.

Depends. Run a benchmark on your own hardware/network. ZFS uses in-flight compression because CPUs are generally faster than disks. That may or may not be the case for your setup.

What? Compression is absolutely essential throughout computing as a whole, especially as CPUs have gotten faster. If you have compressible data sent over the network (or even on disk / in RAM) there's a good chance you should be compressing it. Faster links have not undercut this reality in any significant way.

Whether or not to compress data before transfer is VERY situationally dependent. I have seen it go both ways and the real-world results do not not always match intuition. At the end of the day, if you care about performance, you still have to do proper testing.

(This is the same spiel I give whenever someone says swap on Linux is or is not always beneficial.)