Points 1 & 2. There is no direct communication between a producer and consumer. The producer writes to the broker, the consumer reads from the broker. There is a detailed flow diagram for the producer side operations in the article, and this deck has more of the details of how both the producer and consumer work: https://www.slideshare.net/apurva2/introducing-exactly-once-...
3. Yes, it has been tested empirically. Quoting from the article:
> We wrote over 15,000 LOC of tests, including distributed
> tests running with real failures under load and ran them
> every night for several weeks looking for problems. This
> uncovered all manner of issues, ranging from basic
> coding mistakes to esoteric NTP synchronization issues
> in our test harness. A subset of these were distributed
> chaos tests, where we bring up a full Kafka cluster with
> multiple transactional clients, produce message
> transactionally, read these messages concurrently, and
> hard kill clients and servers during the process to
> ensure that data is neither lost nor duplicated.
3. Yes, it has been tested empirically. Quoting from the article:
> We wrote over 15,000 LOC of tests, including distributed > tests running with real failures under load and ran them > every night for several weeks looking for problems. This > uncovered all manner of issues, ranging from basic > coding mistakes to esoteric NTP synchronization issues > in our test harness. A subset of these were distributed > chaos tests, where we bring up a full Kafka cluster with > multiple transactional clients, produce message > transactionally, read these messages concurrently, and > hard kill clients and servers during the process to > ensure that data is neither lost nor duplicated.