Modeling and Verifying Producer-Consumer Communication in Kafka Using CSP

Apache Kafka is a popular open source and commercially supported distributed publish-subscribe messaging system, which achieves high throughput, low latency and good load balancing. As a widely used messaging system, message delivery between producers and consumers is one of core functions in Kafka, and the reliability and correctness of message delivery are important and concerned. The formal methods can analyze whether it is a highly credible model. Therefore, it is significant to analyze the communication between producers and consumers from the perspective of formal methods. In this paper, we model the communication between producers and consumers in Kafka by the process algebra CSP (Communicating Sequential Processes). Moreover, we also apply the model checking tool PAT (Process Analysis Toolkit) to verify five properties of our system namely Deadlock Freedom, Acknowledgement Mechanism, Parallelism, Sequentiality and Fault Tolerance. The results of verification show the model of message transmission in Kafka messaging system caters for its specification, from which this system can be concluded that it is reliable.