Kafka on PaaSTA: Running Kafka on Kubernetes at Yelp (Part 1 - Architecture)

Yelp’s Kafka infrastructure ingests tens of billions of messages each day to facilitate data driven decisions and power business-critical pipelines and services. We have recently made some improvements to our Kafka deployment architecture by running some of our clusters on PaaSTA, Yelp’s own Platform as a Service. Our Kubernetes (k8s) based deployment leverages a custom Kubernetes operator for Kafka, as well as Cruise Control for lifecycle management. Kafka on PaaSTA on Kubernetes Architectural ...

Submit