A Tale of Two Flink Autoscalers

This blog post discusses Netflix's experience with deploying two different autoscalers for Apache Flink, illustrating the evolution from a homegrown solution to adopting an open-source autoscaler from the community. The authors share lessons learned about metrics, the complexities of scaling stateful jobs, and the benefits of embracing community-driven solutions. Key takeaways include the importance of trusting the right metrics, allowing for configurable defaults, and the value of adopting and extending existing tools rather than going solo.