How LyftLearn Democratizes Distributed Compute through Kubernetes Spark and Fugue

IntroductionIn a previous blog post, we discussed LyftLearn’s infrastructure built on top of Kubernetes. In this post, we will focus on the compute layer of LyftLearn, and will discuss how LyftLearn solves some of the major pain points faced by Lyft’s machine learning practitioners.Efficiently utilizing computing resourcesPain pointsThere is always a tradeoff between user convenience and efficient resource utilization. When LyftLearn users need more compute power, it’s a lot easier to scale vert...

Submit