A brief history of Hadoop at Spotify

1 · Erik Bernhardsson · Dec. 20, 2014, 5 a.m.
I was talking with some data engineers at Spotify and had a moment of nostalgia. 2008 I was writing my master’s thesis at Spotify and had to run a Hadoop job to extract some data from the logs. Every time I started running the job, I kept hearing this subtle noise. I kept noticing the correlation for a few days but I was too intimidated to ask. Finally people starting cursing that their machines had gotten really slow lately and I realized we were running Hadoop on the developer’s desktop machin...