Improving the scalability of a Spark pipeline for conversion attribution

1 · Pinterest · July 26, 2019, 6:17 p.m.
Rohil Bhansali | Software Engineer, Business & CreatorsMany Pinners use Pinterest with a planning mindset and often have commercial intent. Advertisers use Pinterest to reach those Pinners, and Pinterest offers ads measurement solutions that calculate metrics based on attributing Pinners’ actions on Pinterest to advertisers’ conversion events. In order to deliver these solutions at scale, we implemented a data pipeline using Spark.Since the number of Pinners and advertisers on our platform conti...