How we scaled data streaming at Coinbase using AWS MSK

2 · Coinbase · Aug. 24, 2021, 7:04 p.m.
By: Dan Moore, Eric Sun, LV Lu, Xinyu LiuTl;dr: Coinbase is leveraging AWS’ Managed Streaming for Kafka (MSK) for ultra low latency, seamless service-to-service communication, data ETLs, and database Change Data Capture (CDC). Engineers from our Data Platform team will further present this work at AWS’ November 2021 Re:Invent conference.AbstractAt Coinbase, we ingest billions of events daily from user, application, and crypto sources across our products. Clickstream data is collected via web and...