How we seamlessly migrated high volume real-time streaming traffic from one service to another with zero data loss and duplication

207 · Grab · Dec. 5, 2024, midnight
Summary
In this blog post, Grab discusses a project in which they successfully split read and write functionalities of a backend service into separate services to improve scalability and reliability. The migration of the stream processing functionality was done with no data loss or duplication while handling high-volume real-time traffic. Challenges included ensuring a seamless switchover with no downtime. The methodology employed a custom time-based switchover logic, allowing for meticulous scheduling of stream migrations. Results indicated successful migration with zero issues, though the authors caution that such custom logic should be approached carefully due to potential data inconsistencies during unexpected rollbacks.