Lessons Learned from 2+ Years of Nightly Jepsen Tests

1 · CockroachDB · Feb. 21, 2019, midnight
Since the pre-1.0 betas of CockroachDB, we’ve been using Jepsen to test the correctness of the database in the presence of failures. We have re-run these tests every night as a part of our nightly test suite. Last fall, these tests found their first post-release bug. This blog post is a more digestible walkthrough of that discovery (many of the links here point to specific comments in that issue’s thread to highlight the most important moments)....