Tests that sometimes fail

1 · Sam · May 28, 2019, 6:54 a.m.
wolf.jpg695×564 286 KB A liar will not be believed, even when he speaks the truth. : Aesop Once you have a project that is a few years old with a large test suite an ugly pattern emerges. Some tests that used to always work, start “sometimes” working. This starts slowly, “oh that test, yeah it sometimes fails, kick the build off again”. If left unmitigated it can very quickly snowball and paralyze an entire test suite. Most developers know about this problem and call these tests “non determ...