How we built integration testing for fast-moving AI backend

This blog post discusses how the Red Hat team developed an integration testing system for their AI backend that adapts to rapid API changes from its upstream dependency, Llama Stack. They transitioned from mocked unit tests to real Llama Stack servers, utilizing a record-replay mechanism to efficiently manage testing while minimizing costs associated with LLM API calls. The approach includes spins up a real server for testing, maintaining test data, and automating compatibility checks through GitHub workflows. Their daily testing process allows for early detection of breaking changes, enhancing overall reliability in software development.