Heroku is introducing automatic prompt caching starting December 18, 2025, which aims to enhance performance for their Managed Inference and Agents services with faster responses for common tasks. This feature, enabled by default, currently caches system prompts and tool definitions but not user messages. Users can opt-out of caching by setting a specific request header.