Why I Keep Coming Back to Universal Transformers

· Grigory Sapunov · April 27, 2026, 11:15 p.m.
Summary
In this blog post, Grigory Sapunov discusses his experiences with Universal Transformers (UTMs), particularly through the lens of building UTM-Jax. He shares design choices made during development, explores dead ends encountered, and emphasizes the importance of Architecture Decision Records (ADRs) in machine learning research code. The post reflects on both creative and technical aspects of working with UTMs and is geared towards helping developers navigate similar challenges in their projects.