This post discusses the complexities of optimizing software for advanced GPU architectures, focusing on software pipelining (SWP) and warp specialization (WS). It introduces Twill, a novel system that automatically derives optimal scheduling solutions for iterative programs without relying on heuristics, showcasing its capability in rediscovering expert-developed schedules for Flash Attention on NVIDIA GPUs.