This post discusses Dynamic Context Parallelism (Dynamic-CP), a scheduling strategy utilized in NVIDIA Megatron Core for enhancing the efficiency of variable-length training in large language models (LLMs) during post-training and DiT pre-training processes. It aims to optimize performance by leveraging NVIDIA's advanced technologies.