Topic

Exploring and Practicing AI Compilers in the Agent Era

With the rapid advancement of large language models (LLMs), optimization at the individual operator level is increasingly approaching its limits. In certain specialized scenarios, Agents can even surpass human experts. Meanwhile, model-level optimization—crossing operator boundaries and overcoming the memory wall—and adaptation to heterogeneous chips from multiple vendors, amid fragmented DSA/GPGPU ecosystems, have emerged as key challenges. This talk explores how AI compilers can address these challenges in the Agent era through a unified intermediate representation (IR) and an Agent–Compiler collaboration framework. It presents two cutting-edge practices. First, it examines the unified intermediate layer based on BAAI’s TLE/Common IR, showing how tile-level abstractions can establish a common compilation framework across chips from different vendors. Second, it introduces FlagMega, a model-level graph compiler that leverages Agents to perform graph rewriting, distribution planning, and operator generation at editable checkpoints, ultimately producing Megakernels that eliminate repeated trips to and from GPU memory. The talk will also demonstrate real-world end-to-end performance gains on NVIDIA GPUs and domestic chips, including those from Iluvatar CoreX and Huawei Ascend. Finally, it will explore the future evolution of AI compilers toward multi-GPU and multi-node systems, 3D DRAM architectures, and self-evolving compilation systems through the Agent for Compiler paradigm. Outline: 1. Background and Challenges: From Operator-Level Optimization to Model-Level Restructuring 2. Unified IR Layer: A Common Language for Chips from Multiple Vendors 3. Agent + Compiler: Model-Level Restructuring Across Operator Boundaries 4. Performance Validation and Real-World Results 5. Future Outlook: Toward Self-Evolving Compilation Systems Audience Takeaways: 1. AI Infrastructure and Compiler Engineers: Gain insights into sharing compiler optimizations across heterogeneous chip architectures through Common IR. Learn the checkpoint-based feedback loop, an effective engineering paradigm for integrating Agents with compilers and addressing compilation and correctness issues caused by Agent hallucinations. 2. LLM Algorithm and Inference Optimization Engineers: Understand the evolution from operator-level optimization to model-level Megakernel optimization. Learn how to use Agents to automate cross-layer fusion, layout reordering, and SBP (split/broadcast/partial) strategy search to unlock hardware performance in real-world inference scenarios, particularly with low batch sizes and long contexts. 3. AI Chip and Hardware Architects: Explore how compiler frontends can explicitly manage on-chip storage, such as tile.buf, asynchronous data transfers, and cooperative computation across multidimensional processing units to reduce backend instruction-generation complexity in emerging 3D DRAM and NUMA architectures. Gain a software-stack perspective to inform instruction set and microarchitecture design for DSA/GPGPU platforms.