AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search

The five-stage correctness verification pipeline is the paper's strongest technical contribution, systematically catching compilation errors, shape-dependent boundary bugs, numerical stability issues under adversarial inputs, race conditions via determinism checks, and non-power-of-two edge cases before any performance is recorded. This ensures that the automated search does not sacrifice correctness for speed. The Amdahl's law orchestrator correctly prioritizes optimization effort by kernel runtime contribution, ensuring that agent iterations are spent where they impact end-to-end model latency. The dual-backend architecture providing both Triton (for fast iteration) and CUDA C++ (for hardware-level control) is well-designed, and the memory-bound kernel results (RMSNorm reaching 83% of H100 peak bandwidth) demonstrate that the agent can effectively apply fusion and tiling strategies from the six-tier playbook.

The matrix multiplication results are a significant weakness: AutoKernel's Triton kernels achieve only 0.29$\times$ to 0.43$\times$ the performance of PyTorch eager (cuBLAS), and 0.33$\times$ to 0.52$\times$ versus torch.compile on larger sizes (Table 4). Since matmul consumes 60-80% of transformer runtime per the paper's own introduction, failing to optimize the dominant operation limits practical impact for end-to-end model speedup. The paper prominently features 'community deployment' results—specifically a Triton FP4 matmul allegedly beating CUTLASS by 1.63$\times$ to 2.15$\times$—but these are unverified third-party reports with no controlled methodology, reproducible artifacts, or verification that the compared CUTLASS configurations were optimal. The scope is also limited to single-GPU individual kernels; distributed kernels, multi-device memory management, and cross-kernel fusion discovery are explicitly out of scope (Section 11).

The evidence strongly supports the claims for memory-bound operations (normalization, softmax, cross-entropy), where the agent successfully fuses multi-operator ATen decompositions into bandwidth-efficient Triton kernels. However, the comparison to torch.compile is presented selectively: the paper states their kernels 'beat it on 12 of the 16 configurations shown in Table 4,' but notably loses on rotary embedding (0.91$\times$ and 0.61$\times$) and performs poorly on matmul. The comparison to KernelBench (Ouyang et al., 2025) is contextual—the paper correctly notes that prior work achieved $<20$\% one-shot success, but iterative methods like CUDA Agent (Dai et al., 2026) have since reported 92-100$\%$ faster-than-torch.compile rates, suggesting AutoKernel's simple loop design may not represent the state-of-the-art in success rate. The claim that AutoKernel differs by 'starting from a complete PyTorch model' is valid but difficult to verify without end-to-end model speedup numbers being reported.

The system is open-source with over 9,200 lines of Python, 909 lines of agent instructions (program.md), and 18 starter kernel implementations, which supports reproduction. The paper provides specific hardware details (NVIDIA H100 80GB, CUDA 12.8) and measurement methodology (CUDA event timing, 200 iterations, trimmed mean). However, reproducibility may be hindered by the stochastic nature of LLM-based editing—success depends on the specific frontier model used (not always specified), temperature settings, and the non-deterministic trajectory of the 300-400 experiments per run. The 'community deployment' FP4 matmul result lacks code artifacts or exact prompts. While the five-stage harness ensures correctness, reproducing the specific speedups would require re-running the expensive iterative loop. The git-based experiment tracking is good practice, but the paper does not clarify whether the exact experiment histories are logged and available for analysis.