Amplifying Latency and Cost of LLM Inference via Rowhammer Attack
Large language models rely on just-in-time compilers like Triton to generate optimized GPU kernels, caching the results for reuse. This paper asks whether a Rowhammer-induced bit-flip can silently degrade inference efficiency without harming output correctness, and answers affirmatively with LLVMhammer. By corrupting a target-independent optimization pass in Triton, the attack forces emission of a suboptimal kernel that raises latency by 4–16%, while the on-disk cache makes the slowdown persist across restarts.
The paper presents a creative and technically sound attack vector. By targeting Triton’s target-independent optimization passes, the authors achieve cross-vendor slowdowns and rigorously isolate exploitable code locations with Karnage. However, the evaluation simulates bit-flips via software rather than demonstrating a live end-to-end Rowhammer exploit, and the modest slowdown only achieves statistical significance at large batch sizes.
Karnage’s staged pruning is the strongest contribution, reducing over 120,000 functions to 254 target-independent candidates and then pruning roughly 40,000 instruction-level flips to about 1,000 via offline compilation comparison before any GPU profiling. The statistical methodology is also solid: 30 repeated runs per configuration and Welch’s t-test separate the attack signal from vLLM’s continuous-batching noise.
The primary limitation is that the attack outcomes are demonstrated via software-induced bit-flips rather than a full end-to-end Rowhammer exploit inclusive of memory profiling, massaging, and live hammering; the leap from offline-identifiable locations to reliable real-world exploitation remains partially unsupported. Practical impact is also limited at small scale: at n=10 requests the slowdown is inconsistent and often statistically indistinguishable from scheduling noise, and only at n=1000 does the effect become conclusive across all models.
Additionally, the output-corruption results in Section V are less thoroughly benchmarked than the slowdown results, relying on anecdotal examples and BERTScore without the same 30-run statistical treatment.
The evidence supports the claim that a flipped compilation pass can degrade performance, but generalizability rests on evaluation with only two consumer GPUs and four 7B-parameter models. Comparisons to prior work are fair and explicit: the authors distinguish LLVMhammer from PRowhammer by noting that prior work profiles vendor-specific NVIDIA SASS instructions, whereas LLVMhammer’s target-independent locations transfer across binaries without rediscovery.
Reproduction would be difficult because the authors do not release Karnage or the list of identified bit-flip offsets, and exploitability is tightly coupled to Triton 3.6 and the underlying ISA. The experimental pipeline is described in sufficient detail—they use vllm bench serve with ShareGPT prompts, 30 runs per point, Welch’s t-tests, and specify exact software versions—but without published code, an independent group must reimplement Karnage from scratch.
Large language models (LLMs) achieve high inference efficiency using optimized GPU kernels generated by just-in-time (JIT) compilation frameworks such as Triton. Triton compiles GPU kernels through an MLIR/LLVM-based pipeline. We investigate whether a Rowhammer-induced bit-flip in the CPU's DRAM can degrade GPU-based LLM inference performance while preserving output correctness. We present LLVMhammer, the first Rowhammer attack that causes LLM inference slowdown by inducing a bit-flip in Triton's compilation pipeline. A bit-flip in a performance-critical optimization pass causes Triton to generate a functionally identical but suboptimal GPU kernel. Since Triton caches the generated kernel on disk, a single successful bit-flip causes persistent slowdown across subsequent inference executions. Triton's target-independent stages are shared across GPU vendors, which makes our attack GPU vendor-agnostic. To identify exploitable bit-flip locations without manual inspection, we build Karnage, a tool that isolates useful bit-flip locations in Triton's target-independent compilation pipeline. We evaluate LLVMHammer across a diverse set of LLMs, including Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Deepseek-llm-7B-chat, across NVIDIA RTX 4090 and AMD RX 7900 XTX GPUs. LLVMhammer increases inference latency of an LLM inference server running on a single GPU by 4-16\%, which, in turn, can translate into additional annual operating costs at the scale of multi-GPU deployments. We further show that LLVMhammer can also corrupt LLM outputs across different GPU vendors, reducing BERTScore to as low as 25\%, illustrating the broader security implications of bit-flips in Triton's compilation pipeline.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.