AgentDP: Towards Automated Model Deployment for Heterogeneous Embedded AI Applications
Deploying open-source AI models on embedded hardware is notoriously labor-intensive because developers must repeatedly compress, convert, and tune models for heterogeneous runtimes. AgentDP automates this end-to-end pipeline by coordinating specialized LLM agents that standardize fuzzy user requirements, partition models across incompatible inference engines, and bootstrap hyperparameter search from previously solved tasks. The paper reports dramatic time savings and accuracy improvements over manual and generic-agent baselines.
AgentDP is a creative and well-motivated system that meaningfully eases embedded model deployment through multi-agent LLM automation. Its operator-aware partitioning and configuration-reuse ideas are genuine contributions. Yet its 'first system' claim is strong and partly self-supported, and the evaluation—three tasks, small human cohorts, and ablations on only one of them—falls short of fully substantiating such sweeping assertions.
The operator-aware model partition module and cross-engine inference pipeline are the paper's strongest technical advance. By routing unsupported operators to alternate engines instead of falling back to slow CPU execution, they recover substantial inference speed, as shown by the VLU ablation where token generation drops from 15.6 TPS to 4.7 TPS without them.
The evaluation is too limited to support broad conclusions: ablation studies are run only on image recognition, and the other two modalities are asserted to behave similarly without evidence. The human baseline comprises only 15 participants split by self-assessed expertise, making statistical claims unreliable. In addition, the 'no existing system' assertion rests on a self-cited workshop paper rather than an independent literature review.
MapCoder is a SOTA multi-agent coding framework, not an embedded-deployment tool, so its poor performance is predictable and offers limited diagnostic value for AgentDP. The human-expert comparison is interesting but undermined by a tiny sample and lack of standardized tooling. While the reported accuracy and resource metrics are encouraging, they remain confined to three hand-picked models and devices.
Independent reproduction is effectively impossible: the system depends on GPT-4o and other proprietary APIs with undisclosed versioning or temperature settings, and the authors neither release code nor the constructed graph database of hyperparameter FSMs. Although hardware is described at a high level, exact software versions, sandbox configurations, and full prompt templates are omitted, leaving critical experimental details underspecified.
This paper presents AgentDP, which aims to deploy pre-trained AI models on heterogeneous embedded devices by fully automating model optimization and adaptation (O&A) with Large Language Model (LLM) agents. AgentDP not only significantly reduces human effort via automation but also enhances model inference efficacy by integrating LLMs with device-specific domain knowledge. To deliver a robust automation framework, AgentDP develops novel solutions to address three key technical challenges: 1) To help users elicit clear requirements, we propose a user requirement standardization module; 2) To mitigate the execution environment mismatch between AI models and heterogeneous embedded devices, we design an operator-aware model partition paradigm that enables different parts of the model to be efficiently executed on distinct inference engines. 3) To avoid re-searching optimization hyperparameters for new tasks, we propose a configuration reuse module that heuristically retrieves previous hyperparameter configurations for efficient reuse. Extensive experiments demonstrate that by fully automating the entire O&A process, AgentDP reduces entire model adaptation time by up to 80% compared with manual O&A while achieving higher task accuracy (more than 13% on average) than the baselines. Additionally, a comprehensive user study also exhibits the robust generalizability of AgentDP in adapting to different user requirements for heterogeneous embedded devices.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.