An Educator-Guided LLM Pedagogical Agent for Scaffolded Feedback in Conceptual Database Design
Conceptual database design is difficult to tutor at scale because students iterate on entity–relationship diagrams and need feedback timed to each revision. This paper proposes an educator-guided LLM agent that separates hidden diagnosis of ERD errors from a pedagogical workflow controlling what is disclosed and when. A four-stage scaffold—concept check, guided application, low-detail feedback, and Q&A—was deployed across 383 feedback episodes. The work matters because it trades unrestricted LLM dialogue for educator-governed, artifact-grounded scaffolding that preserves student agency.
The paper presents a thoughtfully architected feedback system and an unusually honest deployment discussion, but its empirical claims are narrow and conditional. The authors report that 71.1% of observed target-level changes incorporated the hidden diagnosis, yet they repeatedly note that this measures alignment with the hidden diagnosis rather than its correctness or learning gain. The classroom evidence therefore establishes feasibility and traceability more than efficacy.
The separation of diagnosis, workflow control, and student-facing generation is a clean architectural contribution that directly addresses the risk of LLM over-assistance. The deployment logs are rich and the qualitative analysis is commendably frank about how staged disclosure can buffer diagnostic errors. The system design explicitly prioritizes educator governance through rubrics, question pools, and configurable stages rather than letting the LLM set its own instructional strategy.
The primary empirical metric—hidden-target incorporation—is circular in practice because it evaluates whether a revision matches the system's own hidden diagnosis rather than whether either is correct. The authors acknowledge this openly, but it leaves the claimed 71.1% rate pedagogically uninterpretable.
In addition, only 48.6% of episodes could be linked to a later saved artifact state, and the student survey suffers from a self-selected sample of at most 30 respondents (47.6% of enrolled students) with no control condition. The third environment also showed markedly lower revision and incorporation rates, hinting at fragility when requirements are complex, though the authors appropriately avoid overgeneralizing.
The evidence supports the mechanism of staged disclosure and traceability, but it does not empirically support pedagogical superiority over simpler alternatives. The paper lacks a controlled comparison to one-shot LLM feedback or to the authors' own prior requirement-grounded system, and the three environments are not designed as comparison conditions. Comparisons to classical ITSs (KERMIT, AutoTutor) are fair and well-motivated, yet the empirical section does not close the loop on whether the proposed workflow improves learning outcomes or final artifact quality relative to educator or unconstrained LLM baselines.
While the architecture and four-stage workflow are described in detail, the paper provides no code, prompts, dataset, or hyperparameters (e.g., GPT-4o temperature). The ERD parser, diagnostic pipeline, and stage controller would need to be reimplemented from prose alone. A significant barrier is the use of GPT-4o as a judge for incorporation classification; the exact prompt templates for this automated evaluation are not released, making independent verification of the 71.1% figure impossible. Without access to the versioned artifact traces or educator-authored rubrics, independent reproduction of the analysis is blocked.
We present an educator-guided LLM pedagogical agent for scaffolded feedback in conceptual database design. Integrated into an entity--relationship diagram (ERD) editor, the system grounds feedback in the student artifact, assignment requirements, educator-authored rubrics, and instructional resources. Its architecture separates hidden, artifact-grounded diagnosis from the workflow that controls the form and disclosure level of student-facing support. We instantiate the architecture as a four-stage workflow progressing from concept checks and guided application to low-detail feedback and localized clarification. Each feedback request creates a stateful episode linked to versioned ERD states. In a deployment spanning three ERD environments and 383 feedback episodes, 71.1\% of observed target-level changes fully or partially incorporated the hidden diagnostic target, including many after Stages~1--2. Qualitative analysis showed that staged disclosure sometimes withheld inaccurate details, supported selective uptake, or allowed later recovery, though some errors still shaped revisions. Survey responses from a self-selected sample favored delayed disclosure and student agency but noted indirectness and repetition.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.