Impacts of generative AI skill development on academic adaptation and learning outcomes among university students
This mixed-methods study examines how university students perceive their generative AI skills changing over time and whether those perceived trajectories connect to shifts in learning practices and academic outcomes. Using a retrospective survey of 313 students and interviews with 18 across disciplines, the authors identify heterogeneous developmental patterns—most commonly rapid early improvement followed by a plateau—and find that self-reported academic gains are linked to assignment-process adaptation rather than to a specific trajectory. The work highlights uneven growth across skill domains and signals that institutions should tailor support beyond generic tool training, though the cross-sectional design cannot establish actual longitudinal change.
The paper offers a methodologically conscientious, exploratory description of how students retrospectively perceive their GenAI skill evolution, linking perceived growth to assignment-process adaptation via an ordinal SEM. However, the core constructs of perceived skill growth and practice-related development overlap almost completely (latent correlation $0.954$, HTMT $0.964$), and the saturated structural model cannot discriminate among competing directional accounts. Consequently, claims about mediated developmental benefits remain speculative despite careful packaging, and the reliance on self-reported academic gains without verified performance leaves the outcome side of the model unanchored from objective learning.
The mixed-methods integration is the paper’s clearest strength: interview accounts of verification strategies, skipped reasoning, and selective delegation animate the survey patterns and reveal that verification behaviors do not necessarily equal evaluative competence. The statistical handling is also refreshingly cautious, applying Holm and Benjamini–Hochberg corrections alongside bootstrap intervals. Perhaps most valuable is the finding that perceived academic gains did not clearly differ across developmental trajectory groups after multiplicity correction, which directly challenges the intuition that perceived progress straightforwardly predicts benefit.
The study claims to map developmental trajectories, yet its cross-sectional retrospective design means the trajectories are reconstructed memories rather than observed change. As the authors acknowledge, both components relied on retrospective self-report and did not include repeated competency measurements or independently verified performance changes. This makes it impossible to distinguish actual skill growth from recall bias, shifting tool capabilities, or changes in task difficulty.
Measurement overlap further undermines the structural analysis. Because the structural component was saturated, the estimated paths were interpreted as associations rather than evidence of causal ordering or mediation. Perceived skill growth and practice-related development are nearly collinear (latent correlation $0.954$, HTMT $0.964$), yet the SEM drops rather than integrates the latter, complicating claims that process adaptation mediates anything distinct from perceived growth itself. The reported indirect association ($\beta = 0.452$) is therefore entirely shaped by the measurement restrictions and cannot establish mediation or temporal ordering.
The descriptive survey evidence is generally proportionate to the claims made, and the authors are commendably careful to frame the SEM as exploratory. Comparisons to small-scale longitudinal studies are fair in noting breadth-versus-depth trade-offs, though the paper occasionally overstates its ability to complement temporal evidence given that a single retrospective snapshot cannot establish sequences of change. The contrast with experimental work—particularly the caution that improved assisted performance cannot automatically be interpreted as improved independent capability—appropriately anchors the findings in a broader literature where assisted gains do not guarantee learning transfer.
Independent reproduction is blocked by the absence of shared data, code, and the complete survey instrument. The paper states that All data supporting the findings of this study are available within the paper, which signals no external repository. While analytical procedures are documented in detail—two-step polychoric DWLS estimation, case bootstrapping with 2,000 replications, and Holm and BH corrections—the questionnaire items referenced in Appendix A are not reproduced in the main text, and without raw data and analysis scripts researchers cannot verify the bootstrap handling of inadmissible solutions or reproduce the latent correlation estimates. Convenience sampling combined with targeted discipline and region coverage further limits the reproducibility of sample composition.
Understanding how university students perceive the development of their generative artificial intelligence (GenAI) skills over time can clarify how GenAI use relates to learning and academic outcomes. This study examined perceived skill-development trajectories, changes in learning practices, and associated academic gains using a retrospective questionnaire with 313 analyzed responses and interviews with 18 students across subject areas. Descriptive and comparative analyses and exploratory structural equation modeling (SEM) were integrated with thematic analysis. Perceived developmental trajectories were heterogeneous: 34.2% of students reported rapid early improvement followed by a plateau, 28.4% steady improvement, and 21.1% delayed acceleration. Development was also uneven across skills, with improvement in evaluating GenAI outputs perceived as more challenging; assignments were the most frequently selected driver of skill growth. Interviews further indicated increasingly specific prompting, tailored verification, and selective delegation, although some students reported stable routines, dependence, skipped reasoning, or improvements attributable partly to better GenAI tools. Academic gains were reported more often for assignments than for examinations. The SEM indicated a positive indirect association between perceived skill growth and academic gains through adaptation of assignment processes, whereas academic gains did not clearly differ across developmental trajectory groups after multiplicity correction. The findings suggest that perceived GenAI skill development varies in both trajectory and skill domain and is associated with changes in learning practices and differentiated academic benefits.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.