An analysis of the most comprehensive international evidence yet on what generative AI does to learning, and why it matters more for vocational education than for any other sector: the documented gap between AI-boosted performance and genuine learning, the scale of student and teacher adoption already reshaping delivery, and what it means for assessment integrity, trainer productivity and the future of competency-based education in Australia's RTOs.
The Most Important Finding in the Report
The OECD Digital Education Outlook 2026, released in January 2026 under the title Exploring Effective Uses of Generative AI in Education, is the most comprehensive international synthesis of evidence on what generative AI actually does to learning. It draws on randomised controlled trials, neuroscience research, large-scale field experiments, design studies and expert analysis from across OECD member countries. Its central finding can be stated in a single sentence: generative AI can dramatically improve student performance while simultaneously undermining student learning. As the OECD puts it, where AI is used without pedagogical guidance, outsourcing tasks to it simply enhances performance with no real learning gains.
This is not a theoretical concern. A field experiment in Türkiye involving high school mathematics students found that access to GPT-4 improved practice performance by around 48 per cent using a standard chatbot interface, and by around 127 per cent using a tutoring version specifically designed to support learning. But when access was removed, and students sat closed-book exams, those who had used the standard interface performed about 17 per cent worse than students who had never used AI at all. Only the tutoring version preserved the learning gains. A widely cited 2025 MIT Media Lab study, a preprint based on 54 participants writing essays under three conditions, found that 83 per cent of the ChatGPT group could not quote a single line from essays they had just written, against roughly 11 per cent in the brain-only and search-engine groups, and brain imaging showed that neural connectivity scaled down as external support increased.
For the vocational education and training sector, where the entire credentialing system rests on the principle that a qualification represents demonstrated competency, not just demonstrated output, these findings are not peripheral. They are foundational. They suggest that a learner who uses generative AI to produce competent-looking assessment evidence may genuinely not possess the knowledge or skills that the evidence appears to demonstrate. And they indicate that the difference between AI-enhanced performance and AI-undermined learning depends almost entirely on how the AI is designed, deployed and integrated into the pedagogical process. This article examines the report's key findings, translates them into the context of Australia's VET system, and identifies what RTOs must do to ensure generative AI strengthens rather than erodes the integrity of competency-based education.
1. The Evidence: When AI Helps and When It Harms
The OECD report synthesises evidence across multiple experimental designs and educational contexts to establish a clear pattern. When generative AI provides direct answers, completed solutions or finished outputs, student performance on the immediate task improves, often substantially. But learning, defined as the durable acquisition of knowledge and skills that can be demonstrated independently, does not improve and may actively decline. The report identifies this as a fundamental misalignment between task performance and genuine learning.
The mechanism is what researchers describe as cognitive offloading. When a student uses a general-purpose chatbot to answer a question, generate an essay or solve a problem, the cognitive effort that would normally drive learning, including diagnosis, evaluation, iteration and reflection, is transferred to the AI system. The student receives a competent output but has not performed the mental work required to develop the underpinning knowledge or skill. Studies of essay revision found that students interacting with human experts followed a structured help-seeking process: diagnosing what they needed, asking for help, evaluating the response, iterating and implementing. Students interacting with a general-purpose model frequently skipped the diagnosis, evaluation and iteration stages, going directly from question to implementation, a pattern researchers termed metacognitive laziness.
The evidence is not uniformly negative. The report identifies circumstances where AI demonstrably improves both performance and learning, and the critical variable is how the AI is designed and used. Purpose-built educational AI tools that employ structured tutoring strategies, such as Socratic questioning that guides learners through reasoning rather than providing answers, show more promise than general-purpose chatbots. The Türkiye experiment found that while the standard interface undermined learning, a tutoring version designed to support the learning process produced far larger practice gains while preserving learning once access was removed. Collaborative learning studies found that AI acting as an information hub, a personalised materials generator or a peer contributor in group tasks produced small-to-medium improvements in subject learning and larger improvements in critical thinking and teamwork. The following table consolidates the key experimental evidence, showing the consistent pattern: performance gains are common, but learning gains depend on design.
|
Study or source |
Performance finding |
Learning finding |
|
Türkiye field experiment (Bastani et al., 2024) |
High school students using GPT-4 improved maths practice performance by around 48% with the standard interface and around 127% with a tutoring version designed to support learning |
When access was removed, students who had used the standard interface performed about 17% worse on closed-book exams than students who had never used AI; only the tutoring version preserved learning gains |
|
MIT Media Lab essay study (Kosmyna et al., 2025, preprint, 54 participants) |
Participants wrote essays alone, with a search engine, or with ChatGPT; the ChatGPT group produced well-rated essays |
83% of the ChatGPT group could not quote a single line from essays they had just written, against roughly 11% in the other groups; brain imaging showed neural connectivity scaling down as external support increased |
|
Essay revision studies (cited in the report) |
Students using a general-purpose model to revise essays achieved high task performance scores |
Knowledge gains did not improve; students performed fewer metacognitive tasks, especially evaluation and orientation, with some skipping diagnosis and iteration entirely, demonstrating metacognitive laziness |
|
Teacher productivity (cited in the report) |
Secondary science teachers in England using AI for lesson and resource planning achieved a reported 31% reduction in time spent on these tasks |
The time savings were substantial and measurable; the long-term effects on teaching quality and skill maintenance remain under investigation |
|
AI tutoring support (cited in the report) |
AI support produced its largest reported gains for less-experienced tutors, with smaller gains for experienced ones |
The pattern suggests a potential equalising effect where human expertise is thinnest, though the report flags the risk of skill atrophy among tutors who rely heavily on AI |
|
Collaborative learning studies |
GenAI supporting group learning as information hub, materials generator, feedback provider and peer contributor |
Some studies found small-to-medium improvements in subject learning and larger improvements in critical thinking and teamwork; evidence is still limited, but the direction is positive |
|
The Performance-Learning Gap |
|
The report's central finding is that generative AI can improve task performance while undermining genuine learning. Students produce better outputs but develop weaker skills. The Türkiye experiment quantifies it: around 48% better practice performance with AI, but about 17% worse exam performance once AI is removed. For VET, where qualifications certify competency rather than output quality, this gap is not an academic concern. It is an assessment integrity risk that providers must design against. |
2. The Scale of Adoption: Students and Teachers Are Already Using AI
The report provides the most detailed international picture yet of how extensively students and teachers are using generative AI. The adoption data makes clear that generative AI is not a future challenge for education systems. It is a present reality already shaping how learners engage with their training and how teachers design and deliver instruction.
Student adoption has moved from marginal to mainstream since ChatGPT's launch in 2022. In Estonia, a national survey found that 90 per cent of upper secondary students reported using AI tools to support their studies in 2024, with ChatGPT the dominant tool. Across a seven-country European survey of more than 7,000 students aged 12 to 17, 48 per cent reported having used ChatGPT in 2024, with almost half of them instructed to do so by their teachers. Adoption among higher education students runs higher still in several countries. Australia, where ChatGPT usage as a share of internet users is among the highest in OECD countries, sits firmly within this trend.
The purposes for which students use AI are oriented more toward convenience than deep learning. The most common uses are obtaining information, getting explanations of terms and concepts, and generating ideas, and a substantial share of students report using AI to provide complete solutions to tasks. Only a minority report using AI for self-regulatory functions such as structuring personalised learning plans or tracking progress. Students are, in the report's framing, using AI primarily for cognitive support and production support rather than as a genuine learning tool.
Teacher adoption is substantial and growing. The OECD's TALIS 2024 survey found that around 36 per cent of lower secondary teachers across OECD countries had used AI for work-related tasks in the previous 12 months, with enormous variation across countries. Teachers are primarily using AI to learn about topics, summarise content and support lesson planning. The report documents a reported 31 per cent reduction in time spent on lesson and resource planning among secondary science teachers in England who used AI tools, one of the most clearly quantified productivity gains in the evidence base.
3. What This Means for VET: Five Critical Implications
The report is written for education systems globally, with a primary focus on school and higher education contexts. It does not specifically address vocational education and training. But its findings are, if anything, more consequential for VET than for any other sector. VET's entire credentialing model is built on the principle that a qualification certifies competency: the demonstrated ability to perform to workplace standards, supported by valid, sufficient, authentic and current evidence. If generative AI can produce evidence that appears competent without the learner actually being competent, the foundation of VET's value proposition is at risk. The following table maps the report's key findings to five critical VET domains.
|
VET domain |
OECD evidence |
Why it matters for VET |
What RTOs should do |
|
Assessment integrity |
Students using general-purpose AI can produce competent-looking work without genuine understanding; the Türkiye experiment shows about 17% worse performance once AI is removed |
Competency-based assessment requiring observed demonstration, workplace performance and competency conversations is the system's primary defence against AI-generated evidence that masks genuine skill gaps |
Design AI-resilient assessment that requires learners to demonstrate competence in conditions where AI cannot substitute for genuine capability; observation, practical tasks and oral questioning are essential |
|
Trainer and assessor productivity |
Teachers saved a reported 31% of planning time using AI; less-experienced tutors saw the largest gains in student outcomes with AI support |
VET trainers can use AI for lesson planning, resource development, formative feedback and administration, freeing time for the high-value human interactions that drive competency development |
Pursue the productivity gains but balance them against skill atrophy; trainers who outsource too much to AI may gradually lose the pedagogical expertise that makes human-led assessment irreplaceable |
|
Learner AI literacy |
Student AI use is mainstream across upper secondary and higher education; VET learners arrive with established AI habits |
VET has a responsibility to develop learners' ability to use AI effectively in their future workplaces, including understanding limitations, hallucination risks and the difference between AI-assisted performance and genuine competence |
Embed AI literacy across qualifications as a foundation skill, not a standalone unit; every qualification should address how AI is used and misused in the relevant industry context |
|
Teacher-AI teaming |
The report identifies three paradigms, replacement, complementarity and augmentation, with augmentation through collaborative engagement producing the best outcomes |
VET's competency-based model is naturally suited to augmentation: AI handles routine tasks while trainers focus on complex judgement, workplace observation and learner support that require human expertise |
Frame AI as augmenting, not replacing, trainer capability; focus professional development on using AI well while maintaining the assessment and pedagogical skills that define professional practice |
|
Cognitive offloading risk |
Neuroscience evidence shows most AI-assisted writers could not quote their own work shortly afterwards; researchers identify metacognitive laziness as a systematic risk |
Learners who use AI to generate assessment responses without genuine cognitive engagement may pass tasks without developing the underpinning knowledge or skills the qualification represents |
Require learners to demonstrate the cognitive process, not just the output; competency conversations, reflective tasks and observed practical demonstrations are evidence types AI cannot substitute |
4. Assessment Design in an AI World: The VET Advantage
The report's most urgent message for education systems is that assessment must be redesigned to account for AI's ability to produce competent outputs without competent understanding. Higher education institutions are grappling with this across written assignments, research papers and examination formats designed in a pre-AI world. VET, by contrast, is structurally better positioned than any other education sector to maintain assessment integrity in the age of generative AI, precisely because competency-based assessment was never primarily about written outputs in the first place.
The assessment methods most resistant to AI manipulation are the methods VET has always relied on: direct observation of workplace performance, practical demonstration of skills under realistic conditions, competency conversations where assessors probe understanding through follow-up questioning, third-party workplace reports from supervisors who observe the learner performing real tasks, and portfolio evidence collected over time and verified for authenticity. These methods require the learner to demonstrate competence in person, in real time, under conditions that generative AI cannot replicate or substitute.
This is VET's structural advantage. But it is only an advantage if RTOs actively design their assessment systems to leverage it. An RTO that relies heavily on written assessment tasks, knowledge questions with text-based responses and take-home assignments is as vulnerable to AI-generated evidence as any university. An RTO that designs assessment around observation, practical demonstration, competency conversations and workplace-based evidence is building on the strongest possible foundation for assessment integrity in an AI-disrupted environment.
The report's recommendation that education systems shift from evaluating output to evaluating process aligns precisely with VET's competency-based model. The Standards for RTOs 2025, with their emphasis on assessment system operation, pre-use validation of tools and systematic application of the rules of evidence, provide the regulatory framework within which this shift can be made. The challenge is ensuring that every RTO translates this framework into assessment practice that is genuinely AI-resilient.
5. Teacher-AI Teaming: The Augmentation Model for VET Trainers
The report proposes a conceptual framework for how teachers and AI can work together, identifying three paradigms: replacement, complementarity and augmentation. Replacement, where AI takes over tasks previously performed by teachers, risks the loss of professional skills and teacher-student interaction. Complementarity, where human judgement and machine efficiency are paired, is better. But the most effective approach, the report argues, is augmentation through collaborative engagement: an iterative process in which teachers and AI critique and refine each other's outputs, preserving professional judgement while leveraging AI's capacity for efficiency and scale.
For VET trainers and assessors, the augmentation model is a natural fit. AI can draft lesson plans, generate formative quiz questions, summarise learner progress data, prepare initial assessment mapping reviews and automate administrative tasks such as scheduling and reporting. The trainer's irreplaceable contribution is professional judgement: conducting competency conversations that probe genuine understanding, observing workplace performance and interpreting contextual cues, making holistic competency determinations that account for the complexity of real-world practice, and building the relationships with learners that motivate engagement and support development.
The data on teacher productivity supports this model. The reported 31 per cent time saving on lesson planning among English teachers is a significant efficiency gain that, applied in VET, could free trainers to spend more time on the high-value assessment and learner support activities that drive quality outcomes. The finding that AI support produced its largest gains for less-experienced tutors suggests AI can be particularly valuable for newer trainers still developing their pedagogical expertise. But the report also warns that overreliance on AI for core teaching tasks risks eroding the professional skills that make expert trainers effective. The balance between productivity gains and skill maintenance is a strategic challenge every RTO must manage deliberately.
Conclusion: The Challenge for VET Is Not Whether to Use AI but How
The OECD Digital Education Outlook 2026 makes clear that generative AI is not a future possibility for education. It is a present reality. Students are using it. Teachers are using it. It is improving task performance, saving time and creating genuine productivity gains. It is also, in specific and well-documented circumstances, undermining the cognitive processes that produce genuine learning, reducing metacognitive engagement, and creating a gap between what learners can produce with AI and what they actually know and can do without it.
For VET, this gap is the defining challenge. A qualification that certifies competency must certify genuine competency, not AI-assisted performance. An assessment system that satisfies the rules of evidence must produce evidence that is valid, sufficient, authentic and current, not evidence that was generated or substantially shaped by a technology the learner may not have access to in the workplace. A training system that serves employers must produce graduates who can actually perform, not graduates who performed well in training because they had AI support that will not follow them into the job.
The report provides both the warning and the pathway. The warning is that general-purpose AI used as a shortcut degrades learning. The pathway is that purpose-built educational AI, designed with strong pedagogy, structured tutoring strategies and teacher oversight, can enhance both performance and learning. VET's competency-based model, with its emphasis on observed performance, practical demonstration and human judgement in assessment, is structurally well-positioned to navigate this transition. But structure alone is not enough. RTOs must design AI-resilient assessment, invest in trainer AI literacy, adopt the augmentation model for teacher-AI teaming, and embed learner AI literacy across every qualification. The sector that has always assessed what people can do, not just what they can write, has the strongest foundation for maintaining educational integrity in the age of generative AI. The question is whether it will build on that foundation deliberately.
|
Summary: What RTOs Should Take from the OECD Report |
|
1. General-purpose AI improves performance but can undermine learning; the gap is documented across multiple experimental designs and contexts. 2. Assessment integrity is the primary risk: learners can produce competent-looking evidence without genuine competency, so RTOs must design AI-resilient assessment. 3. VET's competency-based model, with observation, practical demonstration and competency conversations, is structurally stronger than written-output models. 4. Trainer productivity gains from AI are real, with a reported 31% time saving on planning, but must be balanced against skill atrophy from overreliance. 5. The augmentation model, teacher-AI teaming with human oversight preserved, is the recommended approach; replacement and full automation are cautioned against. 6. Student AI adoption is mainstream, so VET learners arrive with established AI habits that assessment must account for. 7. AI literacy should be embedded as a foundation skill across all VET qualifications. 8. Purpose-built educational AI tools outperform general-purpose chatbots for learning; RTOs should advocate for and invest in education-specific solutions. |
References and Further Reading
OECD (2026). OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education. OECD Publishing, Paris. https://doi.org/10.1787/062a7394-en
Bastani, H. et al. (2024). Generative AI Can Harm Learning. The Wharton School. http://dx.doi.org/10.2139/ssrn.4895486
Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Task. arXiv preprint arXiv:2506.08872 (preliminary, not peer reviewed).
OECD (2025). TALIS 2024: Teaching and Learning International Survey. OECD Publishing, Paris.
Liu, Y., Huang, J. and Wang, H. (2025). Who on Earth Is Using Generative AI? Global Trends and Shifts. World Bank Group Policy Research Working Paper.
Australian Skills Quality Authority. Standards for RTOs 2025. https://www.asqa.gov.au/rtos/2025-standards-rtos
