USAf’s Teaching and Learning of Mathematics Community of Practice learns of an innovative use of AI to support learning

An artificially intelligent learning, tutoring and assessment programme, ALEKS – combined with the crafting of a student support system using National Benchmark Test (NBT) data – was incorporated into the teaching and learning of mathematics this year.

This was the message given to participants at the last meeting of the Teaching and Learning of Mathematics Community of Practice (TLM CoP) for 2023, virtually convened under the auspices of Universities South Africa.

• ALEKS 
  ALEKS is a digital solution system founded by Dr Jean-Claude Falmagne (New York University and the University of California, Irvine (UCI)) and owned by McGraw Hill. The artificially intelligent learning and assessment system utilises big data and machine learning.

Leading into her presentation, Ms Sango put into perspective the work her team were engaged in, explaining how, over the five years of work on DMISRS, different themes were set for the various collaborations and symposia.

• The 2018 themes included the curriculum; open educational resources (OERs), staff development and motivation; understanding students and best practices in blended learning.
• 2019 focused on collaboration; courses that impede graduation; Mathematics and/for other disciplines as well as the psychological well-being of students.
• The 2020 themes centred around first year Mathematics solutions as well as the use of assessments, performance, blended Learning and curriculum development.
• In 2021, topics were student profiles and mathematics performance and online teaching and assessment
• 2022 concentrated on teaching 1st year Mathematics and the future of South African Mathematics that incorporated critical thinking as well as support and assessment.

Said Ms Sango: “The first proposal for the DMISRS project in 2018 outlined overall objectives and goals. Even back then there was a leaning towards blended learning.”

The immediate objective was to ensure a large data pool of Mathematics information was well articulated and disseminated throughout the institutions, and informed different current curriculum-integrated support initiatives.

“It was hoped that institutions would be persuaded to use the information resulting from an analysis of NBT results as a tool for placement and support – as well as the curriculum-integrated support initiatives identified – to determine the type and extent to which this could be implemented.”

The long-term objective was to benefit students, supporting them during their transition year and during their time at university.

In 2021/2022, UCT implemented a Know Your Students / Know Your Course project and started using the NBT data as part of the data analytics for student success. “We always say that NBT and National Senior Certificate (NSC) data complement each other but when we zoom in onto a particular cohort of students, we can see the performance over the years.”

Ms Sango explained, through charts, how the research data (which she said was easily available to academics) painted an accurate picture of this.

She said they ran teaching and learning workshops at various institutions to put the NBT domains and subdomains into conversation regarding courses. They performed mapping exercises, identifying where different skills in NBT play a role and what impact they have on teaching and learning activities.

She said: “All that led to the first implementation this year, 2023. We designed student support leveraging the NBT data and the digital solution ALEKS, an established platform as a proof of concept. This allowed us to explore how we can use adaptive learning technology to craft the support we’ve been talking about.”

Outlining the research behind ALEKS, Ms Sango explained that it applies combinatorics and stochastic processes theory to the modelling and empirical description of particular domains of knowledge. It also uses Knowledge Space Theory to determine precisely what each individual student knows, and what the student is ready to learn next.

“Despite the availability of valuable data on student performance, the manual analysis and

implementation of support programmes remains a time-consuming and resource-intensive task for first-year lecturers who are already burdened with their teaching responsibilities. The need is for a smart solution.”

Many students beginning their higher education journey are unprepared for the demands of university-level study which has implications for student retention, attrition rates, and their overall engagement with the learning process.

Ms Sango: “The NBT Adaptive Learning pilot programme aims to develop NBT ALEKS curricula designed to support first year students exhibiting gaps in mathematics knowledge. “The objective of the adopted approach is to investigate the potential of using an adaptive learning system to facilitate a smoother transition for students entering tertiary education.”

The NBT ALEKS solution uses the NBT data and relative importance analysis to inform course- specific parallel intervention programmes that do not require additional time from lecturers. It also integrates with the course and supports students’ concept development by closing the gaps in their prior learning – which has been identified by the NBT diagnostics – while having most impact in the courses they are enrolled in.

“Because this was only implemented half way through 2023, we missed the ideal first six week transitioning period between high school and first year. But we have the relevant importance analysis for different courses and have translated them into the curriculum that will be the most relevant for the students who are studying specific courses.”

Ms Sango said that because the system was adaptive, it would learn something about each student and address their specific gaps within the context of the course they are studying.

She suggested candidates watch these videos.
Demo video: https://www.youtube.com/watch?v=nqwtqJdH-lw

Implementation example: https://mcgrawhillcanada.hubs.vidyard.com/watch/eyAUrsg6fT53R5p46c8WUg

ALEKS implementations and early findings (the new programme only began in June 2023)

They invited students from four first-year mathematics courses in different performance categories: those who failed the first-semester course but qualified for supplementary examination and students who passed the first-semester course, but whose final course mark was low.

“We are still analysing the NBT scores, the course assessment data, and the time spent in ALEKS and the performance patterns.

An intervention for Grade 12 learners was done in September. “This was closer to our ideal thinking because these are soon to be students – they are part of the UCT 100UP Project – some of them will be transitioning into first year in 2024. Those enrolled in our ALEKS programme wrote their first test in August. The course curriculum was designed for them with 168 different topics. ALEKS was complementary to face-to-face weekly tutoring sessions.”

100UP is a UCT initiative addressing the problem of demographic under-representation in higher education by targeting school learners from disadvantaged backgrounds and coaching them towards university access.  The holistic programme strives to build intellectual, social and cultural capital. Learners are prepared and coached over a period of three years by staff and students across the university, working off the key belief that opportunity needs to be made more equal and inclusive.

A third pilot did not materialise while the fourth pilot, with the Faculty of Health Sciences at UCT, looked at foundational mathematical skills. A small study looked at creating a course for psychology students with a subset of 47 different skills in the intervention programme. .

“Where there is a strong involvement of tutors or lecturers (not involvements as in monitoring but in making this programme compulsory for students) we see better engagement and better results,” Ms Sango said.

She added that the data has yet to be analysed and it was too early for any definitive conclusions. “Although we cannot yet claim that ALEKS made a difference, there appeared to be a correlation between time spent on learning and the improvement in performance.

“Results from the tests show that the students were engaged; you could also see the topics the students are learning and how much time they are spending on them.

“Part of the investigation will be to map the sub domains (especially those present in mathematics NBT) back to the NBT scores once we start analysing the data.”

In terms of 100UP, Ms Sango said it was interesting that while initially students claimed that their access to technology made the process difficult, two weeks prior to their second attempt at NBT there were spikes of time spent in the learning with a larger number of topics. “So, there is interest and participation and engagement from students if there is a tangible reward.”

A note for the TLM CoP on Collaborative Projects planned for 2024

• ALEKS licences are available (free for a limited pilot) for the 2024 First Semester 1st year courses
• Centre for Educational Assessments will provide workshops on NBT data analysis, domain mapping, and the design of student support curricula tailored to specific courses
• Upcoming and current engagements with the Faculty of Health Sciences at UCT, Tshwane University of Technology Computer Sciences, Rhodes University, University of Zululand.

Charmain Naidoo is a contract writer for Universities South Africa.