Advancing doctoral assessment strategies; the importance of standardisation and the viva voce mode of assessment

Assessment standards as applied in postgraduate (especially doctoral) research proposals, dissertations and written and oral examinations were the subject of discussion at a recent meeting of the Community of Practice for Postgraduate Education and Scholarship (CoP PGES) of Universities South Africa (USAf). Central to the group’s deliberations on 25 July was the need to standardise implementation of the Doctoral Standard across universities and the potential adoption of viva voce as a key doctoral assessment model.

Introducing this topic The Elements of Assessing Postgraduate Degrees – Dissertations, Theses, Written and Oral Examinations, Professor Stephanie Burton, Chairperson of the CoP PGES and Programme Leader of USAf’s Advancing Early Career Researchers and Scholars (AECRS) programme, noted the importance of high-quality assessment in postgraduate studies. She also mentioned a myriad of divergent practices identified at different institutions, adding that despite some notably good practices in the system, “the varied approaches can sometimes result in inconsistencies in assessment standards.”

Underlining the need to identify best practices that could enhance the consistency and quality of postgraduate assessments, Professor Burton invited the CoP members to use the insights provided by three senior academics invited to the 25 July meeting to explore ways to harmonise assessment practices across institutions. She said that was the only way to ensure that all students are evaluated according to the highest academic standards.

The knowledge component focuses on the graduate’s ability to contribute original insights and develop new theoretical frameworks, while the skills category includes the graduate’s demonstration of communicative competence at an advanced level.

“Inconsistencies in thesis submission, assessment procedures, and supervision practices persist across institutions,” Professor Leitch stated.

To tackle these disparities, Professor Leitch suggested that institutions need to implement consistent assessment strategies throughout the doctoral degree process, from the initial proposal stage to the final thesis submission. He recommended structured feedback mechanisms to ensure candidates effectively develop and demonstrate the required competencies. He pointed to the cohort supervision models as particularly effective in fostering collaborative feedback and producing successful graduate outcomes.

“Cohort supervision offers broader input and shared insights, which can significantly enhance the quality of doctoral research,” he added, contrasting this approach with traditional supervision methods.

Professor Leitch also addressed the limitations of relying solely on remote assessments, advocating for the inclusion of oral examinations, such as viva voce, for a more comprehensive evaluation. He argued that such oral assessments provide for interactive discussion and a deeper exploration of a candidate’s understanding of their research.

“Oral assessments provide a deeper level of evaluation by enabling direct interaction between examiners and candidates,” he remarked, reinforcing the importance of these face-to-face interactions in the doctoral journey.

“Through direct engagement with experts, candidates demonstrate their intellectual ownership of the thesis, which is essential for their integration into the academic community,” she noted.

She also revealed the role of viva voce in safeguarding the integrity of doctoral research, explaining that it “provides a formal platform for assessing the student’s research presentation and engagement”. While this can serve as a check against issues like plagiarism and over-reliance on Artificial Intelligence (AI) in the final thesis, she urged that concerns in this regard should be tackled prior to the viva. “The literature is clear”, she argued, “using the viva to police authorship undermines its potential to attend to its educational benefits”. Nonetheless, the viva helps clarify intellectual ownership and addresses any ambiguities in thesis submissions.

Despite its benefits, the adoption of the viva voce model across universities remains inconsistent. “Implementation remains uneven among institutions,” Professor McKenna observed, stressing the need for institutional policies to enhance the rigour of doctoral evaluations in line with global trends.

Professor McKenna also highlighted the additional benefits of viva voce, particularly its role in enabling examiners to provide targeted feedback, which can significantly refine and enhance the candidate’s research. This process, she noted, deepens the candidate’s integration into the academic community through direct interaction with peers and experts.

She called for the establishment of clear guidelines, suggesting that specific instructions could reduce the secrecy that often surrounds doctoral examination and enhance the overall effectiveness of the process. According to Professor McKenna, the role of the chairperson is crucial in setting the tone for a balanced and supportive environment that fosters constructive feedback and development for the candidate.

Dr Andrew Kaniki (below), retired former Executive Director of Knowledge Advancement and Support at the National Research Foundation (NRF) was invited to the CoP PGES meeting, specifically to respond to the perspectives discussed above.

He called for collaboration between USAf and the CHE’s Higher Education Quality Committee (HEQC) to develop standardised guidelines that would outline clear procedures and standards for evaluating doctoral candidates. This initiative, developed through the similar process as done for the Doctoral Standard according to Dr Kaniki, would “ensure consistency and fairness across institutions.” These would be guidelines, not laws, that universities can adopt without impeding on their academic independence.

Dr Kaniki also proposed the establishment of a robust system for managing the appointment of examiners, designed to “identify and exclude those who do not meet established criteria”, thereby preserving the integrity of the assessment process. He emphasised that this approach would “uphold rigorous standards and prevent potential biases by ensuring that only qualified and impartial examiners are involved.”

In addition, Dr Kaniki highlighted the importance of integrating written doctoral theses with viva voce (oral) examinations, arguing that “combining these components offers a more thorough evaluation of doctoral candidates.” He noted that while written theses provide a detailed account of research, viva voce examinations allow for real-time questioning and discussion, which he believes would “enhance the quality of doctoral training and provide a more accurate assessment of the candidate’s knowledge, research skills, and ability to defend their work.”

This is an edited version of some of the discussion that ensued in the CoP PGES meeting:

Question One: Professor Lesetja Legoabe, Director at the Centre of Excellence for Pharmaceutical Sciences (Pharmacen) at North-West University: Well done to both presenters for their excellent presentations. I just have concerns about the public viva model. How is confidentiality managed when members of the public are invited to participate in viva voce sessions? Could this model compromise the confidentiality of the examination process?

Professor McKenna: While public vivas are used in some universities, students can embargo their oral presentations, like their theses. This allows the process to remain confidential if necessary. This practice ensures that vivas do not have to be public events if confidentiality is a concern.

Question Two: Professor Nontyatyambo Pearl Dastile, Director of the Postgraduate Unit at Walter Sisulu University: How can a single, standardised document address the diverse issues across higher education institutions in South Africa? Also, what are your thoughts on the potential for repeated appointments of the same examiners, which might undermine the integrity of graduate attributes and qualifications?

Professor Leitch: The document is meant to be a guideline; it will not be a one-size-fits-all approach, but rather a framework that guides institutions on effectively employing the viva voce as a doctoral assessment strategy. This document can be adapted based on programmes offered by the institution and should align with institutional policies. I believe that maintaining a database of examiners within faculties could help manage this issue of the repeated appointments of examiners. While a national system might be impractical due to institutional autonomy, a local database could help monitor and address the repeated use of examiners.

Question Three: Dr Carol Nonkwelo, Senior Director of Research, Innovation & Postgraduate Education at the University of Pretoria: How can we improve supervisory practices, especially with established supervisors who may feel they have nothing new to learn? Any suggestions on enhancing the supervision process?

Professor McKenna: I believe that supervision development courses are important to enhance the quality of supervisors but making them compulsory can be challenging. It is beneficial to foster an environment where both experienced and novice supervisors engage in these courses. However, problematic supervisors often do not participate in these courses, deeming them unnecessary; this is a challenge we should consider going forward.

Professor Leitch: While courses should not be compulsory, supervisors need to move beyond outdated practices. The sector is gradually replacing older supervisors, which could introduce new perspectives. Persuasion rather than enforcement is suggested, as enforcing new practices can be difficult if supervisors resist change.

Question Four follow-up by Dr Kaniki: What about regulatory measures? Student-supervisory contracts or Memorandums of Agreement can be crucial in enforcing behaviours and improving supervisory practices.

Professor Leitch: The focus should be on changing attitudes towards students rather than just compliance. It is crucial to foster a mentoring attitude among supervisors and persuade them to adopt more supportive practices. Despite the challenges in changing entrenched habits, efforts must be made to shift attitudes towards more effective supervisory practices.

USAf’s Community of Practice on Postgraduate Education and Scholarship (CoP PGES) supports the national higher education sector in postgraduate teaching and learning, through the training and support for supervisors, linking postgraduate students and candidates to prospective mentors and enables collegial interaction between peers. The CoP PGES also facilitates the sharing of resources and exchanging good practices for the enrichment of postgraduate — especially doctoral study programmes.

Professor Stephanie Burton, who chairs the CoP PGES, is a Professor at the University of Pretoria’s Future Africa campus, and USAf’s Research Fellow.

Kayley Webster is a Communication Consultant contracted to Universities South Africa.