Covid challenges: the case of The Economics of Financial Markets
The rapid move to online teaching during the COVID-19 pandemic created an obvious challenge for universities: how can we design assessments that measure learning when opportunities for academic misconduct increase? Our experience in The Economics of Financial Markets with Prof Rodney Tyers suggested a broader question. Could the design of assessment affect not only cheating at the time of the exam, but also how much students learn throughout the semester?
The Economics of Financial Markets is a postgraduate economics unit at the University of Western Australia, taken primarily by students in the Master of Applied Finance and Master of Economics. Cohorts typically included between 50 and 90 students and were academically diverse, ranging from students with substantial quantitative training to those with relatively little prior economics or finance.
The first part of the course develops core financial concepts, including bond pricing, financial intermediation and information asymmetries, decision-making under uncertainty, portfolio choice and the CAPM. The second part develops a two-period general equilibrium model of a small open economy, which students use to analyse real, financial and policy shocks.
Online delivery
Teaching these concepts online also required experimentation. I used a lightboard that allowed handwritten equations to be superimposed on other material displayed on screen. For example, after deriving the monetary multiplier, I could immediately combine the equations with FRED data showing the surge in money demand during the Global Financial Crisis and discuss the associated monetary-policy problem sometimes described as pushing on a string. The objective was to reproduce some of the interaction of a blackboard lecture while making the link between economic theory and real-world evidence more immediate.
Superimposition of a real-time graph. Video editing gives more options like this example where using Adobe Premiere Pro I superimposed another video source, a FRED graph, on top of the video stream recorded in the lightboard room.
Assessment changes incentives to learn
The more difficult COVID-era problem was assessment.
Online written assessments substantially reduced the cost of coordinating answers, consulting unauthorised material, or using third parties. Our report on the course notes that moving assessment online had made cheating considerably easier, while investigating suspected cases imposed substantial administrative costs on teaching staff.
Remote oral assessment. For students that were outside Western Australia or with symptoms of covid we allowed remote assessment over Zoom.
But the important point is that cheating does not only affect the validity of the final mark. If students anticipate that an assessment can be circumvented, it alters their incentives to study before the assessment.
A simple microeconomic argument makes this clear.
Suppose a student chooses how much study effort, (e), to exert during the semester. Effort is costly, but it improves the student’s expected mark. Cheating provides an alternative way of increasing the expected assessment result. Whether cheating is attractive depends partly on the probability that it will be detected.
When students know in advance that cheating is relatively easy and unlikely to be detected, the expected return to genuine preparation falls. A rational student at the margin therefore chooses less study effort. Conversely, when an assessment is perceived as difficult to cheat, genuine preparation becomes relatively more valuable.
The theoretical prediction is therefore:
The lower the perceived opportunity to cheat in an assessment, the greater the incentive to learn the material before the assessment.
In the report with Rod Tyers, we formalised this intuition using two student types. Some students would attempt to cheat regardless of enforcement, while a second group decides whether to do so depending on the expected return relative to the probability of being caught. For this latter group, easier cheating rationally reduces study effort during the semester.
Assessment design endogenously determines learning. An assessment is not simple measurement device of learning but an input into the production of learning.
Introducing the oral examination
To change these incentives, we replaced the traditional final examination with an individual, closed-book oral assessment worth 30% of the final grade.
Students received 10 representative questions drawn from workshops and assignments one week before the examination. Preparing credible answers required them to review approximately 90% of the course material. During the exam, each student randomly drew 1 question, presented an answer for up to 15 minutes using paper or a whiteboard where necessary, and then answered follow-up questions that could draw on other areas of the course.
This design was important. Students knew the possible questions in advance, so the objective was not to reward rapid recall or surprise them with obscure material. But because they did not know which question they would receive and because follow-up questions could probe their understanding, preparing somebody else’s answer was of little value. The safest strategy was to understand the material.
There were suggestive signs that this changed behaviour before the final assessment. Lecture and workshop attendance was unusually high, and we observed greater depth in students’ questions during the semester. Students also asked considerably more about the structure of the oral assessment, consistent with increased preparation in anticipation of it.
Serious academic misconduct cases fell to zero, compared with seven cases in Semester 2 of 2020 and three in Semester 1 of 2020.
Oral examinations are not costless. Each student required around 20–30 minutes, and scaling this format to very large cohorts would be difficult. Yet written examinations also have costs that are easy to overlook: marking, invigilation, misconduct investigations, interviews and administrative procedures. In our experience, the oral exam also provided something that conventional written assessment often does not: an unusually clear picture of what a student actually understood.
The broader lesson from this COVID-era experiment is therefore not simply that oral examinations can reduce cheating. It is that credible assessment changes incentives. When students believe that success will require demonstrating their own understanding, studying becomes the most reliable strategy.
And that makes even final assessment part of the learning process itself.