Commentary
Rewarding progress: Effective learning strategies through a variable ratio incentive-based approach in medical education

Abanoub Aziz Rizk¹ Nicolas Mourad² Marina Saad³ Emanuel Aziz Rizk⁴ Meena Saad⁵
Published online: February 7, 2022

¹University of Ottawa Faculty of Medicine
²University of Waterloo Honours Science Program
³University of Ottawa Faculty of Medicine
⁴York University Faculty of Health
⁵Memorial University of Newfoundland Faculty of Medicine
DOI: 10.26443/mjm.v20i2.914

Introduction

The traditional medical curriculum offered in North America typically consists of two years of lecture-based science acquisition (termed pre-clerkship years) followed by two years of clinic-based application (termed clerkship years) (1). Although this structure has been repeatedly shown to prepare medical students successfully for a career in healthcare, the curriculum, specifically the lecture-based pre-clerkship years, may consist of learning objectives too copious and too advanced to truly benefit medical students at the introductory level. (2) To combat this, there has been a trend to transition from traditional lecture-based approaches to practical problem-based learning which combines basic science knowledge with clinical skill development. (3) In addition, medical schools have taken the initiative to study and implement flipped classroom teaching methodologies in which students are given the time to independently explore study topics before group sessions. (4) This method uniquely features the student as a leader; meanwhile, the instructor acts as a facilitator rather than an educator. This movement towards a student-centered approach is ideal because it adequately prepares students for real-life situations that they will face throughout their medical career. (2) The aim of this commentary is to present some of the current learning and teaching strategies utilized within medical education across the world and to present a novel supplementary approach to medical education involving a variable ratio (VR) incentive-based system of active recall (AR) and spaced repetition (SR). This paper introduces a recommendation to supplement already existing study and testing techniques utilized by medical schools. Although it has not yet been tested, the aim is to inspire medical school administrations to pilot a program that incorporates such study techniques in order to provide experimental evidence to compare with the supporting theoretical data.

Challenges faced by medical students

Despite numerous advancements in education, medical students are still presented with an overwhelming volume of material that they must comprehend and retain. Moreover, physicians are life-long learners; as such, their medical education should serve as the backbone on which continued learning stands. (1) That being said, studies have shown that simply reading and explaining lecture content to oneself is a suboptimal and superficial learning method for long-term retention of information. (5)

Increased academic demand paired with seemingly inefficient learning strategies have been shown to contribute to anxiety, feelings of inadequacy, inability to cope, self-doubt, and intellectual fraudulence which has become known in academic circles as “imposter syndrome”. (6) Difficulty coping with this stress can promote thoughts of dropping out and cause a deterioration in academic performance. (7) Even more concerning is the rates of psychological comorbidities such as depression and suicidal ideations seen among medical students. A meta-analysis of nearly 63 000 medical students internationally found that 33.5% of medical students reported feelings of depression within their first year with that number dropping slightly to 23.5% towards the end of their medical education (8). This is significantly higher than the range of 8.8-9.6% of young adults with depression reported by the Journal of the American Academy of Pediatrics (9) Also of concern is the need that medical students felt to spend the majority of their time studying and the feeling of guilt whenever they engaged in other activities during their spare time. (10) A qualitative study looking at medical students’ perception of stress found that students began compromising their health by not exercising or manipulating their sleep schedules in order to keep up with their studies. (10) Similarly, a cross-sectional study administered a questionnaire to 700 international medical students and found that nearly 50% of a medical student cohort self-reported psychological distress throughout their medical school education. (11) High levels of exhaustion are often associated with increased medical mistakes, negative attitude toward patients, burnout, and reduced likelihood of success in the healthcare field. (12, 13)

Current Testing Methods

Many different testing methods are currently being utilized by medical schools across the world. Some of these methods include but are not limited to essay style questions, multiple choice exams, group projects, matching items, case assessments and Objective Structured Clinical Exams (OSCE). (14) It has been suggested that there are important factors for faculties to consider prior to choosing which method would work best for their cohorts such as what material is to be assessed, the reason behind the assessment, and the validity and reliability of such methods. (14) For example, although patient management problems, which stimulates a clinical environment, asks students to pick the appropriate solution and provides them with the results of the action they select, seem to be heavily relied upon by medical schools as the preferred testing styles, they have been shown to lack face validity. (15) In fact, utilizing a method such as multiple-choice questions yielded better outcomes for students, with better reliability and validity in practice. (15) In addition, even within multiple choice testing, the number of choices has been a topic of discussion and multiple papers have proposed different ideas for the optimal number of selections. (15) Furthermore, extended discussions have also taken place with regards to implementing open-ended questions in written summative assessments, in which a definitive conclusion about its use was difficult to make. (16) On the other hand, a literature review assessing study strategies found that medical students who engage in self-directed retrieval methods with spaced repetition perform better on formal examinations that assess clinical knowledge. (17) Another study recruited 72 medical students to complete a survey regarding study strategies which showed the benefits of retrieval practice in a classroom setting and illustrated advantages of utilizing many spaced repetition initiatives. (18) While there has not been a consensus on the best testing methods for medical schools, it appears that some strategies have shown more promising outcomes than others.

Current Studying Techniques

Medical school students must commit long hours to understand, retain, and apply the large amount of information which is taught throughout their education. (19) Although each student differs in their approach to learning, there are several common trends in the study techniques utilized by medical students around the world. According to a study investigating medical students’ study habits, the overwhelming majority of students (77%) utilized self-made summary notes based on lecture material which they then committed to memory. (20) Others utilized a slightly more advanced learning strategy by designing charts, tables and flow-charts into their notes. However, only 27% of participants used evidence-based active learning strategies. (20) A study method that is gaining much traction is Anki – an open-source program designed to reinforce concepts by implementing knowledge retrieval via flashcards. Anki is designed to allow students to self-assess their knowledge (i.e., if they feel the concept to be “easy”, it will not be tested for a longer duration than a concept that is deemed “hard”). A literature review study conducted in the United States showed that students who self-test in preparation for exams often use traditional multiple-choice strategies while only 31% of students used Anki as a preparation tool, despite being associated with higher scores on formal examinations. (17) Another successful study strategy, albeit uncommon, is by utilizing case-based learning. This was shown to be positively correlated with exam results in a medical student cohort during their emergency psychiatry learning section. (21) Although studies have shown the advancement of students towards evidence-based learning strategies such as active recall and spaced repetition (22), a significant portion of students remain dedicated to the traditional methods of studying.

Barriers to Change in Medical Education

The many challenges that medical students face are not foreign to medical school faculty. However, implementing new learning strategies has proved challenging due to a lack of resources and limited time. (23) A great deal of consideration must be taken prior to making changes in a medical school curriculum to ensure it appropriately accommodates for the students’ needs. Implementation of new curricula would require constant evaluation by faculty members, frequent communication and feedback between faculty and students, training support to faculty members, an evaluated reward structure, and prevention of a drop in the students’ performance as they adapt to a new strategy. (23) Even after these changes are made, ensuring smooth implementation and efficiency of a novel approach is a time-consuming and resource-consuming endeavour. (5)

Evidence Based Learning Strategies

Although many studies have thoroughly investigated evidence-based learning methods, some proven learning strategies are not a standard component of current medical education. (24) One such strategy is Active Recall (AR), in which a student is prompted to recall information through techniques such as flashcards. This has been repeatedly proven to be superior to recognition strategies such as those utilized by multiple choice style questions. (25) In fact, even If students fail to correctly answer a question, active recall learning strategies have been shown to optimize retention of information when compared to other strategies. (24)

The second method of learning is Spaced Repetition (SR), which individualizes questions to the learner. (27) In other words, concepts with which the learner is struggling are repeated more often than concepts that have been mastered. (27) This has been shown to maximize the efficiency of study time and optimize long-term retention. (26)

Additionally, expanding retrieval practice is a technique which can be combined with spaced repetition and active recall for optimal results. (26) This involves gradually increasing the time interval between testing sessions and has been proven superior to other study strategies. (27) Moreover, if students are consistently participating in such retrieval practice, the information they retain will be accurate and resistant to extinction, relative to other methods of study. (24)

Incentivized Learning

Therefore, such methods of learning are effective for students to grasp and engage with the new content. Despite this, they are not actively used in traditional medical school curricula. (4) As such, students who may be currently applying this method of learning must do so independently, without any direct means for reinforcement or reward.

There are many methods of reinforcement that motivate individuals to engage in a certain task. According to the psychology textbook, The Psychology Around Us (28), reward strategies are typically characterized into four categories: 1) Variable interval (VI) in which a response is rewarded after an unpredictable amount of time has passed. 2) Variable ratio (VR) in which a reward is given after an unpredictable number of responses. 3) Fixed Interval (FI) in which the participant is rewarded after a specific amount of time has passed. 4) Fixed ratio (FR) in which reinforcement occurs after a specific number of responses. Although constant reinforcement has proven to be successful for the early stages of learning, optimal long-term retention of information is achieved most rapidly through VR reinforcement schedules. (29) Along with being the most effective as a motivational strategy, VR has also been identified as the method that is most resistant to extinction. (28)

Innovative Integrative Solution

Based on these evidence-based learning and reward strategies, we propose a system which uses an incentive-based approach to motivate students to engage in daily self-testing and recall activities. Students will be marked on completion of the activities and would receive positive reinforcements in the form of grades, delivered through a VR schedule, which will count towards their final semester marking scheme. This activity will be optional as some students may not benefit from this additional intervention as they are already optimizing their AR and SR strategies.

Implementation

While there are many ways students can implement AR and SR strategies, one practical method we recommend involves a VR reinforcement based on students’ daily flashcard completion. This method uses a pre-existing program named Anki, a free open-source software that uses AR/SR algorithms to improve retention of information through flashcards. (30) Many developers have created public add-ons for users to increase productivity and enhance program features. We chose to utilize Anki as our primary program, instead of other applications, as its resources are easily accessible online and free of charge.

In light of this initiative, we have developed an add-on that keeps track of the daily number of cards completed by students [Figure 1 and Figure 2]. The decks may be pre-made by senior students or faculty members. However, the faculty will be responsible for standardizing decks to ensure high quality and fair assessment. Medical schools will be able to modify the minimum required number of cards per day. Moreover, the faculty will also be able to modify the amount of content that should be covered per session. For instance, one school may choose to utilize a cumulative format whereas another may prefer shorter, non-cumulative testing strategies. For this reason, decisions will be left up to individual schools to adjust the add-on in accordance with their unique objectives and teaching goals.

At the end of each successful day, the add-on will automatically reward students with a grade percentage using a VR reinforcement system. For example, a student may successfully complete their decks for two days and receive a reward on the third day in the amount of 0.5%. Meanwhile, another student will receive their reward on the fourth successful day. This will motivate students to continue using the decks on a daily basis as they do not know when the reward will be given. The addition of all rewards will maximally equate to 10% by the end of the semester (or the agreed-upon percentage by the faculty). The mark will then be incorporated into the traditional semester marking scheme to incentivize daily completion of cue cards.

Depending on the medical school, the way students acquire cards may differ. Some schools may choose to provide students with faculty-reviewed cards, student-curated decks, or United States Medical Licensing Exam (USMLE) decks found online free of charge.

Alongside establishing a question bank, there also needs to be time dedicated to educating students about this initiative and incorporating it into the traditional marking scheme. Medical schools will need to adequately educate students on the benefits of such initiative and provide them with training for the designated software.

In addition to utilizing evidence-based learning strategies and an incentivized participation system, this method provides some additional benefits. First, by incorporating an evidence-based approach to learning and reward, medical schools could potentially assist students in coping with new academic challenges. This would be achieved by shifting the yearly marking scheme from a traditional examination approach to a more engaging daily incentive building initiative. Research has shown that, among other factors, participation and high self-efficacy beliefs among medical students contributed to higher test performances and increased motivation for learning. (31) As such, participation in the proposed activity will allow students to be constant participators and will provide rewards. This will increase self-efficacy and motivation, leading to better overall performance. Furthermore, given the data collected by the software, it will be possible to track the progress of students and to acknowledge the material that students find difficult. Faculty can then use this knowledge when updating or designing exams or other assessments. Moreover, faculty can determine large areas of content that are difficult for students to grasp and dedicate more time and resources to teaching this in future years. The discussed method should not consume much of the staff’s limited time as it will only require the initial setup of the question bank, which can be re-used each year. The questions may be modified over the years as the medical school curriculum advances and adapts. However, the bulk of the curriculum should remain relatively unchanged as it represents well-established scientific principles.

Limitations

Despite the presented effectiveness of AR/SR learning strategies, certain limitations pertaining to our initiative need to be addressed. First, since this activity will be done outside the classroom, students may be inclined to simply guess or randomly select answers to obtain participation marks. However, by doing so, students will not benefit from AR/SR and would be misusing their time. It is important to note that AR/SR are effective learning strategies because they are personalized to the individual’s knowledge. (24) It is therefore necessary to initially educate students on the efficacy of such evidence-based approaches to incentivize participation. Additionally, the purpose of this study technique is not limited to assessing current knowledge, but rather to encourage students to actively participate and become engaged in the subject material. This rationale is supported by the fact that benefits from AR/SR occur even when responses are not correct. (24)

Moreover, the implementation of electronic flashcards may not be suitable for all students, particularly auditory learners. While this initiative may work best for specific groups of students, we believe the concept of utilizing AR and SR should be promoted, even if it is through other means. In the future, we hope to be able to incorporate more Anki add-ons that incorporate different learning strategies. By adding options such as auditory cards or incorporating videos into the explanation of answers, we diversify this learning strategy to more types of learners and facilitate a more inclusive environment. The AR/VR strategy also has the potential to motivate more students to begin and commit to these learning strategies. Although the AR/SR/VR initiative could increase student workload, we recommend that routine feedback surveys be implemented to allow faculty to meet the needs of their cohorts and adjust accordingly.

As this is paper simply recommends a technique that has not yet been tested in medical education, there are quite a few unknowns. Future studies are needed to comprehend the usefulness of this proposed intervention. Feedback forms are one method that may be utilized to evaluate the student interest and participation in this program. However, experimental studies should be considered to discern true effectiveness by looking at variables including but not limited to: Mental health and well-being of medical students (through self-reported feelings of anxiety, depression, imposter syndrome etc.), scores on exams focused on knowledge acquisition (specifically in the pre-clerkship years), and clinical application skills in clerkship years.

Conclusion

Medical education involves a substantial amount of content that must be comprehended and applied successfully before a student can proceed to the next phase of their training. In this paper, we proposed an evidence-based learning approach in which students are motivated to continuously recall content through AR and SR methods. By providing VR incentives in the form of participation grades, students are encouraged to participate in these strategies, which have been proven to increase the persistence of students and minimize extinction. (27) Successful implementation of new strategies will require collaboration among medical school faculty and students. We, therefore, recommend that medical educators implement this strategy and tailor it to their specific curriculum and mission statements.

References

1. Finnerty EP, Chauvin S, Bonaminio G, Andrews M, Carroll RG, Pangaro LN. Flexner Revisited: The Role and Value of the Basic Sciences in Medical Education. Academic Medicine. 2010;85(2):349-355. doi:10.1097/acm.0b013e3181c88b09
2. Leggat PA. Traditional and innovative approaches to medical education in Australia and the move to graduate schools. Medical Teacher. 1997;19(2):93-94. doi:10.3109/01421599709019358
3. Schwartzstein RM, Roberts DH. Saying Goodbye to Lectures in Medical School — Paradigm Shift or Passing Fad? New England Journal of Medicine. 2017;377(7):605-607. doi:10.1056/nejmp1706474
4. Premkumar K, Pahwa P, Banerjee A, Baptiste K, Bhatt H, Lim HJ. Does Medical Training Promote or Deter Self-Directed Learning? A Longitudinal Mixed-Methods Study. Academic Medicine. 2013;88(11):1754-1764.
5. Larsen DP, Butler AC, Iii HLR. Comparative effects of test-enhanced learning and self-explanation on long-term retention. Medical Education. 2013;47(7):674-682. doi:10.1111/medu.12141
6. Mullangi S, Jagsi R. Imposter Syndrome: Treat the Cause, Not the Symptom. JAMA. 2019;322(5):403-404. doi:10.1001/jama.2019.9788
7. Fares J, Al Tabosh H, Saadeddin Z, El Mouhayyar C, Aridi H. Stress, Burnout and Coping Strategies in Preclinical Medical Students. N Am J Med Sci. 2016;8(2):75-81. doi:10.4103/1947-2714.177299
8. Puthran, R., Zhang, M., Tam, W., Ho, R., 2021. Prevalence of depression amongst medical students: a meta-analysis.
9. Mojtabai, R., Olfson, M., Han, B., 2016. National Trends in the Prevalence and Treatment of Depression in Adolescents and Young Adults. PEDIATRICS, 138(6), pp.e20161878-e20161878.
10. Bergmann C, Muth T, Loerbroks A. Medical students’ perceptions of stress due to academic studies and its interrelationships with other domains of life: a qualitative study. Medical Education Online. 2019;24(1):1603526. doi:10.1080/10872981.2019.1603526
11. Fawzy M, Hamed SA. Prevalence of psychological stress, depression and anxiety among medical students in Egypt. Psychiatry Res. 2017;255:186-194. doi:10.1016/j.psychres.2017.05.027
12. Patel RS, Bachu R, Adikey A, Malik M, Shah M. Factors Related to Physician Burnout and Its Consequences: A Review. Behav Sci (Basel). 2018;8(11):98. Published 2018 Oct 25. doi:10.3390/bs8110098
13. Jafari N, Loghmani A, Montazeri A. Mental health of Medical Students in Different Levels of Training. Int J Prev Med. 2012;3(Suppl 1):S107-S112.
14. Assessment methods in medical education. Int J Health Sci (Qassim). 2008;2(2):3-7
15. Norcini JJ, Swanson DB, Grosso LJ, Webster GD. Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence. Med Educ. 1985;19(3):238-247. doi:10.1111/j.1365-2923.1985.tb01314.x
16. Vyas R, Supe A. Multiple choice questions: a literature review on the optimal number of options. Natl Med J India. 2008;21(3):130-133.
17. Hift RJ. Should essays and other "open-ended"-type questions retain a place in written summative assessment in clinical medicine?. BMC Med Educ. 2014;14:249. Published 2014 Nov 28. doi:10.1186/s12909-014-0249-2
18. Deng F, Gluckstein JA, Larsen DP. Student-directed retrieval practice is a predictor of medical licensing examination performance [published correction appears in Perspect Med Educ. 2016 Nov 18;:]. Perspect Med Educ. 2015;4(6):308-313. doi:10.1007/s40037-015-0220-x
19. Radcliffe C, Lester H. Perceived stress during undergraduate medical training: a qualitative study. Med Educ. 2003;37(1):32-38. doi:10.1046/j.1365-2923.2003.01405.x
20. Sleight DA, Mavis BE. Study Skills and Academic Performance among Second-Year Medical Students in Problem-Based Learning. Med Educ Online. 2006;11(1):4599. doi:10.3402/meo.v11i.4599
21. Hirshbein LD, Gay T. Case-based independent study for medical students in emergency psychiatry. Acad Psychiatry. 2005;29(1):96-99. doi:10.1176/appi.ap.29.1.96
22. Schmidmaier R, Ebersbach R, Schiller M, Hege I, Holzer M, Fischer MR. Using electronic flashcards to promote learning in medical students: retesting versus restudying. Med Educ. 2011;45(11):1101-1110. doi:10.1111/j.1365-2923.2011.04043.x
23. Bland CJ, Starnaman S, Wersal L, Moorhead-Rosenberg L, Zonia S, Henry R. Curricular Change in medical schools: how to succeed. Academic Medicine. 2000;75(6):575-594. doi:10.1097/00001888-200006000-00006
24. Augustin M. How to learn effectively in medical school: test yourself, learn actively, and repeat in intervals. Yale J Biol Med. 2014;87(2):207-212. Published 2014 Jun 6.
25. Hauswirth M., Adamoli A. Identifying Misconceptions with Active Recall in a Blended Learning System. Data Driven Approaches in Digital Education. Lecture Notes in Computer Science. 2017;10474(EC-TEL 2017):416-421. doi.org/10.1007/978-3-319-66610-5_36
26. Vance DE, Farr KF. Spaced Retrieval for enhancing memory: implications for nursing practice and research. J Gerontol Nurs. 2007;33(9):46-52. doi:10.3928/00989134-20070901-08
27. Ausubel DP, Youssef M. The effect of spaced repetition on meaningful retention. J Gen Psychol. 1965;73:147-150. doi:10.1080/00221309.1965.9711263
28. Comer RJ, Ogden N, Boyes M, Gould E. Chapter 7: Learning. In: Psychology around Us. Hoboken, NJ: John Wiley & Sons; 2018:269-275.
29. Berger, C.J., Cummings, L.L., & Hene-man, H.G., III. (1975). Expectancy theory and operant conditioning predictions of performance under variable ratio and continuous sched-ules of reinforcement. Organizational Behavior and Human Performance, 14, 227-243
30. Elmes D. Anki—powerful, intelligent flashcards. 2015. http://ankisrs.net/.
31. Stegers-Jager KM, Cohen-Schotanus J, Themmen AP. Motivation, learning strategies, participation and medical school performance. Med Educ. 2012;46(7):678-688. doi:10.1111/j.1365-2923.2012.04284.x

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.