Abstract

Objectives: To evaluate the quality of the current multiple choice questions (MCQ) in our bank, and to review the appropriateness of the student assessment processes by faculty in this Doctor of Pharmacy program. Also to identify barriers to the reduction of item writing flaws (IWF).

Methods: Faculty, in teams of two, reviewed 7620 MCQs of the question bank. Two more follow up reviews were then performed to further understand the reasons behind the MCQ IWF: (1) 2185 randomly selected MCQ were monitored for timely submission (which is one week before the exam), and (2) the deleted MCQ post exam administration among the randomly selected 2875 MCQ. In both projects, the level of cognition was also investigated.

Results: IWF in 7620 were conservatively assessed at 29.5%. IWF increased as the number of instructors in a given course increased. Late submission of questions in the studied sample of 2185 MCQs was at 22%. That has gotten worse as the years progressed year 1 to year 3 implicating pharmacy practice joint faculty more than others. MCQ deletion from the 2875 random sample was 4.6% across all levels of cognition. Negative correlation was found between the number of teaching faculty and percent of flawless MCQs.

Discussion: To reduce IWF, all processes of assessment must be monitored, and appropriate interventions instituted through education to faculty, review of MCQs prior to processing, engaging faculty in the improvement process. Full time college-based dedicated faculty tends to create an environment of cooperation between all faculty which in turn reduces weaknesses in the assessment processes.

Keywords: Item-writing Flaw; MCQs; Quality Improvement; Pharm.D.; Education

References

1. Pate Adam and David J Caldwell. "Effects of Multiple-Choice Item-Writing Guideline Utilization on Item and Student Performance”. Currents in Pharmacy Teaching and Learning 1 (2014): 130-134.
2. Steven M Downing. "The Effects of Violating Standard Item Writing Principles on Tests and Students: The Consequences of Using Flawed Test Items on Achievement Examinations in Medical Education”. Advances in Health Sciences Education 2 (2005): 133-143.
3. Downing SM. "Construct-Irrelevant Variance and Flawed Test Questions: Do Multiple-Choice Item-Writing Principles Make Any Difference?" Acadamic Medicine 10 (2002): S103-104.
4. Marie Tarrant and James Ware. "Impact of Item-Writing Flaws in Multiple-Choice Questions on Student Achievement in High-Stakes Nursing Assessments”. Medical Education 2 (2008): 198-206.
5. DiSantis D J., et al. "Journal Club: Prevalence of Flawed Multiple-Choice Questions in Continuing Medical Education Activities of Major Radiology Journals”. AJR American Journal of Roentgenology 4 (2015): 698-702.
6. Nedeau-Cayo R., et al. "Assessment of Item-Writing Flaws in Multiple-Choice Questions”. Journal for Nurses in Professional Development 2 (2013): 52-57; quiz E1-2.
7. Pais J., et al. "Do Item-Writing Flaws Reduce Examinations Psychometric Quality?" BMC Research Notes 9 (2016).
8. Tarrant M., et al. "The Frequency of Item Writing Flaws in Multiple-Choice Questions Used in High Stakes Nursing Assessments”. Nurse Education in Practice 6 (2006): 354-363.
9. Masters JC., et al. "Assessment of Multiple-Choice Questions in Selected Test Banks Accompanying Text Books Used in Nursing Education”. Journal of Nursing Education 1 (2001): 25-32.
10. Tarrant M and J Ware. "A Framework for Improving the Quality of Multiple-Choice Assessments”. Nurse Education 3 (2012): 98-104.
11. Vyas R., and A. Supe. "Multiple Choice Questions: A Literature Review on the Optimal Number of Options”. The National Medical Journal of India 3 (2008): 130-133.
12. Gopalakrishnan S and P M Udayshankar. "Question Vetting: The Process to Ensure Quality in Assessment of Medical Students”. Journal of Clinical and Diagnostic Research 9 (2014): XM01-XM03.
13. Clifton Sandra L and Cheryl L Schriner. "Assessing the Quality of Multiple-Choice Test Items”. Nurse Education 1 (2010): 12-16.
14. Abdulghani H M., et al. "Faculty Development Programs Improve the Quality of Multiple Choice Questions Items' Writing”. Scientific Report 5 (2015): 9556.
15. Sanders S. "Assisting Nursing Students through Linguistic Modification of Multiple Choice Questions”. 42nd Biennial Convention Kaplan Nursing (2013).
16. Albekairy Abdulkareem M., et al. "Strategic Initiatives to Maintain Pharmaceutical Care and Clinical Pharmacists Sufficiency in Saudi Arabia”. SAGE Open Medicine 3 (2015): 2050312115594816.
17. Craddock Esther. "Developing the Facilitator Role in the Clinical Area”. Nurse Education Today 3 (1993): 217-224.
18. Jones Francis and Sara Harris. "Benefits and Drawbacks of Using Multiple Instructors to Teach Single Courses”. College Teaching 4 (2012): 132-139.

Citation

Citation: Wesam W Ismail., et al. “Evaluation and Improvement of Multiple-choice Questions in the Doctor of Pharmacy Program". Acta Scientific Pharmaceutical Sciences 4.12 (2020): 03-10.

Copyright

Copyright: © 2020 Wesam W Ismail., et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.