The Efficacy of an Online Cognitive Assessment Tool for Enhancing and Improving Student Academic Outcomes
DOI:
https://doi.org/10.24059/olj.v23i2.1490Keywords:
Online assessment, student engagement with online tools, online learningAbstract
With technology at the fingertips of most undergraduate students, it has been difficult for instructors to fully engage students in the classroom, which has resulted in the creation of several innovative assessment platforms, such as mTuner. mTuner integrates several cognitive learning strategies within an assessment, with the goal of actually enhancing learning, as opposed to just measuring it. In the current study, students’ level of engagement and performance with mTuner were compared to their final multiple-choice paper and pencil exam mark to determine the efficacy of mTuner in achieving improved learning outcomes. Results indicated that students had high performance scores on the mTuner assessment despite their limited engagement in the cognitive learning features, putting to question mTuner’s facilitation of long-term learning. Implications and future recommendations of mTuner implementation in educational environments are discussed.
References
References
Baleni, Z. (2015). Online formative assessment in higher education: its pros and cons. The
Electronic Journal of e-Learning, 13(40, 228-236.
Beck, H.P., Rorrer-Woody, S., & Pierce, L. (1991). The relations of learning and grade
orientations to academic performance. Teaching of Psychology, 18(1), 35-37.
Birenbaum, M., & Feldman, R.A. (1998). Relationships between learning patterns and attitudes
towards two assessment formats. Educational Research, 40(1), 90-97.
Cain, J., Romanelli, F., & Smith, K.M. (2012). Academic entitlement in pharmacy education.
American Journal of Pharmaceutical Education, 76(10), 1-8.
Carpenter, S.K., & DeLosh, E.L. (2006). Impoverished cue support enhances subsequent
retention: Support for the elaborative retrieval explanation of the testing effect. Memory
& Cognition, 34, 268-276.
Cassady, J.C. (2004). The impact of cognitive test anxiety on test comprehension and recall in
the absence of external evaluative pressure. Applied Cognitive Psychology, 18(3), 311
Cassady, J.C., & Johnson, R.E. (2002). Cognitive test anxiety and academic performance.
Contemporary Educational Psychology, 27(2), 270 295.
Chappuis, S., Stiggins, R., Arter, J., & Chappuis, J. 2004. Assessment for learning: An action
guide for school leaders. Portland, OR: Assessment Training Institute.
Coates, H. (2007). A model of online and general campus-based student engagement. Assessment
and Evaluation in Higher Education, 32(2), 121-141.
Cohen, J. Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences, 3rd ed. Mahwah, NJ: Lawrence Erlbaum.
DeCaro, M. S., Thomas, R. D., Albert, N. B., & Beilock, S. L. (2011). Choking under pressure:
Multiple routes to skill failure. Journal of Experimental Psychology: General, 140(3),
Dermo, J. (2009). E-Assessment and the student learning experience. A survey of student
perceptions of e-assessment. British Journal of Educational Technology, 40(2), 203-214.
Eison, J.A., Pollio, H.R., & Milton, O. (1983). Manual for use with LOGO-II. Theory and
Practice in Teacher Education and Other Works. Retrieved from:
http://trace.tennessee.edu/utk_theopubs/2
Eison, J.A., Pollio, H.R., & Milton, O. (1986). Educational and personal characteristics of four
different types of learning- and grade-oriented students. Contemporary Educational
Psychology, 11, 54-67.
Everson, H.T., Smodlake, I., & Tobias, S. (1995). Exploring the relationship of test anxiety and
metacognition on reading test performance: A cognitive analysis. Anxiety, Stress and
Coping, 7, 85-96.
Frymier, A.B., & Weser, B. (2001). The role of student predispositions on student expectations
for instructor communication behaviour. Communication Education, 50, 314-326.
Furlan, L.A., Cassady, J.C., & Pérez, E.R. (2009). Adapting the cognitive test anxiety scale for
use with Argentinean university students. International Journal of Testing, 9(1), 3-19.
Furnham, A., Batey, M., & Martin, N. (2011). How would you like to be evaluated? The
correlates of students’ preferences for assessment methods. Personality and Individual
Differences, 50, 259-263.
Gikandi, J.W., Morrow, D., & Davis, N.E. (2011). Online formative assessment in higher
education: A review of the literature. Computers & Education, 57, 2333-2351.
Greenberger, E., Lessard, J., Chen, C., & Farruggia, S.P. (2008). Self-entitled college students:
Contributions of personality, parenting and motivational factors. Journal of Youth and
Adolescence, 37,1193-1204.
Holbrook, J., Dupont, C., Power, M., & Joordens, S. (2015). Enhancing learning through
testing. Presented at the OND Centre for Teaching Excellence Conference, Waterloo,
Ontario. Retrieved from https://uwaterloo.ca/centre-for-teaching excellence /sites/ca.centre-for-teaching excellence/files/uploads/files/mtuner_ond_2015_final.pdf.
Kalechstein, P., Hocevar, D., Zimmer, J.W., & Kalechstein, M. (1989). Procrastination over test
preparation and test anxiety. In R. Schwarzer, H. M. van der Ploeg, & G. D. Spielberger
(Eds.), Advances in test anxiety research. Vol. 6. (pp. 63–76). Lisse, The Netherlands:
Swets & Zeitlinger.
Karpicke, J.D. (2012). Retrieval-based learning: Active retrieval promotes meaningful learning.
Current Directions in Psychological Science, 21(3), 157-163.
Karpicke, J.D., & Roediger, H.L. (2007). Repeated retrieval during learning is the key to long
term retention. Journal of Memory and Language, 57, 151-162.
Karpicke, J.D., & Roediger, H.L. (2008). The critical importance of retrieval for learning.
Science, 319, 966-968.
Karpicke, J.D., & Zaromb, F.M. (2010). Retrieval mode distinguishes the testing effect from the
generation effect. Journal of Memory and Language, 62, 227-239.
Kember, D., Ho, A., & Hong, C. (2008). The importance of establishing relevance in motivating
student learning. Active Learning in Higher Education, 9, 249-263.
Kopp, J.P., Zinn, T.E., Finney, S.J., & Jurich, D.P. (2011). The development and evaluation of
the academic entitlement questionnaire. Measuring and Evaluation in Counseling and
Development, 44(2), 105-129.
Kornell, N., Hays, M.J., & Bjork, R.A. (2009). Unsuccessful retrieval attempts enhance
subsequent learning. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 35(4), 989-998.
Kuh, G.D., & Hu, S. (2001). The effects of student faculty interaction in the 1990s. Review of
Higher Education, 24(30), 309-332.
Lawrence, K.R., & Frymier, A.B. (2002). The relationship of student-centered teaching with
learning and grade orientation. Paper presented at the annual meeting of the National
Communication Association, New Orleans, LA.
Levine, N. B. (2003). A multidimensional scaling analysis of perceptions of academic dishonesty
among learning- and grade-oriented students. Dissertation Abstracts International,
(4), 1175.
Marsden, H., Carroll, M., & Neill, J.T. (2005). Who cheats at university? A self-report study of
dishonest academic behaviours in a sample of Australian university students. Australian
Journal of Psychology, 57(1), 1-10.
Marsh, E.J., Roediger, H.L, Bjork, R.A., & Bjork, E.L. (2007). Memorial consequences of
multiple-choice testing. Psychonomic Bulletin & Review, 14, 194-199.
McDaniel, M.A., & Masson, M.E.J. (1985). Altering memory representations through retrieval.
Journal of Experimental Psychology: Learning, Memory and Cognition, 11, 371-385.
Merrel, J.D., Cirillo, P.F., Schwartz, P.M., Webb, J.A. (2015). Multiple-choice testing using
immediate feedback-assessment technique forms: Second-chance guessing vs. second
chance learning. Higher Education Studies, 5(5), 50-55.
Miller, T. (2009). Formative computer-based assessment in higher education: The effectiveness
of feedback in supporting student learning. Assessment & Evaluation in Higher
Education, 34(2), 181-192.
Noll, C.L., & Wilkins, M. (2002) Critical skills of IS professionals: A model for curriculum
development. Journal of Information Technology Education, 1(3), 144-154.
Oosterhof, A., Conrad, R.M., & Ely, D.P. (2008). Assessing learners online. New Jersey:
Pearson.
Özden, M.Y., Ertürk, I., & Sanli, R. (2004). Students’ perceptions of online assessment: a case
study. Journal of Distance Education, 19(2), 77-92.
Pare, D.E., Joordens, S. (2009). The invisible researcher: Using educational technologies as
research tools for education. Systemics, Cybernetics and Informatics, 7(5), 37-42.
Plass, J.A., & Hill, K.T. (1986). Children’s achievement strategies and test performance: The
role of time pressure, evaluation anxiety, and sex. Developmental Psychology, 22, 31-36.
Purcell, J.M. (2010). Learning- and grade-orientations of community college students:
Implications for instruction. Community College Journal of Research and Practice, 34,
-511.
Race, P. (2005). Making Learning Happen. London: Sage.
Roedel, T.D., Schraw, G., & Plake, B.S. (1994). Validation of a measure of learning and
performance goal orientations. Educational and Psychological Measurement, 54(4),
-1021.
Roediger, H.L., & Karpicke, J.D. (2006a). The power of testing memory: Basic research and
implications for educational practice. Perspectives on Psychological Science, 1, 181-210.
Saade, R.G., Morin, D., & Thomas, J.D. (2012). Critical thinking in e-learning environments.
Computers in Human Behaviour, 28, 1608-1617.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches:
Multiple choice question examination versus assignment essay. Higher Education, 35,
–472.
Scouller, K., & Prosser, M. (1994). Student’s experiences in studying for multiple choice
question examinations. Studies in Higher Education, 19, 267–279.
Smith, G. (2007). How does student performance on formative assessment relate to
learning assessed by exams? Journal of College Science Teaching, 36(7), 28-34.
Smith, S.D., & Caruso, J.B. (2010). The ECAR study of undergraduate students and information
technology. ECAR Research Study 6, Center for Applied Research, Boulder: CO.
Stöeber, J. (2004). Dimensions of test anxiety: Relations to ways of coping with pre-exam
anxiety and uncertainty. Anxiety, Stress & Coping, 17(3), 213–226.
Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation and
assessment in higher education: A review. Assessment and Evaluation in Higher
Education, 30, 325–341.
Tendhar, C., Culver, S.M., & Burge, P.L. (2013). Validating the national survey of student
engagement (NSSE) at a research-intensive university. Journal of Education and
Training Studies, 1(1), 182-193.
Thomas, C.L., Cassady, J.C., & Heller, M.L. (2017). The influence of emotional intelligence,
cognitive test anxiety, and coping strategies on undergraduate academic performance.
Learning and Individual Differences, 55, 40-48.
Tippin, G.K., Lafreniere, K., & Page, S. (2012). Student perception of academic grading:
personality, academic orientation and effort. Active Learning in Higher Education, 13(1),
-61.
Traub, R.E. & MacRury, K. (1990). Multiple-choice vs. free response in the testing of scholastic achievement, in: K. Ingenkamp & R.S. Jager (Eds) Test und tens 8: jahrbuch der pädagogischen diagnostik (Weinheim und Base, Beltz Verlag), 128–159.
Tulving, E. (1974). Cue-dependent forgetting. American Scientist, 62(1), 74-82.
Walker, D.J., Topping, K., & Rodrigues, S. (2008). Student reflections on formative e
assessment: Expectations and perceptions. Learning, Media and Technology, 33, 221
Williams, K.D., & Frymier, A.B. (2007). The relationship between student educational
orientation and motives for out-of-class communication. Communication Research
Reports, 24, 249-256.
Zeidner, M., & Matthews, G. (2005). Evaluation anxiety. In A. J. Elliot, & C. S. Dweck (Eds.),
Handbook of competence and motivation (pp. 141–163). London: Guildford Press.
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
