The Impact of Attitudes, Beliefs, and Cognitive Reflection on the Development of Critical Thinking Skills in Online Students

Boban Simonovic, Katia Vione, Dean Fido, Edward Stupple, James Martin, Richard Clarke


Learning and development of critical thinking (CT) skills in higher education is essential for academic achievement. The following experiment is the first to examine the effect of online student’s perceptions and attitudes towards CT across dimensions of confidence, valuing, misconceptions, cognitive reflection, and authors writing. Furthermore, a CT intervention was developed, and the effects of the intervention examined with an aim to help students improve their grade point average. The analyses demonstrated that student’s confidence and cognitive reflection predict academic achievement. Moreover, the online CT intervention was associated with improved students’ CT attitudes, skills, and academic performance. Significant interactions were observed between time (pre- and post-intervention) and intervention in cognitive reflection, confidence, beliefs, and attitudes related to CT, and student grade point average (GPA, as a measure of student’s performance on online modules). It was concluded that the CT can be taught and that an intervention based on “how to think” rather than a “what to think” mixed approach can help online students develop CT, strengthen their confidence in CT and help students improve their academic performance in an online setting.


Online learning

Full Text:



Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102-1134.

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, A., & Persson, T. (2015). “Strategies for Teaching Students to Think Critically: A Meta-Analysis.” Review of Educational Research, 85(2), 275–314.

Abasi, A.R., Akbari, N., & Graves, B. (2006). Discourse appropriation, construction of identities, and the complex issue of plagiarism: ESL students writing in graduate school. Journal of Second Language Writing, 15(2) 102–17.

Ackerman, R., & Thompson, V. A. (2017). Meta-reasoning: Monitoring and control of thinking and reasoning. Trends in Cognitive Sciences, 21(8), 607-617.

Alvarez-Ortiz, C. (2007). Does philosophy improve critical thinking skills? (Master’s thesis). Australia: University of Melbourne.

Bernard, R. M., Zhang, D., Abrami, P. C., Sicoly, F., Borokhovski, E., & Surkes, M. (2008). Exploring the structure of the Watson-Glaser Critical Thinking Appraisal: One scale or many subscales? Thinking Skills and Creativity, 3, 15–22.

Beyer, B. (2008). How to teach thinking skills in social studies and history. Social Studies, 99(5), 196-201.

Bonnefon, J. F. (2016). The pros and cons of identifying critical thinking with system 2 processing. Topoi, 1–7. DOI:10.1007/S11245-016-9375-2

Butler, H. A., Pentoney, C., & Bong, M. P. (2017). Predicting real-world outcomes: Critical thinking ability is a better predictor of life decisions than intelligence. Thinking Skills and Creativity, 25, 38-46.

Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306–307.

Campitelli, G., & Gerrans, P. (2014). Does the cognitive reflection test measure cognitive reflection? A mathematical modeling approach. Memory & Cognition, 42(3), 434-447. DOI: 10.3758/s13421-013-0367-9

Celuch, K., Black, G., & Warthan, B. (2009). Student self-identity as a critical thinker. Journal of Marketing Education, 31, 31–39.

Cheung, K. Y. F., Stupple, E. J. N., & Elander, J. (2015). Development and validation of the Student Attitudes and Beliefs about Authorship Scale: A psychometrically robust measure of authorial identity. Studies in Higher Education, 1–18.

Cloete, M. B. (2019). The impact of an integrated assessment on the critical thinking skills of first-year university students. Accounting Education, 27(5).

Dunn, D. S., Halonen, J. S., & Smith, R. A. (Eds.). (2008). Teaching critical thinking in psychology: A handbook of best practices. Malden, MA: Wiley-Blackwell, (Chapter 4).

Duro, E., Elander, J., Maratos, F. A., Stupple, E. J., & Aubeeluck, A. (2013). In search of critical thinking in psychology: an exploration of student and lecturer understandings in higher education. Psychology Learning & Teaching, 12(3), 275-281.

Dwyer, C. P., Hogan, M. J., & Stewart, I. (2012). An evaluation of argument mapping as a method of enhancing critical thinking performance in e-learning environments. Metacognition and Learning, 7, 219–244.

Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52.

Elander, J., Pittam, G., Lusher, J., Fox, P., & Payne, N. (2010). Evaluation of an intervention to help students avoid unintentional plagiarism by improving their authorial identity. Assessment & Evaluation in Higher Education, 35 (2), 157–171.

Ennis, R. H. (1985). A logical basis for measuring critical thinking skills. Educational Leadership, 43 (2), 44. Retrieved from

Ennis, R. H. (2016). “Critical Thinking Across the Curriculum: A Vision.” Topoi 37 (1): 1–20. DOI: 10.1007/s11245-016-9401-4

Entwistle, N., Tait, H., & McCune., V. (2000). Patterns of response to an approaches to studying inventory across contrasting groups and contexts. European Journal of Psychology of Education, 15, 33–48. DOI: 10.1007/BF03173165

Evans, J. St. B. T., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives in Psychological Science, 8, 223–241.

Facione, P. A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Millbrae, CA: California Academic Press (ERIC document ED 315-423). Retrieved from

Facione, P. A., Facione, N. C., & Giancarlo, C. (1992). The California critical thinking skills test. California Academic Press.

Facione, P. A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking. Informal Logic, 20(1), 61–84.

Facione, P.A., Facione, N.C., & Giancarlo, C.A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 20(1), 61-84.Retrievied from

Facione, P. A., & Facione, N. C. (2001). Analyzing explanations for seemingly irrational choices: Linking argument analysis and cognitive science. International Journal of Applied Philosophy, 15(2), 267–286. Retrieved from

Facione, P. A. (2013). Critical thinking what it is and why it counts. Retrieved from

Fong, C. J., Kim, Y., Davis, C. W., Hoang, T., & Kim, Y. W. (2017). A meta-analysis on critical thinking and community college student achievement. Thinking Skills and Creativity, 26, 71-83.

Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25-42. DOI: 10.1257/089533005775196732

Gambrill, E. (2006). Evidence-based practice and policy: Choices ahead. Research on Social Work Practice, 16(3), 338–357.

Guiller, J., Durndell, A., & Ross, A. (2008). Peer interaction and critical thinking: face-to-face or online discussion? Learning and Instruction, 18, 187-200.

Halpern, D. F. (1993). Assessing effectiveness of critical thinking instruction. The Journal of General Education, 42(4), 238-254.

Halpern, D. F. (2003). Thought and knowledge: An introduction to critical thinking (4th ed.). New Jersey: Laurence Erlbaum Associates.<179::AID-ACP453>3.0.CO;2-S

Halpern, D (2014) Critical Thinking across the Curriculum: A Brief Edition of Thought and Knowledge. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Heijltjes, A., Van Gog, T., Leppink, J., & Paas, F. (2014). Improving critical thinking: Effects of dispositions and instructions on economics students’ reasoning skills. Learning and Instruction, 29,31-42.

Heijltjes, A., Van Gog, T., Leppink, J., & Paas, F. (2015). Unraveling the effects of critical thinking instructions, practice, and self-explanation on students’ reasoning performance. Instructional Science, 43, 487-506.

Huber, C. R., & Kuncel, N. R. (2016). Does college teach critical thinking? A meta-analysis. Review of Educational Research, 86(2), 431-468.

Janssen, E. M., Meulendijks, W., Mainhard, T., Verkoeijen, P.P.J.L., Heijltjes, A.E.G., van Peppen, V., van Gog, T. (2019). Identifying characteristics associated with higher education teachers’ Cognitive Reflection Test performance and their attitudes towards teaching critical thinking. Teaching and Teacher Education, 84, 139-149.

Hattie, J. (2008). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Routledge Press.

Hussin, W. N. T. W., Harun, J., & Shukor, N. A. (2019). Online Interaction in Social Learning Environment towards Critical Thinking Skill: A Framework. Journal of Technology and Science Education, 9(1), 4-12.

Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 39, 341–350.

Kahneman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux, (Chapter 1).

Kember, D., Leung, D., Jones, A., & Loke, A. Y. (2000). Development of a questionnaire to measure the level of reflective thinking. Assessment and Evaluation in Higher Education, 25(4), 380–395.

Klaczynski, P. A. (2014). Heuristics and biases: interactions among numeracy, ability, and reflectiveness predict normative responding. Frontiers in Psychology, 5, 665.

Kraft, N. P. (2002). Teacher research as a way to engage in critical reflection: A case study. Reflective Practice, 3, 175–189.

Kuiper, R. (2002). Enhancing metacognition through the reflective use of self-regulated learning strategies. The Journal of Continuing Education in Nursing, 33(2), 78–87. PMID: 11916343

Lamont, P. (2020). The construction of “critical thinking”: Between how we think and what we believe. History of Psychology. Advance online publication.

Liu, O. L., Frankel, L., & Roohr, K. C. (2014). “Assessing Critical Thinking in Higher Education: Current State and Directions for Next-generation Assessment.” ETS Research Report Series, i–23.

MacKnight, C. B. (2000). Teaching critical thinking through online discussions. Educause Quarterly, 23(4), 38–41. Retrieved from[1].pdf

Maurino, P. S. M. (2007). Looking for critical thinking in online threaded discussions. Journal of Educational Technology Systems, 35(3), 241-260.

McGuinness, C. (2013). Teaching thinking: Learning how to think. In Presented at the Psychological Society of Ireland and British Psychological Association’s Public Lecture Series Galway, Ireland, 6th March.

Morsanyi, K., Busdraghi, C., & Primi, C. (2014). Mathematical anxiety is linked to reduced cognitive reflection: a potential road from discomfort in the mathematics classroom to susceptibility to biases. Behavioral and Brain Functions, 10(1), 31. DOI: 10.1186/1744-9081-10-31

Osborne, R., Dunne, E., & Farrand, P. (2013). Integrating technologies into ‘authentic’ assessment design: An affordance approach. Research in Learning Technology, 21,1–18.

Parker S, Mayner L, Michael Gillham D. (2015) E-learning for critical thinking: Using nominal focus group method to inform software content and design. Nursing and Midwifery Studies, 4(4), e30471. 10.17795/nmsjournal30471

Pask, G. (1988). Learning strategies, teaching strategies, and conceptual or learning style. R.R. Schmeck (Ed.), Learning strategies and learning styles, Plenum Press, New York (1988), pp. 83-100.Retrieved from

Phan, H. P. (2006). Examination of student learning approaches, reflective thinking, and epistemological beliefs: A latent variables approach. Journal of Research in Educational Psychology, 4(3), 577–610.

Puig, B., Blanco-Anaya, P., Bargiela, I. M., & Crujeiras-Pérez, B. (2019). A systematic review on critical thinking intervention studies in higher education across professional fields. Studies in Higher Education, 44 (5), 860-869.

Richardson, J., & Ice, P. (2010). Investigating students' level of critical thinking across instructional strategies in online discussions. Internet and Higher Education, 13, 52-59.

Ross, D., Loeffler, K., Schipper, S., Vandermeer, B., Allan, G. M. (2013). Do scores on three commonly used measures of critical thinking correlate with academic success of health professions trainees? A systematic review and meta-analysis. Academic Medicine, 88(5), 724-734. doi: 10.1097/ACM.0b013e31828b0823

Author et al., (2018).

Solon, T. (2007). Generic critical thinking infusion and course content learning in introductory psychology. Journal of Instructional Psychology, 34(2), 95–109.

Stanovich, K. E., & West, R. F. (1997). Reasoning independently of prior belief and individual differences in actively open minded thinking. Journal of Educational Psychology, 89, 342–357.

Stanovich, K. E., & West, R. F. (2007). Natural myside bias is independent of cognitive ability. Thinking & Reasoning, 13,


Stanovich, K. E. (2011). Rationality and the reflective mind. New York, NY: Oxford University Press, (Chapter 3).

Stanovich, K. E., West, R. F., & Toplak, M. E. (2016). The Rationality Quotient (RQ): Toward a test of rational thinking. Cambridge, MA: MIT Press, (Chapter 2).

Stanovich, K. E. (2018). Miserliness in human cognition: the interaction of detection, override and mindware. Thinking and Reasoning. 423-444.

Stupple. E. J. N, Maratos, F. A., Elander, J., Hunt, T. E., Cheung, K. Y.F., Aubeeluck, A. V. (2017). Development of the Critical Thinking Toolkit (CriTT): A measure of student attitudes and beliefs about critical thinking. Thinking Skills and Creativity, 23, 91–100.

Thompson, C. (2011). Critical Thinking across the Curriculum: Process over Output. International Journal of Humanities and Social Science, 1 (9), 1-7. Retrieved from

Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision making, 11(1), 99.

Tiruneh, D. T., Verburgh, A. & Elen, J. (2014). Effectiveness of critical thinking instruction in higher education: a systematic review of intervention studies. Higher Education Studies, 4(1), 1–17. DOI:10.5539/hes.v4n1p1

Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: an expansion of the Cognitive Reflection Test. Thinking and Reasoning,20, 147–168.

Tsui, L. (2002). Fostering critical thinking through effective pedagogy. Research in Higher Education, 40(2), 740-763.

Tversky, A. & Kahneman, D. (1983) Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4), 293– 315.

Van Peppen, L. M., Verkoeijen, P. P. J. L., Heijltjes, A. E. G., Janssen, E. M., Koopmans, D., & van Gog, T. (2018). Effects of self-explaining on learning and transfer of critical thinking skills. Frontiers in Education, 3.

Watson, G., Glaser, E. M. (1964). Watson-Glaser Critical Thinking Appraisal Manual: Forms YM and ZM. New York: Harcourt, Brace, and World.

West, R. F., Toplak, M. E., & Stanovich, K. E. (2008). Heuristics and biases as measures of critical thinking: Associations with cognitive ability and thinking dispositions. Journal of Educational Psychology, 100(4), 930– 941.


Copyright (c) 2022 Boban Simonovic, Katia Vione, Dean Fido, Edward Stupple, James Martin, Richard Clarke

License URL: