Evaluating a Structured Online Peer Evaluation System Among Graduate-Level Communication Capstone Students Through Action Research
DOI:
https://doi.org/10.24059/olj.v26i1.3077Keywords:
Constructivism, cognitive apprenticeship, cognitive and mind tools, cognitive presence, community of inquiry, educational technology, peer review, student retention, VygotskyAbstract
Although enrollment in online courses continues to accelerate, challenges exist in online learning. A failure to experience collaboration and interaction can impact student retention and success. While peer review activity promotes student interaction, a collaborative community of learners, and critical thinking skills, higher education environments have failed to equip students with the knowledge and tools to ensure adept participation. As students offered limited participation and low-quality engagement in routine online peer review activities, the purpose of this action research was to implement and evaluate the impact of a structured online peer evaluation system for Graduate Communication Capstone students at the University of North Coast Muscari (UNCM). This study incorporated a structured peer evaluation system, including an interactive educational technology peer review tool kit innovation. The theoretical framework of the innovation was aligned to learning theory and grounded in Vygotsky’s zone of proximal development, cognitive and mind tools, and Constructivist theory of cognitive apprenticeship. Data collection offered seven methods and data analysis included quantitative and qualitative approaches as part of a triangulation mixed methods design. Community of Inquiry (CoI) deductive analysis was performed to denote social and cognitive presences, while further validating the themes that had emerged through qualitative data analysis. As an impact of this research study, students used the structured peer evaluation system to transform anxiety into social and cognitive freedom, producing a focused, responsible approach to peer learning.
References
Allen, I. E., & Seaman, J. (2013). Changing course: Ten years of tracking online education in the United States. Sloan Consortium. Retrieved from https://www.onlinelearningsurvey.com/reports/changingcourse.pdf
Alman, S. W., Frey, B. A., & Tomer, C. (2012). Social and cognitive presence as factors in learning and student retention: An investigation of the cohort model in an iSchool setting. Journal of Education for Library and Information Science, 53(4), 290-302. Retrieved from https://www.jstor.org/stable/43686922?seq=1#page_scan_tab_contents
Alnasser, S. M. (2018). Exploring student-writers’ views on replacing teacher feedback with peer feedback and computer-based feedback. Arab World English Journal, 9(3), 345-366. https://doi.org/10.24093/awej/vol9no3.23
Baker, D. (2008). Peer assessment in small groups: A comparison of methods. Journal of Management Education, 32(2), 183-209. https://doi.org/10.1177/1052562907310489
Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179-192. https://doi.org/10.1177/1469787416654794
Barnard, R., de Luca, R., & Li, J. (2015). First-year undergraduate students’ perceptions of lecturer and peer feedback: A New Zealand action research project. Studies in Higher Education, 40(5), 933-944. http://dx.doi.org/10.1080/03075079.2014.881343
Barría, J., Scheihing, E., & Parra, D. (2014). Visualizing student participation in a collaborative learning environment. In F. Cena, A. S. da Silva, & C. Trattner (Eds.), Hypertext HT (Doctoral Consortium/Late-breaking Results/Workshops), CEUR Workshop Proceedings (Vol. 1210). Santiago, Chile: CEUR-WS.org.
Biggs, J. B., & Tang, C. S. (2011). Teaching for quality learning at university: What the student does (4th ed.) New York, NY: McGraw-Hill Education. Retrieved from https://books.google.com/books?id=XhjRBrDAESkC&printsec=frontcover&dq=teaching+for+quality+learning+at+university&hl=en&sa=X&ved=0ahUKEwj3kqKVo9bhAhWPmeAKHVV6ClQQ6AEIKjAA#v=onepage&q=teaching%20for%20quality%20learning%20at%20university&f=false
Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151-167. https://doi.org/10.1080/713695728
Boud, D. (2013). Introduction: Making the move to peer learning. In D. Boud, R. Cohen, & J. Sampson (Eds.), Peer learning in higher education: Learning from and with each other. New York, NY: Routledge.
Brill, J. (2016). Investigating peer review as a systemic pedagogy for developing the design knowledge, skills, and dispositions of novice instructional design students. Educational Technology Research & Development, 64(4), 681-705. https://doi.org/10.1007/s11423-015-9421-6
Brown, J. A., & Stefaniak, J. E. (2016). The design of a cognitive apprenticeship to facilitate storytime programming for librarians. Contemporary Educational Technology, 7(4), 331-351. Retrieved from https://files.eric.ed.gov/fulltext/EJ1117606.pdf
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42. https://doi.org/10.3102/0013189X018001032
Brutus, S., Donia, M. B., & Ronen, S. (2013). Can Business students learn to evaluate better? Evidence from repeated exposure to a peer-evaluation system. Academy of Management Learning & Education, 12(1), 18-31. https://doi.org/10.5465/amle.2010.0204
Caddy, I. (2014). Using an online peer review and assessment tool in a large undergraduate business subject: Analysing student behaviour and performance. Employment Relations Record, 14(1), 55-80. Retrieved from https://search.informit.com.au/documentSummary;dn=552924605721692;res=IELBUS
Cheng, G., & Chau, J. (2016). Exploring the relationships between learning styles, online participation, learning achievement and course satisfaction: An empirical study of a blended learning course. British Journal of Educational Technology, 47(2), 257-278. https://doi.org/10.1111/bjet.12243
Ching, Y.-H., & Hsu, Y.-C. (2013). Peer feedback to facilitate project-based learning in an online environment. The International Review of Research in Open and Distributed Learning, 14(5), 258-276. doi:10.19173/irrodl.v14i5.1524
Ching, Y.-H., & Hsu, Y.-C. (2016). Learners’ interpersonal beliefs and generated feedback in an online role-playing peer-feedback activity: An exploratory study. International Review of Research in Open and Distributed Learning, 17(2), 105-122. Retrieved from https://eric.ed.gov/?id=EJ1093774
Chittum, J. R., & Bryant, L. H. (2014). Reviewing to learn: Graduate student participation in the professional peer-review process to improve academic writing skills. International Journal of Teaching and Learning in Higher Education, 26(3), 473-484. Retrieved from https://eric.ed.gov/?id=EJ1060832
Choi, J. (2014). Online peer discourse in a writing classroom. International Journal of Teaching and Learning in Higher Education, 26(2), 217-231. Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE1730.pdf
Clark, K. R. (2018). Learning theories: Constructivism. Radiologic Technology, 90(2), 180-182. Retrieved from http://www.radiologictechnology.org/content/90/2/180.extract#
Clark, K. R., & Vealé, B. L. (2018). Strategies to enhance data collection and analysis in qualitative research. Radiologic Technology, 89(5), 482-485. Retrieved from http://www.radiologictechnology.org/content/89/5/482CT.extract
Collins, A. (1988). Cognitive apprenticeship and instructional technology (Technical Report No. 6899). BBN Laboratories, Cambridge, MA. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a203609.pdf
Collins, A., Brown, J. S., & Newman, S. E. (1987). Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics (Technical Report No. 403). BBN Laboratories, Cambridge, MA. Centre for the Study of Reading, University of Illinois. January, 1987. Retrieved from https://www.ideals.illinois.edu/bitstream/handle/2142/17958/ctrstreadtechrepv01987i00403_opt.pdf?sequence=1
Conrad, R., & Donaldson, J. (2004). Engaging the online learner: Activities and resources for creative instruction. San Francisco, CA: John Wiley & Sons, Inc.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
Dar, M. F., Zaki, S., & Kazmi, H. H. (2014). Peer assessment in EAP writing: An effective strategy for large classes. Journal of Educational Research, 17(1), 50-59. Retrieved from https://search.proquest.com/openview/b45d19a21ac8ab9137cb4281ed3a04c8/1?pq-origsite=gscholar&cbl=1616349
Davies, J., & Graff, M. O. (2005). Performance in e-learning: Online participation and student grades. British Journal of Educational Technology, 36(4), 657-663. Retrieved from http://www.elsinnet.org.uk/research/mgg_files/BJET2005.pdf
De Grez, L., Valcke, M., & Roozen, I. (2012). How effective are self- and peer assessment of oral presentation skills compared with teachers’ assessments? Active Learning in Higher Education, 13(2), 129-142. https://doi.org/10.1177/1469787412441284
Delve. (n.d.) Retrieved from https://delvetool.com/getstarted
Demirbilek, M. (2015). Social media and peer feedback: What do students really think about using wiki and Facebook as platforms for peer feedback? Active Learning in Higher Education, 16(3), 211-224. doi:10.1177/1469787415589530
Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. New York: Macmillan. Retrieved from https://archive.org/stream/democracyandedu00dewegoog#page/n6/mode/2up
Dewey, J. (1938). Experience & education. New York, NY: Kappa Delta Pi. Retrieved from https://archive.org/details/ExperienceAndEducation
Dijks, M. A., Brummer, L., & Kostons, D. (2018). The anonymous reviewer: The relationship between perceived expertise and the perceptions of peer feedback in higher education. Assessment & Evaluation in Higher Education, 43(8), 1258-1271. https://doi.org/10.1080/02602938.2018.1447645
Elshami, W., & Abdalla, M. E. (2017). Diagnostic radiography students’ perceptions of formative peer assessment within a radiographic technique module. Radiography, 23, 8-13. http://dx.doi.org/10.1016/j.radi.2016.06.001
Ertmer, P. A., & Newby, T. J. (2013). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 26(2), 43-71. doi:10.1002/piq.21143
Esterberg, K. G. (2002). Qualitative methods in social research. New York, NY:
McGraw-Hill.
Evans, P. (2015). Open online spaces of professional learning: Context, personalisation and facilitation. TechTrends: Linking Research & Practice to Improve Learning, 59, 31-36. https://doi.org/10.1007/s11528-014-0817-7
Flick, U. (2009). An introduction to qualitative research (4th ed.). Retrieved from http://books.google.com
Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: review, issues, and future directions. Internet and Higher Education, 10(3), 157-172. https://doi.org/10.1016/j.iheduc.2007.04.001
Ghadirian, H., Ayub, A. F., Bakar, K. B., & Hassanzadeh, M. (2016). Growth patterns and e-moderating supports in asynchronous online discussions in an undergraduate blended course. International Review of Research in Open & Distance Learning, 17(23), 189-208. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2397
Gielen, M., & De Wever, B. (2015). Scripting the role of assessor and assessee in peer assessment in a wiki environment: Impact on peer feedback quality and product improvement. Computers & Education, 88, 370-386. http://dx.doi.org/10.1016/j.compedu.2015.07.012
Gikandi, J. W., & Morrow, D. (2016). Designing and implementing peer formative feedback within online learning environments. Technology, Pedagogy & Education, 25(2), 153-170. https://doi.org/10.1080/1475939X.2015.1058853
Hamer, J., Purchase, H., Luxton-Reilly, A., & Denny, P. (2015). A comparison of peer and tutor feedback. Assessment & Evaluation in Higher Education, 40(1), 151-164. https://doi.org/10.1080/02602938.2014.893418
Hampel, R., & Pleines, C. (2013). Fostering student interaction and engagement in a virtual learning environment: An investigation into activity design and implementation. CALICO Journal, 30(3), 342-370. https://doi.org/10.11139/cj.30.3.342-370
Heyman, E. (2010). Overcoming student retention issues in higher education online programs: A Delphi study (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (748309429)
Hogg, L. M. (2018). Empowering students through peer assessment: Interrogating complexities and challenges. Reflective Practice, 19(3), 308-321. https://doi.org/10.1080/14623943.2018.1437404
Huang, E. Y., Lin, S. W., & Huang, T. K. (2012). What type of learning style leads to online participation in the mixed-mode e-learning environment? A study of software usage instruction. Computers & Education, 58(1), 338-349. https://doi.org/10.1016/j.compedu.2011.08.003
Jaramillo, J. A. (1996). Vygotsky’s sociocultural theory and contributions to the development of constructivist curricula. Education, 117(1), 133. Retrieved from https://met512.weebly.com/uploads/4/2/2/5/42253875/anas_article_re-_vygotsky___constructivism.pdf
Jin, S.-H. (2017). Using visualization to motivate student participation in collaborative online learning environments. Journal of Educational Technology & Society, 20(2), 51-62. Retrieved from http://www.jstor.org/stable/90002163
Johnson, A. P. (2008). A short guide to action research (3rd ed.). Boston: Allyn & Bacon.
Jonassen, D. H. (1991). Evaluating constructivistic learning. Educational Technology, 31(9), 28-33. Retrieved from https://eric.ed.gov/?id=EJ433315
Kaufman, J. H., & Schunn, C. D. (2011). Students’ perceptions about peer assessment for writing: Their origin and impact on revision work. Instructional Science, 39(3), 387-406. Retrieved from http://www.lrdc.pitt.edu/schunn/research/papers/KaufmanSchunn-StudentPerceptions-.pdf
Kearney, S. (2013). Improving engagement: The use of “Authentic self-and peer-assessment for learning” to enhance the student learning experience. Assessment & Evaluation in Higher Education, 38(7), 875-891. https://doi.org/10.1080/02602938.2012.751963
Kelly, L. (2015). Effectiveness of guided peer review of student essays in a large undergraduate biology course. International Journal of Teaching & Learning in Higher Education, 27(1), 56-68. Retrieved from https://eric.ed.gov/?id=EJ1069821
Kirschner, P. A., & Erkens, G. (2006). Cognitive tools and mindtools for collaborative learning. Journal of Educational Computing Research, 35(2), 199-209. https://doi.org/10.2190/R783-230M-0052-G843
Lee, Y. & Choi, J. (2011). A review of online course dropout research: Implications for practice and future research. Educational Technology Research and Development, 59, 593-618. https://doi.org/10.1007/s11423-010-9177-y
Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536. https://doi.org/10.1111/j.1467-8535.2009.00968.x
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newberry Park, CA: Sage Publications, Inc.
Llado, A. P., Soley, L. F., Fraguell Sansbello, R. M., Pujolras, G. A., Planella, J. P., Roura-Pascual, N., … Moreno, L. M. (2014). Student perceptions of peer assessment: An interdisciplinary study. Assessment & Evaluation in Higher Education, 39(5), 592-610. https://doi.org/10.1080/02602938.2013.860077
Lynch, R., McNamara, P. M., & Seery, N. (2012). Promoting deep learning in a teacher education programme through self-and peer-assessment and feedback. European Journal of Teacher Education, 35(2), 179-197. https://doi.org/10.1080/02619768.2011.643396
Magda, A. J., Capranos, D., & Aslanian, C. B., (2020). Online college students 2020: Comprehensive data on demands and preferences. Louisville, KY: Wiley Education Services.
Man, D., Xu, Y., & O’Toole, J. M. (2018). Understanding autonomous peer feedback practices among postgraduate students: A case study in a Chinese university. Assessment & Evaluation in Higher Education, 43(4), 527-536. https://doi.org/10.1080/02602938.2017.1376310
McMahon, T. (2010). Peer feedback in an undergraduate programme: Using action research to overcome students’ reluctance to criticise. Educational Action Research, 18(2), 273-287. https://doi.org/10.1080/09650791003741814
Mertler, C. A. (2017). Action research: Improving schools and empowering educators (5th ed.). Los Angeles, CA: Sage Publications, Inc.
Michinov, N., Brunot, S., Le Bohec, O., Juhel, J., & Delaval, M. (2011). Procrastination, participation, and performance in online learning environments. Computers & Education, 56(1), 243-252. https://doi.org/10.1016/j.compedu.2010.07.025
Moneypenny, D. B., Evans, M., & Kraha, A. (2018). Student perceptions of and attitudes toward peer review. American Journal of Distance Education, 32(4), 236-247. https://doi.org/10.1080/08923647.2018.1509425
Mulder, R., Baik, C., Naylor, R., & Pearce, J. (2014). How does student peer review influence perceptions, engagement and academic outcomes? A case study. Assessment & Evaluation in Higher Education, 39(6), 657-677. doi:10.1080/02602938.2013.860421
Nagori, R., & Cooper, M. (2014). Key principles of peer assessments: A feedback strategy to engage the postgraduate international learner. IAFOR Journal of Education, 2(2), 211-237. Retrieved from https://eric.ed.gov/?id=EJ1080371
Negash, S. (2008). Handbook of distance learning for real-time and asynchronous information technology education. Retrieved from http://books.google.com
Ng, E. M. (2018). Are students receptive to formative assessment when authoring wiki projects? Ubiquitous Learning: An International Journal, 11(3), 1-15. http://doi.org/10.18848/1835-9795/CGP/v11i03/1-15
Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: A peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102-122. https://doi.org/10.1080/02602938.2013.795518
O’Connor, E. A., & McQuigge, A. (2013). Exploring badging for peer review, extended learning and evaluation, and reflective/critical feedback within an online graduate course. Journal of Educational Technology Systems, 42(2), 87-105. https://doi.org/10.2190/ET.42.2.b
Patton, M. Q. (2014). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.
Powell, K. C., & Kalina, C. J. (2009). Cognitive and social constructivism: Developing tools for an effective classroom. Education, 130(2), 241-250. Retrieved from https://www.researchgate.net/publication/234717752_Cognitive_and_Social_Constructivism_Developing_Tools_for_an_Effective_Classroom
Purarjomandlangrudi, A., Chen, D., & Nguyen, A. (2016). Investigating the drivers of student interaction and engagement in online courses: A study of state-of-the-art. Informatics in Education, 15(2), 269-286. https://doi.org/10.15388/infedu.2016.14
Ratminingsih, N. M., Artini, L. P., & Padmadewi, N. N. (2017). Incorporating self and peer assessment in reflective teaching practices. International Journal of Instruction, 10(4), 165-184. https://doi.org/10.12973/iji.2017.10410a
Reinholz, D. L. (2018). Three approaches to focusing peer feedback. International Journal for the Scholarship of Teaching & Learning, 12(2), 1-8. https://doi.org/10.20429/ijsotl.2018.120210
Roever, C., & Phakiti, A. (2018). Quantitative methods for second language research: A problem-solving approach. Retrieved from http://books.google.com
Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.
Saldaña, J., & Omasta, M. (2017). Qualitative research: Analyzing life. Thousand Oaks, CA: Sage Publications, Inc.
Schmuck, R.A. (1997). Practical action research for change. Arlington Heights, IL: Skylight Professional Development.
Schunk, D. H. (2008). Learning theories: An educational perspective (5th ed.). Upper Saddle River, NJ: Pearson.
Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591-611.
Sridharan, B., Muttakin, M. B., & Mihret, D. G. (2018). Students’ perceptions of peer assessment effectiveness: An explorative study. Accounting Education, 27(3), 259-285. https://doi.org/10.1080/09639284.2018.1476894
Streiner, D. L., & Norman, G. R. (2011). Correction for multiple testing: Is there a resolution? Chest, 140(1), 16-18.
Tavalok, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53-55. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4205511/
Tricio, J., Woolford, M., & Escudier, M. (2018). Analysis of dental students’ written peer feedback from a prospective peer assessment protocol. European Journal of Dental Education, 20, 241-247. https://doi.org/10.1111/eje.12187
Van der Merwe, M. D. (2012). Applying the community of inquiry framework: A novel tool for systematic and economic coding and analysis of forum discourse in situ and in context. International Journal of Learning Technology, 7(3), 246-260. Retrieved from https://pdfs.semanticscholar.org/9b82/f6890ee5a1a05f643dd7d2f597182e2f31cb.pdf?_ga=2.202346834.2132451076.1564110631-1017552094.1564110631
Vygotsky, L. S. (1962). Thought and language. Cambridge, MA: MIT Press. (Original work published in 1934).
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
Wang, W. (2016). Peer feedback in Chinese College English Writing class: Using action research to promote students’ English writing. Journal of Language Teaching and Research, 7(5), 958-966. http://dx.doi.org/10.17507/jltr.0705.17
Webb, E. J., Campbell, D. T., Schwartz, R., & Sechrest, L. (1966). Unobtrusive measures: Nonreactive research in the social sciences. Chicago, IL: Rand McNally.
Wen, L. M., & Tsai, C. C. (2006). University students’ perceptions of and attitudes toward (online) peer assessment. Higher Education, 51(1), 27-44. doi:10.1007/s10734-004-6375-8
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press.
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80-83.
Willging, P. A., & Johnson, S. D. (2009). Factors that influence students’ decision to dropout of online courses. Journal of Asynchronous Learning Networks, 13(3), 115-127.
Xu, D., & Xu, Y. (2019). The promises and limits of online higher education: Understanding how distance education affects access, cost, and quality. Washington, DC: American Enterprise Institute. Retrieved from https://www.aei.org/wp-content/uploads/2019/03/The-Promises-and-Limits-of-Online-Higher-Education.pdf
Yee, K. C., Wong, M. C., & Turner, P. (2017). Qualitative research for patient safety using ICTs: Methodological considerations in the Technological Age. Studies in Health Technology and Informatics, 241, 36-42. Retrieved from https://
pdfs.semanticscholar.org/5b54/1b7a07fe11c6653db4954554856d5e84a4a7.pdf
Yuan, J., & Kim, C. (2014). Guidelines for facilitating the development of learning communities in online courses. Learning, Design, and Technology, 30(3), 220-232. https://doi.10.1111/jcal.12042
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
