Massive Omission of Consent (MOOC): Ethical Research in Educational Big Data Studies
Keywords:MOOCs, Ethics, Learning Analytics, Big Data, Privacy, AIED, AI in Education, Research Ethics
Ethical reviews of research plans function as a cornerstone of good research practice in order that no harm should come to participants. Ethical concerns have taken on a new salience in a digital world where data can be generated at scale. Big data research has grown rapidly, raising increased ethical concerns. Several intersecting areas of big data research exist within educational research, such as learning analytics, artificial intelligence (AI), and Massive Open Online Courses (MOOCs). In the current study, an investigation was made of peer-reviewed papers on MOOC teaching and learning to determine if they explicitly refer to (a) ethical considerations in their studies, and (b) obtaining formal ethical approval for their research. This investigation was accomplished through a review of MOOC-related, English-language papers available in Scopus database, over the course of a year. The review produced a total of 1,249 articles, of which, 826 articles related to empirical studies involving human participants where full text of the articles could be obtained. The string “ethic” was searched for within these articles, and resulting articles analyzed, which found that a small fraction, 42 articles (5.08%), mention ethics in relation to the study presented in the article, and only 13 articles (1.57%) explicitly mention obtaining formal ethical approval for the research. The findings show a lack of transparency in reporting on and/or engagement with ethical considerations in MOOC teaching and learning research. These findings indicate the need for further stakeholder engagement and sectoral dialogue in relation to ethics education and training for researchers; consideration of ethics in big data studies in education; and norms/policies in academic publishing for authors to report how ethical issues have been considered.
About COPE | Committee on Publication Ethics: COPE. (n.d.). Retrieved January 15, 2019, from https://publicationethics.org/about
Alario-Hoyos, C., Estévez-Ayres, I., Delgado Kloos, C., & Villena-Román, J. (2017). From MOOCs to SPOCs… and from SPOCs to flipped classroom. In Lecture Notes in Computer Science. Data Driven Approaches in Digital Education (pp. 347–354). doi:10.1007/978-3-319-66610-5_
Annabi, C. A., & Wilkins, S. (2016). The use of MOOCs in transnational higher education for accreditation of prior learning, programme delivery, and professional development. International Journal of Educational Management, 30(6), 959–975. doi:10.1108/ijem-05-2015-0057
Annear, M. J., Eccleston, C. E., McInerney, F. J., Elliott, K.-E. J., Toye, C. M., Tranter, B. K., & Robinson, A. L. (2016). A new standard in dementia knowledge measurement: Comparative validation of the Dementia Knowledge Assessment Scale and the Alzheimer’s Disease Knowledge Scale. Journal of the American Geriatrics Society, 64(6), 1329–1334. doi: 10.1111/jgs.14142.
ALLEA. (2017). The European code of conduct for research integrity (revised edition). https://ec.europa.eu/research/participants/data/ref/h2020/other/hi/h2020-ethics_code-of-conduct_en.pdf
Association for Learning Technology. (2023). Research in Learning Technology: Instructions for authors. https://journal.alt.ac.uk/index.php/rlt/guidelines
Astaneh, B., & Khani, P. (2019). The frequency of reporting ethical issues in human subject articles published in Iranian medical journals: 2009–2013. Science and Engineering Ethics, 25, 159–170. https://doi.org/10.1007/s11948-017-9989-9
Atenas, J., Havemann, L. & Timmermann, C. (2023) Reframing data ethics in research methods education: a pathway to critical data literacy. International Journal of Educational Technology in Higher Education 20(11) https://doi.org/10.1186/s41239-023-00380-y
Costello, E., Soverino, T., & Bolger, R. (2022). Mapping the MOOC research landscape: insights from empirical studies. International Journal of Emerging Technologies in Learning, 17(14), 4-19. https://dx.doi.org/10.3991/ijet.v17i14.28721
Barrow, M., Brandan, G., & Khandhar, P. (2021). Research Ethics. In StatPearls. StatPearls Publishing.
Bedenlier, S., Bond, M., Buntins, K., Zawacki-Richter, O., Kerres, M. (2020). Learning by Doing? Reflections on conducting a systematic review in the field of educational technology. In O. Zawacki-Richter, S. Kerres, M., Bedenlier, M. Bond, K. Buntins (Eds.), Systematic reviews in educational research. Springer VS. https://doi.org/10.1007/978-3-658-27602-7_7
Berman, A. H., Biguet, G., Stathakarou, N., Westin-Hägglöf, B., Jeding, K., McGrath, C., … Kononowicz, A. A. (2017). Virtual patients in a behavioral medicine Massive Open Online Course (MOOC): A qualitative and quantitative analysis of participants’ perceptions. Academic Psychiatry: The Journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry, 41(5), 631–641. doi:10.1007/s40596-017-0706-4
Birhane, A. (2021). Algorithmic injustice: a relational ethics approach. Patterns, 2(2). https://doi.org/10.1016/j.patter.2021.100205
Bonsu, D. O., Afoakwah, C. B., Abedi, M., Higgins, D., & Austin, J. J. (2022). Ethics reporting
in forensic science research publications—A review. Forensic Science International, 335, 111290. https://doi.org/10.1016/j.forsciint.2022.111290
Bouchet, F., Labarthe, H., Yacef, K., & Bachelet, R. (2017, July 9). Comparing peer recommendation strategies in a MOOC [Conference presentation]. 25th Conference on User Modeling, Adaptation and Personalization, Bratislava, Slovakia. doi:10.1145/3099023.3099036
Bujang, M. A., & Baharum, N. (2017). Guidelines of the minimum sample size requirements for Kappa agreement test. Epidemiology, Biostatistics and Public Health, 14(2). https://doi.org/10.2427/12267
Castaño-Muñoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education, 29(1), 28–46. doi:10.1007/s12528-016-9123-z
Coates, A. (2021). How often are basic details of the research process mentioned in social science research papers? Learned Publishing, 34(2), 128–136. https://doi.org/10.1002/leap.1330
Colledge, L., Moya-Anegón, F. de, Guerrero-Bote, V., López-Illescas, C., Aisati, M. E. & Moed, H. (2010). SJR and SNIP: Two new journal metrics in Elsevier’s Scopus. Serials, 23(3), 215–221. https://doi.org/10.1629/23215
Costello, E., Donlon, E., & Brown, M. (2019). Research ethics of Twitter for MOOCs. Online Learning, 23(3). https://doi.org/10.24059/olj.v23i3.1564
Czerniewicz, L., Deacon, A., Glover, M., & Walji, S. (2017). MOOC—making and open educational practices. Journal of Computing in Higher Education, 29(1), 81–97. doi:10.1007/s12528-016-9128-7
de Lima, M., & Zorrilla, M. E. (2017). Social networks and the building of learning communities: An experimental study of a social MOOC. The International Review of Research in Open and Distributed Learning, 18(1). https://doi.org/10.19173/irrodl.v18i1.2630
Duru, I., Sunar, A. S., Dogan, G., & White, S. (2017). Challenges of identifying second language English speakers in MOOCs. In Lecture Notes in Computer Science. Digital Education: Out to the World and Back to the Campus (pp. 188–196). doi:10.1007/978-3-319-59044-8_22
El Emam, K., Jonker, E., Arbuckle, L., & Malin, B. (2011). A Systematic Review of Re-Identification Attacks on Health Data. PLOS ONE, 6(12), e28071. https://doi.org/10.1371/journal.pone.0028071
Eriksson, T., Adawi, T., & Stöhr, C. (2017). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133–146. doi:10.1007/s12528-016-9127-8
Evans, D. P., Luffy, S. M., Parisi, S., & Del Rio, C. (2017). The development of a massive open online course during the 2014–15 Ebola virus disease epidemic. Annals of Epidemiology, 27(9), 611–615. doi:10.1016/j.annepidem.2017.07.137
Favaretto, M., De Clercq, E., Gaab, J., & Elger, B. S. (2020). First do no harm: An exploration of researchers’ ethics of conduct in Big Data behavioral studies. PLOS ONE, 15(11), e0241865. https://doi.org/10.1371/journal.pone.0241865
Falkner, K., Falkner, N., Szabo, C., & Vivian, R. (2016, July 11). Applying validated pedagogy to MOOCs. Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. doi:10.1145/2899415.2899429
Filimowicz, M. A., & Tzankova, V. K. (2017). Creative making, large lectures, and social media: Breaking with tradition in art and design education. Arts and Humanities in Higher Education, 16(2), 156–172. doi:10.1177/1474022214552197
Ferguson, R., and Clow, D. (2017). Where is the evidence? A call to action for learning analytics. Proceedings of the Seventh International Learning Analytics & Knowledge Conference, 56–65.
Flores, E. G. R., Montoya, M. S. R., & Mena, J. (2016, November 2). Challenge-based gamification as a teaching’ open educational innovation strategy in the energy sustainability area. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality. doi:10.1145/3012430.3012658
Frank, E., Tairyan, K., Everton, M., Chu, J., Goolsby, C., Hayes, A., & Hulton, A. (2016). A test of the first course (Emergency Medicine) that is globally available for credit and for free. Healthcare (Amsterdam, Netherlands), 4(4), 317–320. doi:10.1016/j.hjdsi.2016.02.003
Frick, T., & Dagli, C. (2016). MOOCs for research: The case of the Indiana university plagiarism tutorials and tests. Technology Knowledge and Learning, 21(2), 255–276. doi:10.1007/s10758-016-9288-6
Gallagher, S. E., & Savage, T. (2016). Comparing learner community behavior in multiple presentations of a Massive Open Online Course. Journal of Computing in Higher Education, 28(3), 358–369. doi:10.1007/s12528-016-9124-y
Grady, C. (2015). Institutional Review Boards. CHEST, 148(5), 1148–1155. https://doi.org/10.1378/chest.15-0706
Henderikx, M. A., Kreijns, K., & Kalz, M. (2017). Refining success and dropout in massive open
online courses based on the intention–behavior gap. Distance Education, 38(3), 353–368. doi:10.1080/01587919.2017.1369006
Higashi, R. M., Schunn, C. D., & Flot, J. B. (2017). Different underlying motivations and abilities predict student versus teacher persistence in an online course. Educational Technology Research and Development, 65(6), 1471–1493. doi:10.1007/s11423-017-9528-z
Higgins, J. P., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2019). Cochrane handbook for systematic reviews of interventions. John Wiley & Sons.
Hirsch, D. D. (2013). The glass house effect: Big Data, the new oil, and the power of analogy. Maine. Law. Review., 66, 373.
Holmes, W., & Tuomi, I. (2022). State of the art and practice in AI in education. European Journal of Education, 57(4), 542–570. https://doi.org/10.1111/ejed.12533
Ienca, M., Ferretti, A., Hurst, S., Puhan, M., Lovis, C., & Vayena, E. (2018). Considerations for ethics review of big data health research: A scoping review. PLOS ONE, 13(10), e0204937. https://doi.org/10.1371/journal.pone.0204937
Impey, C., & Formanek, M. (2021). MOOCS and 100 Days of COVID: Enrollment surges in massive open online astronomy classes during the coronavirus pandemic. Social Sciences & Humanities Open, 4(1), 100177. https://doi.org/10.1016/j.ssaho.2021.100177
International Data Sets. (n.d.). Retrieved January 11, 2019, from http://gsociology.icaap.org/dataupload.html
Jansen, R. S., van Leeuwen, A., Janssen, J., Kester, L., & Kalz, M. (2017). Validation of the self-regulated online learning questionnaire. Journal of Computing in Higher Education, 29(1), 6–27. doi:10.1007/s12528-016-9125-x
Johnson, J. A. (2014). The ethics of Big Data in higher education. The International Review of Information Ethics, 21, 3–10. https://doi.org/10.29173/irie365
Khalil, M., Prinsloo, P., & Slade, S. (2018). User consent in MOOCs—micro, meso, and macro perspectives. The International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3908
Khalil, M., & Ebner, M. (2017). Clustering patterns of engagement in Massive Open Online Courses (MOOCs): The use of learning analytics to reveal student categories. Journal of Computing in Higher Education, 29(1), 114–132. doi:10.1007/s12528-016-9126-9
Kim, W., Watson, S. L., & Watson, W. R. (2016). Perceived learning in three MOOCs targeting attitudinal change. Educational Media International, 53(3), 168–183. doi:10.1080/09523987.2016.1236890
Kitto, K., & Knight, S. (2019). Practical ethics for building learning analytics. British Journal of Educational Technology, 50(6), 2855–2870. https://doi.org/10.1111/bjet.12868
Komić, D., Marušić, S. L., & Marušić, A. (2015). Research integrity and research ethics in professional codes of ethics: Survey of terminology used by professional organizations across research disciplines. PLOS ONE, 10(7), e0133662. https://doi.org/10.1371/journal.pone.0133662
Lane, B., & Costello, E. (2019, October 16). Learning analytics, ethics and participant privacy. Proceedings of the The Online, Open and Flexible Higher Education Conference. https://doi.org/10.5281/ZENODO.3566979
Leach, M., & Hadi, S. M. (2017). Supporting, categorising and visualising diverse learner behaviour on MOOCs with modular design and micro-learning. Journal of Computing in Higher Education, 29(1), 147–159. doi:10.1007/s12528-016-9129-6
Loizzo, J., & Ertmer, P. A. (2016). MOOCocracy: The learning culture of massive open online courses. Educational Technology Research and Development, 64(6), 1013–1032. doi:10.1007/s11423-016-9444-7
Loizzo, J., Ertmer, P. A., Watson, W. R., & Watson, S. L. (2017). Adult MOOC learners as self-directed: Perceptions of motivation, success, and completion. Online Learning, 21(2). doi:10.24059/olj.v21i2.889
Longstaff, E. (2017). How MOOCs can empower learners: a comparison of provider goals and user experiences. Journal of Further and Higher Education, 41(3), 314–327. doi:10.1080/0309877x.2015.1100715
Luaces, O., Díez, J., Alonso-Betanzos, A., Troncoso, A., & Bahamonde, A. (2017). Content-based methods in peer assessment of open-response questions to grade students as authors and as graders. Knowledge-Based Systems, 117, 79–87. doi:10.1016/j.knosys.2016.06.024
New code of conduct for researchers. (n.d.). ScienceDaily. https://www.sciencedaily.com/releases/2010/07/100721194803.htm
Nuñez, M. A., Barlow, J., Cadotte, M., Lucas, K., Newton, E., Pettorelli, N., & Stephens, P. A. (2019). Assessing the uneven global distribution of readership, submissions and publications in applied ecology: Obvious problems without obvious solutions. Journal of Applied Ecology, 56(1), 4–9. https://doi.org/10.1111/1365-2664.13319
Metcalf, J., Crawford, K., & Keller, E. (2015). Pedagogical approaches to data ethics. Council for Big Data, Ethics, and Society. https://bdes.datasociety.net/council-output/pedagogical-approaches-to-data-ethics-2/.
Milligan C., Littlejohn, A. (2017). Tutors and gatekeepers in sustainability MOOCS. On the Horizon, 92–102 https://doi.org/10.19173/irrodl.v18i2.3033
Mishra, D., Cayzer, S., & Madden, T. (2017). Tutors and gatekeepers in sustainability MOOCS. On the Horizon, 25(1), 45–59. doi:10.1108/oth-04-2016-0017
Mohapatra, S., & Mohanty, R. (2017). Adopting MOOCs for affordable quality education. Education and Information Technologies, 22(5), 2027–2053. doi:10.1007/s10639-016-9526-5
Morgan, M., Nylén, A., Butler, M., Eckerdal, A., Thota, N., & Kinnunen, P. (2017, November 16). Examining manual and semi-automated methods of analysing MOOC data for computing education. Proceedings of the 17th Koli Calling International Conference on Computing Education Research. Koli, Finland. doi:10.1145/3141880.3141887
Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18, 1–7. https://doi.org/10.1186/s12874-018-0611-x
Prinsloo, P., & Slade, S. (2017). Big data, higher education and learning analytics: Beyond justice, towards an ethics of care. In Big Data and Learning Analytics in Higher Education (pp. 109–124). doi:10.1007/978-3-319-06520-5_8
Prinsloo, P., Khalil, M., & Slade, S. (2023). Learning analytics as data ecology: A tentative proposal. Journal of Computing in Higher Education, 1–29. https://doi.org/10.1007/s12528-023-09355-4
Resnik, D. B., & Shamoo, A. E. (2011). The Singapore Statement on Research Integrity. Accountability in Research, 18(2), 71–75. https://doi.org/10.1080/08989621.2011.557296
Rieber, L. P. (2017). Participation patterns in a massive open online course (MOOC) about statistics. British Journal of Educational Technology, 48(6), 1295–1304. doi:10.1111/bjet.12504
Riofrío-Calderón, G., Ramírez-Montoya, M.-S., & Rodríguez-Conde, M.-J. (2016, November 2). Mediation practices for learning in MOOC courses to promote open innovation. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality. doi:10.1145/3012430.3012664
Salmon, G., Pechenkina, E., Chase, A.-M., & Ross, B. (2017). Designing massive open online courses to take account of participant motivations and expectations. British Journal of Educational Technology: Journal of the Council for Educational Technology, 48(6), 1284–1294. doi:10.1111/bjet.12497
Shapiro, H. B., Lee, C. H., Wyman Roth, N. E., Li, K., Çetinkaya-Rundel, M., & Canelas, D. A. (2017). Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers. Computers & Education, 110, 35–50. doi:10.1016/j.compedu.2017.03.003
Schuwer, R., Jaurena, I. G., Aydin, C. H., Costello, E., Dalsgaard, C., Brown, M., Jansen, D., & Teixeira, A. (2015). Opportunities and threats of the MOOC movement for higher education: The European perspective. The International Review of Research in Open and Distributed Learning, 16(6). https://doi.org/10.19173/irrodl.v16i6.2153
Swinnerton, B. J., Morris, N. P., Hotchkiss, S., & Pickering, J. D. (2017a). The integration of an anatomy massive open online course (MOOC) into a medical anatomy curriculum. Anatomical Sciences Education, 10(1), 53–67. doi:10.1002/ase.1625
Swinnerton, B., Hotchkiss, S., & Morris, N. P. (2017b). Comments in MOOCs: Who is doing the talking and does it help? Journal of Computer Assisted Learning, 33(1), 51–64. doi:10.1111/jcal.12165
Shuster, E. (1997). Fifty years later: The significance of the Nuremberg Code. New England Journal of Medicine, 337(20), 1436–1440. https://doi.org/10.1056/NEJM199711133372006
Slade, S., Prinsloo, P., & Khalil, M. (2019). Learning analytics at the intersections of student trust, disclosure and benefit. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 235–244. https://doi.org/10.1145/3303772.3303796
Springer Nature (2023, February 24) Springer open editorial policies https://www.springeropen.com/get-published/editorial-policies#ethics+and+consent
Tawfik, A. A., Reeves, T. D., Stich, A. E., Gill, A., Hong, C., McDade, J., … Giabbanelli, P. J. (2017). The nature and level of learner–learner interaction in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education, 29(3), 411–431. doi:10.1007/s12528-017-9135-3
The National Commission for the Protection of Human Subjects of Biomedical and Behavioural Research. (n.d.). The Belmont Report. Retrieved January 15, 2019, from https://www.hhs.gov/ohrp/sites/default/files/the-belmont-report-508c_FINAL.pdf
Tsai, Y.-S., Whitelock-Wainwright, A., & Gašević, D. (2021). More than figures on your laptop: (Dis)trustful Implementation of Learning Analytics Journal of Learning Analytics, 8(3), 81-100. https://doi.org/10.18608/jla.2021.7379
Veletsianos, G. (2017). Toward a generalizable understanding of Twitter and social media use
across MOOCs: Who participates on MOOC hashtags and in what ways? Journal of Computing in Higher Education, 29(1), 65–80. doi:10.1007/s12528-017-9131-7
Wewer Albrechtsen, N. J., Poulsen, K. W., Svensson, L. Ø., Jensen, L., Holst, J. J., & Torekov,
S. S. (2017). Health care professionals from developing countries report educational benefits after an online diabetes course. BMC Medical Education, 17(1), 97. https://doi.org/10.1186/s12909-017-0935-y
World Medical Association. (2013). WMA Declaration of Helsinki—Ethical principles for medical research involving human subjects. World Medical Association.
Wu, Y., Howarth, M., Zhou, C., Hu, M., & Cong, W. (2019). Reporting of ethical approval and
informed consent in clinical research published in leading nursing journals: a retrospective observational study. BMC Medical Ethics, 20, 1–10.
Yacobson, E., Fuhrman, O., Hershkovitz, S., & Alexandron, G. (2021). De-identification is insufficient to protect student privacy, or—what can a field trip reveal? Journal of Learning Analytics, 8(2), 83–92. https://doi.org/10.18608/jla.2021.7353
Wang, Z., Anderson, T., Chen, L., & Barbera, E. (2017). Interaction pattern analysis in cMOOCs based on the connectivist interaction and engagement framework. British Journal of Educational Technology: Journal of the Council for Educational Technology, 48(2), 683–699. doi:10.1111/bjet.12433
Wewer Albrechtsen, N. J., Poulsen, K. W., Svensson, L. Ø., Jensen, L., Holst, J. J., & Torekov, S. S. (2017). Health care professionals from developing countries report educational benefits after an online diabetes course. BMC Medical Education, 17(1), 97.
Wu, Y., Howarth, M., Zhou, C., Hu, M., & Cong, W. (2019). Reporting of ethical approval and informed consent in clinical research published in leading nursing journals: a retrospective observational study. BMC Medical Ethics, 20, 1–10. https://doi.org/10.1186/s12910-019-0431-5
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education—where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1–27. https://doi.org/10.1186/s41239-019-0171-0
Zeide, E. & Nissenbaum, H. F. (2018). Learner privacy in MOOCs and virtual education. Theory and Research in Education, 16(3), 280–307. https://doi.org/10.1177/1477878518815340
Zhang, M., Yin, S., Luo, M., & Yan, W. (2016). Learner control, user characteristics, platform difference, and their role in adoption intention for MOOC learning in China. Australasian Journal of Educational Technology. doi:10.14742/ajet.2722
Zheleva, E., & Getoor, L. (2009). To join or not to join: The illusion of privacy in social networks with mixed public and private user profiles. Proceedings of the 18th International Conference on World Wide Web, 531. https://doi.org/10.1145/1526709.1526781
Zook, M., Barocas, S., Boyd, D., Crawford, K., Keller, E., Gangadharan, S. P., Goodman, A., Hollander, R., Koenig, B. A., Metcalf, J., Narayanan, A., Nelson, A., & Pasquale, F. (2017). Ten simple rules for responsible big data research. PLOS Computational Biology, 13(3), e1005399. https://doi.org/10.1371/journal.pcbi.1005399
Copyright (c) 2023 Eamon Costello, James Brunton, Richard Bolger, Tiziana Soverino, Clément Juillerac
This work is licensed under a Creative Commons Attribution 4.0 International License.
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions