Are There Metrics for MOOCS From Social Media?

Alan Ruby, Laura Perna, Robert Boruch, Nicole Wang

Abstract


Since “the year of the MOOC” in 2012, the effectiveness of massive open online courses (MOOCs) has been widely debated. Some argue that MOOCs are not an effective mode of instructional delivery because of low completion rates. In the interest of developing alternative indicators of performance this study draws from recent efforts to measure engagement in social media, as well as from research on indicators of student engagement in traditional college courses. Using data from 16 Coursera MOOCs offered by the University of Pennsylvania we calculate standardized access rates for lectures and assessments. While these indicators have clear limitations as measures of educational progress they offer a different, more nuanced understanding of the level and nature of users’ engagement with a MOOC. This paper shows that a very small share of users takes up available opportunities to access course content but notes that the standardized access rates compare favorably with those for social media sites and with response rates to large-scale direct mail marketing programs. For MOOC providers and platform managers, indicators like the ones developed in this study may be a useful first step in monitoring the extent to which different types and combinations of activities may be providing better opportunities for learning.

Keywords


MOOCS, post secondary education,measurement

Full Text:

PDF

References


Anand, S., & Hanson, K. (1997). Disability-adjusted life years: a critical review. Journal of health economics, 16(6), 685-702.

Astin, A.W., (1984). Student involvement: a development theory for higher education. Journal of College Student Personnel, 25(4), 297-308.

Perna, L., Ruby, A., Boruch, R., Wang, N. Scull, J. Ahmad, S., & Evans C. (2014). Moving through MOOCs: understanding the progression of users in massive open online courses, Educational Researcher, 43, 421-432.

Azevedo, R. ( 2015). Defining and measuring engagement and learning in science: conceptual, theoretical, methodological and analytical issues. Educational Psychologist. 50(1), 84-94.

Beer, C. Clark, K., & Jones, D. (2010). Indicators of engagement. In C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp.75-86).

Bhupathiraju, S. N., Pan, A., Manson, J. E., Willett, W. C., van Dam, R. M., & Hu, F. B. (2014). Changes in coffee intake and subsequent risk of type 2 diabetes: three large cohorts of U.S. men and women. Diabetologia, 57(7), 1346-1354.

Brinton, C.G., Lam, H., Chiang, M., Liu, Z., Jain, S., Wong, F. (2013, December). Learning about social learning in MOOCs: from statistical analysis to generative model. Unpublished paper.

Chen, P., Gonyea, R., & Kuh, G. (2008). Learning at a distance: engaged or not. Innovate: Journal of Online Education, 4(3), 1-8.

Chi, M.T.H. ( 2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Topics in Cognitive Science. 1. 73-105.

Coates, H. (2007). A model of online and general campusâ€based student engagement. Assessment & Evaluation in Higher Education, 32(2), 121-141.

Coffrin, C., Corrin, L., de Barba, P., & Kennedy, G. (2014, March). Visualizing patterns of student engagement and performance in MOOCs. In Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (pp. 83-92). ACM.

DMA (2012). Direct mail response rates beat digital. http://www.dmnews.com/dma-direct-mail-repsonse-rates-beat-digital/article/245780/

Dunham, K., (2014). The beginner’s guide to social media metrics: engagement. http://blog.hootsuite.com/beginners-guide-engagement

Eyl, S., (2013, February 13). Study on Facebook engagement and interaction rates. fanpage karma. Retrieved from http://blog.fanpagekarma.com/2013/02/13/facebook-engagement-interaction-rates/

French, D., (2012). Reaching today’s buyers through social media. Furniture Today, 36(33).

Gee, J.P., (2013a). The anti-education era: creating smarter students through digital learning. New York: Palgrave MacMillan.

Gee, J.P., (2013b). Games for learning. Educational Horizons 19(4), 16-20.

Gee, J.P. & Hayes, E.R. (2011). Language and learning in the digital age. New York: Routledge.

Guo, P.J., Kim, J., & Rubin, R. (2014) How Video Production Affects Student Engagement: An Empirical study of MOOC Videos, L @ S 2014, March 4-5, Atlanta Georgia. http://dx.doi.org/10.1145/2556325.2566239.

Haggard, S., (2013). The maturing of the MOOC. Department of Business, Industry and Skills Research Paper No. 130.

Ho, A.D., Reich, J., Nesterko, S., Seaton, D.T., Mullaney, T., Waldo, J., & Chuang, I. (2014). Harvard X and MITx: the first year of open online courses. Harvard X & MITx Working Papers No.1.

Hoffman, D.L, & Fodor, M., (2010). Can you measure the ROI of your social media marketing? MIT Sloan Management Review 52(1), 41-49.

Hollebeek, L.D., Glynn, M.S., & Brodie, R.J., (2014). Consumer brand engagement in social media: conceptualization, development and validation. Journal of Interactive Marketing 28, 149-165.

Ken, D. (2014, March 15). Why engagement rate is more important than likes on your Facebook. Social media today. Retrieved from http://www.socialmediatoday.com/content/why-engagement-rate-more-important-likes-your-facebook

Kietzmann, J.H., Hermkens, K., McCarthy, I.P., & Silvestre, B.S., (2011). Social Media? Get serious! Understanding the functional building blocks of social media. Business Horizons, 54, 241-251.

King, M.A. (1972). Primary and secondary indicators of education. In A. Shonfield and S. Shaw (eds.), Social indicators and social policy (pp. 53-66). London: Heinemann Educational Books.

Koller, D., Ng, A., Chuong, D., & Chen, Z. (2013, May/June). Retention and intention in massive open online courses. Educause Review.

Kromann, N., & Green, A. (1980). Epidemiological studies in the Upernavik district, Greenland. Acta Medica Scandinavica, 208(1â€6), 401-406.

Kuh, G.D. (2009a). The national survey of student engagement: conceptual and empirical foundations. New directions for Institutional Research, 2009(141), 5-20.

Kuh, G.D. (2009b). What student affairs professionals need to know about student engagement. Journal of College Student Development. 50(6), 683-706.

Kuh, G.D., Kinzie, I., Bridges, B.K., & Hayek, J.C., (2006). What matters to student success: a review of the literature. Washington D.C.: National Postsecondary Education Cooperative.

MOOCs @ Edinburgh Group (2013). MOOCS @ Edinburgh 2013, Report #1. University of Edinburgh. Retrieved from https://www.era.lib.ed.ac.uk/handle/1842/6683

OECD, (2014). Education at a glance 2014: OECD indicators. Paris: OECD Publishing.

Perna, L.W., Ruby, A., Boruch, R., Wang, N., Scull, J., Ahmad, S., & Evans, C. (2014). Moving through MOOCs: understanding the progression of users in MOOCs. Educational Researcher, 43, 421-432.

Price, D.V. & Tovar, E. (2014). Student engagement and institutional graduation rates: identifying high impact educational practices for community colleges. Community College Journal of Research and Practice, 38(9), 766-782.

PR Newswire (2013, July 24). Engagement rates for Stuart Weitzman’s international Facebook campaigns exceed industry average by more than 5000%. Retrieved from http://www.prnewswire.com/news-releases/engagement-rates-for-stuart-weitzmans-international-facebook-campaigns-exceed-industry-average-by-more-than-5000-216736261.html

Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: student engagement in online learning. Journal of Education for Business, 84(2), 101-109.

Sfard, A., (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27(2), 4-13.

Social Bakers, (2013, February 25). Engagement rate: a metric you can count on. Retrieved from http://www.socialbakers.com/blog/1427-engagement-rate-a-metric-you-can-count-on

Steffens, K. (2015). Competences, learning theories and MOOCs: recent developments in lifelong learning. European Journal of Education 51(1), 41-58.

Stone, R. (1970). Mathematical models of the economy and other essays. London: Chapman and Hall.

Thoms, B., & Eryilmaz, E. (2014). How media choice affects learner interactions in distance learning classes. Computers & Education, 75, 112-126.

Tinto, V., (1975). Dropout from higher education: a theoretical synthesis of recent research. Review of Educational Research 45(1), 89-125.

Tinto,V., (1987). Leaving college: rethinking the causes and cures of student attrition. Chicago: University of Chicago Press.

White, D., & Le Cornu, A. (2011). Visitors and residents: a new typology for online engagement. First Monday, 16(9). doi:10.5210/fm.v16i9.3171

Wright, F., White, D., Hirst, T., & Cann, A. (2014). Visitors and residents: mapping student attitudes to academic use of social networks. Learning, Media and Technology, 39(1), 126-141.

Yoon, E., & Eckels, J., (2013). Make these changes to help digital marketing fulfill its potential. HBR Blog Network, http://blogs.hbr.org/2013/05/mahe-these-chngaes-to-hekp-dig/




DOI: http://dx.doi.org/10.24059/olj.v19i5.567

Bookmark and Share