The Effects of Nudges on Students’ Use of the Diagnostic Assessment and Achievement of College Skills

David W Franklin Jr, Jason Bryer, Angela M Lui, Heidi L Andrade, Diana Akhmedjanova

Abstract


The purpose of this study is to examine the effects of nudges on online college students’ use of the Diagnostic Assessment and Achievement of College Skills (DAACS), a suite of free, online assessments, feedback, and resources designed to optimize student success in college. The results indicate that the nudges had an effect on students’ completion of the DAACS and on accessing the feedback. The effectiveness varied by type of nudge and the order in which a series of nudges was sent. Simply sending the nudges did not have a direct effect on academic outcomes, but students who responded to one series of nudges were more successful than those who did not.


Keywords


academic readiness; online learning; nudges; feedback; postsecondary education

Full Text:

PDF

References


ACT. (2019). The condition of college and career readiness 2019. ACT. https://www.act.org/content/dam/act/secured/documents/cccr-2019/National-CCCR-2019.pdf

Akhmedjanova, D., Lui, A. M., Andrade, H. L., & Bryer, J. (2019, April 4–8). Validity and reliability of the DAACS writing assessment [Conference presentation]. Annual meeting of the National Council on Measurement in Education (NCME), Toronto, Canada.

Alan, S., Boneva, T., & Seda, E. (2019). Ever failed, try again, succeed better: Results from a randomized educational intervention on grit. The Quarterly Journal of Economics, 134(3), 1121–1162. https://doi.org/10.1093/qje/qjz006

Azmat, G., Bagues, M., Cabrales, A., & Iriberri, N. (2019). What you don’t know…can’t hurt you? A natural field experiment on relative performance feedback in higher education. Management Science, 65(8), 3714–3736. https://doi.org/10.1287/mnsc.2018.3131

Bailey, T., Bashford, J., Boatman, A., Squires, J., Weiss, M., Doyle, W., ... & Young, S. H. (2016). Strategies for postsecondary students in developmental education: A practice guide for college and university administrators, advisors, and faculty. NCEE 2017-4011. What Works Clearinghouse. https://files.eric.ed.gov/fulltext/ED570881.pdf

Bettinger, E. P., & Baker, R. B. (2014). The effects of student coaching: An evaluation of a randomized experiment in student advising. Educational Evaluation and Policy Analysis, 36(1), 3–19. https://doi.org/10.3102/0162373713500523

Bird, K. A., Castleman, B. L., Denning, J. T., Goodman, J., Lamberton, C., & Rosinger, K. O. (2021). Nudging at scale: Experimental evidence from FAFSA completion campaigns. Journal of Economic Behavior & Organization, 183, 105–128. https://doi.org/10.1016/j.jebo.2020.12.022

Bryer, J., Lui, A. M., Andrade, H. L., Franklin, D., & Cleary, T. (2019, April 5–9). Efficacy of the Diagnostic Assessment and Achievement of College Skills on multiple success indicators [Roundtable presentation]. Annual meeting of the American Educational Research Association (AERA), Toronto, Canada.

Castleman, B. L., & Page, L. C. (2015). Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates. Journal of Economic Behavior & Organization, 115, 144–160. https://doi.org/10.1016/j.jebo.2014.12.008

Castleman, B. L., & Page, L. C. (2016). Freshman year financial aid nudges: An experiment to increase FAFSA renewal and college persistence. Journal of Human Resources, 51(2), 389–415. https://doi.org/10.3368/jhr.51.2.0614-6458R

Castleman, B. L., & Page, L. C. (2017). Parental influences on postsecondary decision making: Evidence from a text messaging experiment. Educational Evaluation and Policy Analysis, 39(2), 361–377. https://doi.org/10.3102/0162373716687393

Coffman, L. C., Featherstone, C. R., & Kessler, J. B. (2017). Can social information affect what job you choose and keep? American Economic Journal: Applied Economics, 9(1), 96–117. https://doi.org/10.1257/app.20140468

Damgaard, M. T., & Nielsen, H. S. (2018). Nudging in education. Economics of Education Review, 64, 313–342. https://doi.org/10.1016/j.econedurev.2018.03.008

Dee, T. S., & Jacob, B. A. (2012). Rational ignorance in education: A field experiment in student plagiarism. Journal of Human Resources, 47(2), 397–434. https://doi.org/10.1353/jhr.2012.0012

Dobronyi, C. R., Oreopoulos, P., & Petronijevic, U. (2019). Goal setting, academic reminders, and college success: A large-scale field experiment. Journal of Research on Educational Effectiveness, 12(1), 38–66. https://doi.org/10.1080/19345747.2018.1517849

Eyink, J. R., Motz, B. A., Heltzel, G., & Liddell, T. M. (2020). Self‐regulated studying behavior, and the social norms that influence it. Journal of Applied Social Psychology, 50(1), 10–21. https://doi.org/10.1111/jasp.12637

Fay, M. P., Barnett, E. A., & Chavarín, O. (2017). How states are implementing transition curricula: Results from a national scan (CCRC Research Brief). Community College Research Center, Columbia University. https://ccrc.tc.columbia.edu/media/k2/attachments/ccrc-research-brief-how-states-implementing-transition-curricula-results-national-scan.pdf

Gurantz, O., Howell, J., Hurwitz, M., Larson, C., Pender, M., & White, B. (2020). A national‐level informational experiment to promote enrollment in selective colleges. Journal of Policy Analysis and Management. https://doi.org/10.1002/pam.22262

Hertwig, R., & Grüne-Yanoff, T. (2017). Nudging and boosting: Steering or empowering good decisions. Perspectives on Psychological Science, 12(6), 973–986. https://doi.org/10.1177/1745691617702496

Hyman, J. (2020). Can light‐touch college‐going interventions make a difference? Evidence from a statewide experiment in Michigan. Journal of Policy Analysis and Management, 39(1), 159–190. https://doi.org/10.1002/pam.22155

Jaggars, S. S., & Stacey, G. W. (2014). What we know about developmental education outcomes. Community College Research Center. https://ccrc.tc.columbia.edu/media/k2/attachments/what-we-know-about-developmental-education-outcomes.pdf

Lipnevich, A., & Smith, J. (Eds.) (2018). Cambridge handbook of instructional feedback. Cambridge University Press.

Martinez, I. (2013). The effects of nudges on students’ effort and performance: Lessons from a

MOOC. EdPolicyWorks Working Paper. https://curry.virginia.edu/working-paper-effects-nudges-students%E2%80%99-effort-and-performance-lessons-mooc

Mokher, C. G., Barnett, E., Leeds, D. M., & Harris, J. C. (2019). Re-envisioning college readiness reforms: Florida’s statewide initiative and promising practices in other states. Change: The Magazine of Higher Learning, 52(2), 14–23. https://doi.org/10.1080/00091383.2019.1569968

Motz, B. A., Mallon, M. G., & Quick, J. D. (2021). Automated educative nudges to reduce missed assignments in college. IEEE Transactions on Learning Technologies, 14(2), 189–200. https://doi.org/10.1109/TLT.2021.3064613

National Assessment of Educational Progress. (2019, September 1). The nation’s report card. NAEP. https://www.nationsreportcard.gov/

National Center for Public Policy and Higher Education, & Southern Regional Education Board (2010). Beyond the rhetoric: Improving college readiness through coherent state policy (A special report). (ERIC Document Reproduction Service No. ED 510 711). http://www.highereducation.org/reports/college_readiness/CollegeReadiness.pdf

Page, L. C., Castleman, B. L., & Meyer, K. (2020). Customized nudging to improve FAFSA completion and income verification. Educational Evaluation and Policy Analysis, 42(1), 3–21. https://doi.org/10.3102/0162373719876916

Thaler, R. H., & Sunstein, C. R. (2008). Nudge: Improving decisions about health, wealth and happiness. Yale University Press.

Tran, A., & Zeckhauser, R. (2012). Rank as an inherent incentive: Evidence from a field experiment. Journal of Public Economics, 96(9–10), 645–650. https://doi.org/10.1016/j.jpubeco.2012.05.004

Unkovic, C., Sen, M., & Quinn, K. M. (2016). Does encouragement matter in improving gender imbalances in technical fields? Evidence from a randomized controlled trial. PLOS One, 11(4), 1–15. https://doi.org/10.1371/journal.pone.0151714

Vanderslice-Barr, J. (2020). Demographic differences in post-secondary outcomes following participation in the Diagnostic Assessment and Achievement of College Skills intervention. [Doctoral dissertation, University at Albany, SUNY]. ProQuest.

Wagner, V., & Riener, G. (2015). Peers or parents? On non-monetary incentives in schools. Working Paper. https://ideas.repec.org/p/zbw/dicedp/203.html

Yeomans, M., & Reich, J. (2017). Planning prompts increase and forecast course completion in massive open online courses. Proceedings of the Seventh International Conference on Learning Analytics & Knowledge (pp. 464–473).

Yi, A. (2019). The power of the nudge: A guide to behavioral interventions for student success. EAB. http://jupiter.clarion.edu/~pwoodburne/ECON510/Ch%2016a%20Rationality/Power%20of%20the%20nudge%20A%20guide%20to%20behavioral%20interventions%20for%20student%20success.pdf




DOI: http://dx.doi.org/10.24059/olj.v26i2.2674



Copyright (c) 2022 David W Franklin Jr, Jason Bryer, Angela M Lui, Heidi L Andrade, Diana Akhmedjanova

License URL: https://creativecommons.org/licenses/by/4.0/