Assessing the Reliability of Merging Chickering & Gamson’s Seven Principles for Good Practice with Merrill’s Different Levels of Instructional Strategy (DLISt7)
DOI:
https://doi.org/10.24059/olj.v20i2.784Keywords:
Community of Practice, Blended Learning, Online Learning, Measurement ModelAbstract
Based on Chickering and Gamson’s (1987) Seven Principles for Good Practice, this research project attempted to revitalize the principles by merging them with Merrill’s (2006) Different Levels of Instructional Strategy. The aim was to develop, validate, and standardize a measurement instrument (DLISt7) using a pretest-posttest Internet quasi-experiment. It was proposed that the instrument could then be used as a rubric either for facilitating the implementation of DLISt7, or as a set of unobtrusive diagnostic indicators for assessing the quality of learning experienced by students in blended and online courses. The study was conducted across five faculties at a regional Australian multi-campus university. The intent was to contribute to knowledge building by leveraging the data that had been collected, analyzed, and reported to generate awareness about the likelihood of scaffolding and scaling, varying levels of instructional strategies for communicating expectations, and relaying information. The idea was to produce a tool that would create more opportunities for more of the principles to be put to good use as an effectiveness multiplier. The findings from the analysis conducted using exploratory and confirmatory factor analysis verified the validity of DLISt7 and demonstrated excellent internal reliability values.References
Albion, P. R., & Redmond, P. (2006). Returning the Favour: Using Insights from Online Learning to Enhance On-Campus Courses. Paper presented at the 17th International Conference of the Society for Information Technology & Teacher Education (SITE 2006), Orlando, U.S.A. Retrieved September, 21, 2012, http:///www.editlib.org/f/22444
Anderson, T., & Dron, J. (2011). Three Generations of Distance Education Pedagogy. The International Review of Research in Open and Distance Learning, 12 (3). Retrieved August 23, 2012, from http://www.irrodl.org/index.php/irrodl/article/view/890/1663
Anderson, T., & Dron, J. (2012). Learning technology through three generations of technology enhanced distance education pedagogy. European Journal of Open, Distance and E-Learning, 2: 1-16. Retrieved January 9, 2013, from http://www.eurodl.org/materials/contrib/2012/Anderson_Dron.pdf
Ary, D., Jacobs, L. C., & Sorenson, C. (2010). Introduction to Research in Education (8th ed.). Belmont: Wadsworth.
Bangert, A. W. (2004). The Seven Principles of Good Practice: A framework for evaluating on-line teaching. The Internet and Higher Education, 7(3): 217-232. Retrieved September 16, 2009, from http://dx.doi.org/10.1016/j.iheduc.2004.06.003
Bangert, A. W. (2008a). The Influence of Social Presence and Teaching Presence on the Quality of Online Critical Inquiry. Journal of Computing in Higher Education, 20(1): 34-61. Retrieved May 27, 2011, from https://www.springerlink.com/content/g054232413m24746/fulltext.pdf
Bangert, A. W. (2008b). The Development and Validation of the Student Evaluation of Online Teaching Effectiveness. Computers in the Schools, 25(1): 25-47. Retrieved September 16, 2009, from http://dx.doi.org/10.1080/07380560802157717
Batts, D. (2008). Comparison of Student and Instructor Perceptions of Best Practices in Online Technology Courses. MERLOT Journal of Online Learning and Teaching, 4(4). Retrieved June 22, 2009, from http://jolt.merlot.org/vol4no4/batts_1208.htm
Brown, T. A. (2006). Confirmatory Factor Analysis for Applied Research. New York: The Guildford Press.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research (on Teaching). Boston: Houghton Mifflin Company.
Case, R., & Bereiter, C. (1984). From Behaviourism to Cognitive Behaviourism to Cognitive Development: Steps in the Evolution of Instructional Design*. Instructional Science, 13: 141-158.
Cattell, R. B., & Schuerger, J. M. (1978). Personality Theory in Action. Champaign: Institute for Personality and Ability Testing (IPAT).
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. Retrieved July 26, 2005, from http://honolulu. hawaii.edu/intranet/committees/ FacDevCom/guidebook/techtip/7 princip.htm
Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: technology as lever. Retrieved August 4, 2005, from http://www.tltgroup.org/ programs/seven.html
Chickering, A. W., & Gamson, Z. F. (1999). Development and Adaptations of the Seven Principles for Good Practice in Undergraduate Education. New Directions for Teaching and Learning, 80: 75-81. Retrieved January 25, 2010, from http://dx.doi.org/10.1002/tl.8006
Christensen, L. B. (1997). Experimental Methodology (7th ed.). Needham Heights: Allyn & Bacon.
Coakes, S. J., Steed, L., & Price, J. (2008). SPSS: Analysis without Anguish; version 15.0 for Windows. Milton: John Wiley & Sons.
Coakes, S. J., & Ong, C. (2011). SPSS: Analysis without Anguish; version 18.0 for Windows. Milton: John Wiley & Sons.
Coates, H. (2006). Student Engagement in Campus-based and Online Education. Abingdon: Routledge.
Cobbett, S. (2007). A Re-Conceptualized Model of Good Online Pedagogical Practices. In T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007 (pp. 2410-2415) by the Association for the Advancement of Computing in Education (AACE). Chesapeake: VA, Retrieved September 16, 2009, from http://www.editlib.org/f/26718
Cochran, G. W. (1977). Sampling Techniques. (3rd ed.). New York: John Wiley & Sons.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Mahwah: Lawrence Erlbaum Associates.
Creswell, J. W. (2012). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (4th ed.). Boston: Pearson Education.
Cronbach, L. J. (1990). Essential of Psychological Testing. New York: Harper & Row.
Field, A. (2009). Discovering Statistics using SPSS (3rd Ed.). London: Sage Publication.
Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating Cognitive Presence in Online Learning: Interaction is Not Enough. American Journal of Distance Education, 19(3), 133-148. http://dx.doi.org.ezproxy.usq.edu.au/10.1207/s15389286ajde1903_2
Garrison, D. R., Cleveland-Innes, M., & Fung, T. S. (2010). Exploring causal relationships among teaching, cognitive and social presence: Student perceptions of the community of inquiry framework. The Internet and Higher Education, 13(1-2), 31-36. http://dx.doi.org.ezproxy.usq.edu.au/10.1016/j.iheduc.2009.10.002
George, D., & Mallery, P. (2011). SPSS for Windows Step by Step: A simple guide and reference 18.0 update (11th ed.). Boston: Allyn & Bacon.
Glass, G. V., & Hopkins, K. D. (1996). Statistical Methods in Education and Psychology. Needham Heights: Allyn & Bacon.
Gorsuch, R. L. (1983). Factor Analysis (2nd ed.). New Jersey: Lawrence Erlbaum Associates.
Grant, M. R., & Thronton, H. R. (2007). Best Practices in Undergraduate Adult-Centered Online Learning: Mechanisms for Course Design and Delivery. MERLOT Journal of Online Learning and Teaching, 3(4), 346-356.
Guidera, S. G. (2003). Perceptions of the effectiveness of online instruction in terms of the seven principles of effective undergraduate education. Journal of Educational Technology Systems, 32(2 & 3), 139-178.
Hatcher, L. (2007). A Step-by-Step Approach to using SAS for Factor Analysis and Structural Equation Modelling (9th ed.). North Carolina: SAS Publishing.
Haughton, N. A., & Romero, L. (2009) The Online Educator: Instructional Strategies for Effective Practice. MERLOT Journal of Online Learning and Teaching, 5(3): 570-576.
Hodge, D. R. (2007). The Spiritual Competence Scale: A New Instrument for Assessing Spiritual Competence at the Programmatic Level. Research on Social Work Practice, 17(2): 287-295. Retrieved June 2, 2009, from http://rsw.sagepub.com/cgi/content/abstract/17/2/287
Hodge, D. R., & Gillespie, D. F. (2003). Phrase completions: An Alternative to Likert scales. (Note on Research Methodology). Social Work Research, 27 (1): 45-55. Retrieved May 27, 2009, from http://www.articlearchives.com/science-technology/behavior-cognition-psychology/986300-1.html
Hyman, T. (1974). Ways of Teaching (2nd ed.). New York: Harper & Row Publishers.
Johnson, B., & Christensen, L. (2008). Educational Research: Quantitative, Qualitative and Mixed Approaches (3rd ed.). Thousand Oaks: Sage Publications.
Johnson, B., & Christensen, L. (2012). Educational Research: Quantitative, Qualitative and Mixed Approaches (4th ed.). Thousand Oaks: Sage Publications.
Kehrwald, B., Reushle, S., Redmond, P., Cleary, K., Albion, P., & Maroulis, J. (2005). Online Pedagogical Practices in the Faculty of Education at the University of Southern Queensland. Working Paper Series (pp. i - 33). Toowoomba, Queensland: University of Southern Queensland.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41(2), 75-86. Retrieved October 4, 2012 from http://www.tandfonline.com/doi/abs/10.1207/s15326985ep4102_1#tabModule
Kuh, G. D., Pace, C. R., & Vesper, N. (1997). The Development of Process Indicators to Estimate Student Gains Associated with Good Practices in Undergraduate Education. Research in Higher Education, 38(4), 435-454.
Kuh, G. D., & Hu, S. (2001). The Effects of Student-Faculty Interaction in the 1990s. The Review of Higher Education, 24(3), 309-332.
Kuhn, T. S. (1970). The Structure of Scientific Revolutions (2nd ed.). Chicago: University of Chicago Press.
Lever-Duffy, J., McDonald, J. B., & Mizell, A. P. (2003). Teaching and Learning with Technology. Boston: Pearson Education.
Lever-Duffy, J., McDonald, J. B., & Mizell, A. P. (2005). Teaching and Learning with Technology (2nd ed.). Boston: Pearson Education.
McMillan, J. H., & Schumacher, S. (2009). Research in Education: Evidence-based Inquiry (7th ed.). New Jersey: Prentice-Hall.
Merrill, M. D. (2006). Hypothesized Performance on Complex Tasks as a Function of Scaled Instructional Strategies. 1-33. Retrieved October 26, 2009, from http://cito.byuh.edu/merrill/text/papers/Scaled%20Strategies.pdf
Merrill, M. D. (2008). What makes e3 (effective, efficient, engaging) instruction? Paper presented at the Association for Education Communication and Technology (AECT) Research Symposium. Retrieved March 23, 2010 from www.aect.org/events/symposia/Docs/Merrill_paper.doc
Merrill, M. D. (2009). First Principle of Instruction. In C.M. Reigeluth & A. A. Carr-Chellham (Eds.), Instructional Design Theories and Models; Building a Common Knowledge Base (Vol. III, pp. 41-56). Madison Ave: Routledge.
Morrison, R. G., Ross, M. S., & Kemp, E. J. (2001). Designing Effective Instruction (3rd ed.). New York: John Wiley & Sons.
Netemeyer, R. G., Beardon, W. O., & Sharma, S. (2003). Scaling Procedures: Issues and Applications. Thousand Oaks: Sage Publications.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). New York: McGraw-Hill.
Pedhazur, E. J., & Schmelkin, P. (1991). Measurement, Design, and Analysis: An Integrated Approach (Student ed.). New Jersey: Lawrence Erlbaum.
Reips, U.-D. (2000). The Web Experiment Method: Advantages, Disadvantages, and Solutions. In M. H. Birnbaum, Psychological Experiments on the Internet (pp.89-114). San Diego: Academic Press.
Reips, U.-D. (2002). Standards for Internet-Based Experimenting. Experimental Psychology, 49(4): 243-256.
Reiser, R. A. (2012). What Field Did You Say You Were In? Defining and Naming Our Field. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and Issues in Instructional Design and Technology (3rd ed., pp. 1-7). Boston: Allyn & Bacon.
Rust, J., & Golombok, S. (1989). Modern Psychometrics. London: Routledge.
Scardamalia, M., & C. Bereiter. (2006). Knowledge Building: Theory, Pedagogy, and Technology. In K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 97-118). New York,:Cambridge University Press. Retrieved September 24, 2009, from http://ikit.org/fulltext/2006_KBTheory.pdf
Shea, P., Li, C. S., & Pickett, A. (2006). A study of teaching presence and student sense of learning community in fully online and web-enhanced college courses. The Internet and Higher Education, 9, 175−190.
Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., Pickett, A., Gozza-Cohen, M., Wilde, J., & Jian, S. (2012). Learning presence: Additional research on a new conceptual element within the Community of Inquiry (CoI) framework. The Internet and Higher Education, 15(2):89-95.
Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., Gozza-Cohen, M., Jian, S., Pickett, A., Wilde, J., & Tseng, C. (2013). Online Learner Self-Regulation: Learning Presence Viewed through Quantitative Content-and Social Network Analysis. The International Review of Research in Open and Distance Learning, 14(3):427-461.
Shneiderman, B. (1998). Relate-Create-Donate: a teaching/learning philosophy for the cyber-generation. Computers & Education, 31, 25-39.
Skinner, B. F. (1954). The science of learning and the art of teaching. Harvard Educational Review, 24, 86-97.
Skinner, B. F. (1962). Walden Two (Paperback ed.). New York: The Macmillan Company.
Skinner, B. F. (1968). The Technology of Teaching. Englewood Cliffs, New Jersey: Prentice-Hall, Inc.
Spector, J. M., Ohradza, C., Van Schaack, A., & Wiley, D. A. (2005). Innovations in Instructional Technology; Essays in Honor of M. David Merril. Mahwah: Lawrence Erlbaum Associates.
Author. (2007). Student Perception Towards the Effectiveness of Instructional Strategies for Online Learning. Unpublished M.Ed dissertation. Tanjong Malim: UPSI.
Author. (2012a). Assessing the Effectiveness of the Different Levels of Instructional Strategies [DLISt] for Online Learning. Paper presented at the 2nd Malaysian Postgraduate Conference 2012, Bond University, Gold Coast, Queensland, Australia, http://msda.org.au/index.php/students-corner/malaysian-postgraduate-conference-2012-proceedings
Author. (2012b). Assessing the Effectiveness of the Different Levels of Instructional Strategies [DLISt] for Online Learning. Paper presented at the International Conference on Quality of Teaching and Learning (ICQTL 2012) organized by the Centre of Quality and Academic Development, University Malaysia Terengganu.
Author. (2012c). Assessing the Effectiveness of the Different Levels of Instructional Strategies [DLISt7] for Online Learning by Undergraduate Students. Unpublished Ph.D dissertation. Toowoomba.
Tuckman, B. W. (1999). Conducting Educational Research (5th ed.). Orlando: Harcourt Brace College Publishers.
Tuckman, B. W., & Harper, B. E. (2012). Conducting Educational Research (6th ed.). Lanham: Rowman & Littlefield Publishers.
Vandewaetere, M., & Desmet, P. (2009). Introducing Psychometrical Validation of Questionnaires in CALL Research: The Case of Measuring Attitude towards CALL. Computer Assisted Language Learning, 22(4): 349-380.
Wuensch, K. L., Shahnaz, A., Ozan, E., Kishore, M., & Tabrizi M. H. N. (2009). Technology and Pedagogy: The Association between Students’ Perceptions of the Quality of Online Courses and the Technologies Employed. MERLOT Journal of Online Learning and Teaching, 5(2): 253-262.
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
