The Impact of Online or F 2 F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course : Closing the Gap

Blended learning options vary and universities are exploring an assortment of instructional combinations, some involving video lectures as a replacement for face-to-face (f2f) lectures. This methodological study investigates the impact of the provision of lecture choice (online or f2f) on overall student achievement and course engagement. This research uses a within-group design to obtain baseline data on a single set of physics students (n=168), and investigates the impact of providing a lecture viewing choice (online, f2f) mid-semester on student achievement (tests, homework, and standardized conceptual evaluation scores), and course engagement (student lecture viewing, homework submissions, bonus project submissions, and note taking behaviors). The study reveals that the type of lecture does not serve to significantly impact overall student achievement or engagement. However, although recorded and f2f lectures demonstrate an overall educationally equivalent impact, students who elect a high level of recorded lecture use were significantly lower performing and less engaged before the option to watch recorded lectures was introduced and largely continued to be so after the option was introduced, but there was evidence of a reduction in achievement and engagement differences after the option is introduced. Therefore, results of this study suggest weaker performing students self-select higher levels of recorded lecture use, and the use of these video lectures may assist this specific group of students in closing the gap between themselves and students who were initially higher performing and more engaged.


Introduction
Web-based lecture technologies have been available for over fifteen years, and usage has increased greatly in the past decade.In a push to provide blended learning options, higher education institutions are increasingly adopting technologies such as lecture capture, webcasting, and video podcasting to supplement and even replace face-to-face (f2f) lectures (Danielson, Preast, Bender, & Hassall, 2014).Video lectures are viewed as a way to extend classroom instruction to online learning (Brecht, 2012), and are also touted for the ability to provide flexibility to students who are juggling competing demands for their time (Gysbers, Johnston, Hancock, & Denyer, 2011).
Although reported use of video lectures is increasing, our knowledge about the impact of recorded video lecture access on academic achievement and student engagement is limited.As acknowledged by Gorissen, Van Bruggen, and Jochems (2012), "Most studies are limited to the overall opinions and perceptions of students and lecturers about the usefulness of the recorded lectures" (p.298).Additionally, studies that have investigated the impact of recorded video lecture access on achievement and student engagement have resulted in mixed findings with a lack of consensus regarding the relationship between recorded video lecture use and learning outcomes (Danielson et al., 2014).
This study furthers the research literature on video lecture use by providing a robust examination of the impact of the provision of recorded video lectures on academic achievement and student engagement.While previous video lecture research has been conducted, many of these studies have relied on perception and self-report data, which have limitations (Stone, Bachrach, Jobe, Kurtzman, & Cain, 1999).Other studies have used self-selection and/or randomized research designs in which students are placed into groups, with each group given differing levels of access to recorded video lectures.Although these designs allow for an exploration of the impact of video lecture use, they fail to take into account self-selection biases or student differences that may exist prior to the beginning of the research treatment.
To address the aforementioned issues, the current study utilizes a within-subjects design that allows for the gathering of student baseline data prior to the provision of video lecture options within a large lecture-based course, and collection of data during and after optional video lecture use.This design affords the current researchers the opportunity to fully explore the impact of optional recorded video lectures on academic achievement and student engagement.

Lecture Attendance and Achievement
According to Behr (1988), the most common teaching method used in higher education is the lecture, and lecture attendance has been shown to have a positive impact on student achievement (Budig, 1991;Jenne, 1973;Kantartzi, Allen, Lokhi, Grier, & Abdelmajid, 2010;Moore, 2003;Nist, 1995;Slem, 1983;Van Blerkom, 1996).In fact, a recent meta-analysis by Credé, Roch, and Kieszczynka (2010) reports attendance to be the best predictor of academic achievement, including attendance in classrooms that predominantly incorporate lecture methods.
However, previous studies have failed to determine whether increased achievement associated with lecture attendance is related to the act of viewing the lecture, or if other social presence factors are involved (Credé et al., 2010).The use of technologies which allow recorded video lectures to act as a replacement for f2f lectures offers researchers the opportunity to more closely explore relationships between lecture viewing (f2f or online) and student achievement.

Student Absenteeism
Despite the plethora of research supporting the relationship between lecture attendance and higher academic achievement, student absenteeism remains an issue (Barlow & Fleischer, 2011;Bati,

Recorded Video Lectures
Recorded video lectures may extend instruction and potentially address factors associated with lecture absenteeism (Brecht, 2012).Recorded video lectures offer students flexibility relative to instructional time, pace of learning, study location, and opportunities for review.More specifically, researchers report that recorded lectures give students the ability to view lectures at the time of the student's choosing; allow students to make up for unavoidable conflicts which result in missed class; enable student control of lecture delivery to fit the student's learning pace; allow students to view lectures without environmental distractions such as overcrowded classrooms; demand fewer trips to campus; and provide extra flexibility for busy students (Bati et al., 2013;Ealy, 2013;Gorissen et al., 2012;Gysbers et al., 2011;Simpson, 2006;Yudko, Hirokawa, & Chi, 2008).
Researchers argue recorded video lectures have transformed learning by extending the lecture experience and providing choice in relation to lecture.However, the use of recorded video lectures is not without concern.The use of recorded video lectures has been shown to relate to a reduction in f2f class attendance (Grabe, Christopherson, & Douglas, 2005;McKinlay, 2007;Traphagan, Kucsera, & Kishi, 2010;von Konsky, Ivins, & Gribble, 2009), and the concern for class absenteeism is a primary barrier to recorded video lecture use by faculty (Chang, 2007).

Student Lecture Viewing
Student preferences relative to live versus recorded lectures correspond to faculty concerns.Results from Artino (2010) show that although students appreciate the convenience of online options, "if given the choice, many would rather complete courses in a traditional, classroom-based format" (p.275).Students report that live lectures provide a better learning environment than online recorded lectures because of the discipline imposed by scheduled live lectures, and suggest web based lectures may predispose students to procrastination (Gysbers et al., 2011).Students also find recorded video lectures to be of benefit.Despite a preference for f2f lectures, students believe they can learn as well from recorded lectures as from f2f lecture attendance (Gorissen et al., 2012;Gosper et al., 2008).
However, there is a concern that students may not be capable of making good decisions regarding lecture viewing and attendance, particularly if they are already struggling.While some studies suggest students may be more likely to attend lectures they perceive to be difficult (Clay & Breslow, 2006;Gysbers et al., 2011), others demonstrate that lower performing students prefer educational software or online resources to traditional lectures (Albert, 2004;Traphagan et al., 2010).Similarly, research found lower performing students are more likely to supplement attendance with web-based lectures (von Konsky et al., 2009), which raises the concern that students may choose to replace live lectures with web-based lectures to their detriment.

Recorded Video Lectures and Achievement
Investigations on the impact of recorded lectures on achievement have resulted in mixed findings.As Danielson et al. (2014) report, "We could not identify any systematic meta-analyses providing a consensus regarding the relationship between lecture-capture use and learning outcomes in a post-secondary or any other learning context" (p.121).
To date much of the research on the relationship between recorded lecture viewing and achievement has primarily focused on student perceptions, with a few studies using self-selection or randomized research designs to investigate differences between the achievement of students receiving f2f versus recorded lectures (Gosper et. al, 2008;Hove & Corcoran, 2008;McCredden & Baldock, 2009;Owston, Lupshenyuk, & Wideman, 2011;Solomon, Ferenchick, Laird-Fick, & Kavanaugh, 2004;Traphagan et al., 2010).Findings from these studies vary, and the link between live lecture attendance versus video lecture viewing and achievement has not been strongly established.This has led to the suggestion that student achievement may be influenced more by overall engagement in the course than by f2f versus online lecture attendance (Gysbers et al., 2011;McCredden, & Baldock, 2009;von Konsky et al., 2009).

Recorded Video Lectures and Engagement
Student engagement has been touted as the single best predictor of student learning (Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008).Despite the stated importance of student engagement, researchers indicate that engagement in lecture courses can be limited, and students who do not attend lectures miss out on both social and content engagement opportunities that can enhance learning (Black, 2005;Cooper & Robinson, 2000;Massingham & Herrington, 2006;McGarr, 2009;Phillips et al., 2007).
While technologies such as recorded video can potentially increase opportunities for student engagement with the content (Laird & Kuh, 2005), there remains the concern that the incorporation of these technologies can also serve as a distractor for students (Reisberg, 2000).Thus, as within other areas of study related to video lecture research, findings are mixed regarding the relationship between recorded video lectures and student engagement.

Summary and Need for Study
There is a lack of consistent research findings identifying the impact of lecture choice (online versus f2f) on student achievement and engagement.Offering students a lecture viewing choice is lauded by some researchers since it provides busy students a means to view lectures that would have otherwise been missed, thereby potentially increasing lecture viewing.Other scholars report students are not capable of making good decisions regarding lecture viewing, particularly if they are already struggling, and raise the concern that recorded video lectures could lead to less lecture viewing.Existing research related to the use of online lectures has primarily involved self-report data gathering methods, and more study is needed to determine the overall impact of lecture choice.
Similarly, existing research demonstrates a relationship between lecture attendance and student achievement, but what remains unclear is whether the act of viewing lectures online versus attending f2f lectures impacts achievement.Some studies find the use of recorded video lectures supports increased student achievement, while others report no impact of recorded lecture viewing on student achievement.Many of these studies utilize research designs that fail to account for initial student differences or establish a baseline for student achievement prior to the introduction of video lectures.Additional research that controls for student variability while examining both f2f and recorded lecture viewing is needed to more systematically study the relationship between the types of lecture viewing (online versus f2f) and measures of student achievement.
Lastly, researchers indicate multiple forms of student engagement are critical to student success; however, findings identifying the impact of lecture choice (online versus f2f) on student engagement are mixed.Some researchers report that student engagement in a traditional lecture

Research Questions
This study furthers previous research by using a within-group research design to examine the potential impacts of optional recorded video lectures on student achievement and course engagement.To guide this investigation, the following research questions are addressed: What is the impact of student lecture choice (recorded vs. f2f) on 1) achievement and 2) course engagement?More specifically, the use of this study's within-group design adds to previous research related to online versus f2f achievement by affording a pre-post controlled examination of student achievement (as measured by examinations, homework scores, and a standardized test) between those who choose to watch recorded video lectures and those who attend f2f lectures.Additionally, the study addresses researcher concerns relative to how online video lectures may impact student engagement in multiple course-related activities by measuring student lecture viewing (f2f and recorded), submission of homework assignments, completion of optional projects, and self-reported note taking behaviors both before and after the introduction of the video lecture option to determine whether those who choose to view recorded video lectures maintain the same amount of engagement in course-related activities as those who choose to watch lectures in a f2f format.

Method Context
This study examined the effect of the mid-semester introduction of the option to either attend lectures live in the traditional f2f setting or watch recorded video lectures delivered over the internet.This research was performed in a second-semester university Physics course at a public land-grant institution serving approximately 25,000 students.The course was a required course for most of the students enrolled (n=168), and the course featured both required lecture and laboratory components so that all students were required to attend two f2f laboratory sessions each week.Students electing to watch online video lectures still had to complete on-campus laboratory requirements for the course.The f2f lectures were given in a large lecture theater and were presented in 50-minute sessions.A total of twenty-nine lectures were given throughout the semester, approximately two per class week.The lectures were given with the lecturer working from a set of notes, using those notes to present material on a whiteboard, and then projecting the notes onto a screen above the whiteboard.The lecturer periodically solicited questions of the students, but did not perform any demonstrations; these were left for the laboratory segment of the course.The class had been presented in the same format by the same instructor for over a decade.All homework and examination materials for the course were prepared by this seasoned instructor, producing a highly stable educational setting for this research.The only substantive change to this stable class environment in the semester studied was the mid-semester introduction of the video lecture option.

Description of Experiment
At the beginning of the semester all students within the Physics course were required to attend the f2f lectures; attendance was monitored with a lecture quiz administered at the end of most lectures.In preparation for offering a future online lecture section of the course, the instructor produced video versions of the f2f lectures for the course.With knowledge of the availability of the video lecture resource, some students requested the option of viewing the recorded video lectures instead of attending the lectures in person.In response to student requests, the instructor agreed to allow students to watch lectures on video in place of attending the f2f lecture once the class reached The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap the mid-semester point.The class administered four in-semester examinations, and the video option was offered to students immediately after the second in-semester examination.Prior to course enrollment, no student was made aware that the option to watch the lecture on video would be available.
The video lectures were provided through the university's learning management system (LMS) course site, and requirements and instructions for watching the lectures on video were explained in the face-to-face lecture immediately following the second in-semester examination, and in an email to the class.Face-to-face lecture attendance continued to be monitored with a lecture quiz.Students electing to watch recorded video lectures were given the same lecture quizzes as those attending lectures in person through the course LMS site.The quizzes appeared at the end of the videos to encourage students to watch the entire video lecture prior to taking the quiz.Students were also asked to fill out a form on the course web site confirming that the video had been watched.
The recorded video lecture option was maintained for the remainder of the semester.Students could watch as many of the remaining lectures (12) as they wished on video or could attend f2f lectures at any time.Eight of the twelve remaining lectures were monitored with an online quiz that allowed the determination of lecture viewing choice.The other four remaining lectures were monitored with a common paper quiz because of the lecture content or the material tested.The instructor and grading structure neither encouraged, nor discouraged, the video lecture option; its election was left completely up to the individual student.
Video production.The lecture videos were recorded with the same lecturer as the f2f lecture sessions.This lecturer had taught the course many times from the same notes and, as such, was well rehearsed when the video was recorded.The lecture was given by writing on a chalkboard and recorded with a camera on a fixed tripod.The videos were presented to students in single segments of an average length of 50 minutes.Although research suggests that students learn more effectively if recorded lecture videos are broken into smaller segments (Zhang, Zhou, Briggs, & Nunamaker, 2006), single lecture-length videos were used for the semester studied to provide symmetric experiences for the f2f and recorded lecture.
To address video quality and parity with f2f lectures, the empty classroom was used during recording to reduce noise and distractions sometimes present in video that is live-captured during a large lecture class.The f2f lecture was given supported by writing on whiteboards.In the f2f lecture, each page of lecture notes from which the lecture was given was projected on a screen above the whiteboard after the material on a page had been presented.These notes were made available to all students before the lecture was given through the course LMS site.These same notes were used when recording the video lectures.
The recorded and f2f lecture were very similar with important strengths for each format.The class features careful timing between the f2f lecture and laboratory; a fixed amount of material must be covered each f2f lecture so the students have sufficient information for the upcoming laboratory.The instructor's notes, published to the students, acted as a script for each lecture and defined the information presented.As such, the f2f and recorded lectures contained nearly identical information.The camera was placed in the empty classroom in a location that gave the online viewer effectively a seat very near the front of the class.Visual aids used in the f2f lecture were also used in the recorded lecture with the proximity of the camera allowing superior viewing for online students.The recorded video was examined for both accidental misstatements and places where the board work was difficult to read and these problems corrected.This gave the online students a better viewing experience than many students in the f2f lecture theater, particularly those toward the back of the classroom.The recorded lecture had none of the distractions of a large lecture hall and included the instructor asking the questions he had come to expect from long experience teaching the class.The f2f lecture did feature the ongoing supports for engagement of instructor and student questions as well as the natural engagement of the live experience.The instructor felt the videos should be a superior learning experience for the majority of the students, particularly those with seats far from the lecturer except The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap for the reduced engagement of the recorded experience.There were no additional differences that would inform the results of this experiment.
Video quality survey.To address concerns regarding student perceptions of recorded video quality (Lauer, Muller, & Trahasch, 2004), the students were given a short survey about the general quality of the recorded videos approximately three-quarters of the way through the semester, or at the mid-point of the video option.This survey was given under the condition that individual student responses would remain confidential and not be communicated to course personnel.As an incentive for survey participation, all students received three bonus points for the time required to complete the survey.
Strengths and weakness of experimental design.The experimental design had important strengths and weaknesses.The within-subjects design allowed for the control of the differences between student groups with different self-selected levels of video use.While a randomized design where students were assigned video levels would also have allowed a similar determination of the efficacy of the online option, the self-selected design with the within-groups control permitted the identification of a significant self-selection effect.A primary weakness of the experiment was that a 2x2 design was not possible.It would have been beneficial to split the class into two groups; one group which received all f2f instruction for the first half of the semester and f2f/online instruction in the second half and a second group which receive f2f/online instruction in the first half of the semester and all f2f instruction in the second half.This design would have allowed for superior control of the semester-long evolution effects that had to be removed mathematically.However, implementing a 2x2 design would have been difficult because it would have required the removal of a popular resource from some students mid-semester.

Measures of Student Achievement
The class evaluated student learning and encouraged behaviors conducive to learning through a number of required and optional assignments.Student achievement was measured by homework grades, scores on four in-semester examinations, and the score on a standardized conceptual evaluation.
Homework grades.Outside homework was assigned each week; this homework was split into a set of multiple-choice problems whose answers were entered online through the course's LMS, and a set of open-response homework problems that the student worked on paper.The students were to complete 23 multiple-choice homework assignments and 21 open-response homework assignments over the course of the semester.The multiple-choice homework was due online at the beginning of the f2f lecture at 10:30am Monday and Wednesday.Before the introduction of the video option, the open-response paper-based homework was collected at the beginning of the f2f lecture.After the introduction of the video lecture option, the open-response homework was due at the end of the last laboratory session on Tuesday/Thursday afternoon.This change was made to provide students electing the video lecture option the flexibility of not having to attend the beginning of the f2f lecture to turn in paper-based homework.All outside homework assignments received a grade.
Examination scores.Students were given four in-semester examinations; each examination covered the readings and activities of the previous four weeks and was not comprehensive.Two of the examinations occurred prior to the introduction of the video lecture option, while the last two examinations occurred after the video lecture option was made available to students.Each examination consisted of ten multiple-choice and two open-response questions.
Conceptual evaluation scores.In addition to in-semester examinations, students' overall conceptual learning for the semester was measured with the nationally recognized Conceptual Survey of Electricity and Magnetism (CSEM) (Maloney, O'Kuma, Hieggelke, & Van Heuvelen, 2001).The test was administered as a pretest and posttest in the laboratory.This instrument features 32 conceptual questions covering the major topics in electricity and magnetism, and has been widely used to measure conceptual understanding in introductory physics classes.The pretest was given The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap during the first laboratory meeting in the first week of class, while the posttest was administered in three segments in the laboratory meetings after each of the first three in-semester examinations.

Measures of Student Engagement
Within this study, student engagement was measured through the number of lectures viewed, the rate at which assignments (optional and required) were submitted for grading, and student selfreport information regarding note taking behaviors.
Lecture viewing.Student engagement with the lecture portion of the class was measured by the number of lectures viewed, either f2f or online.Lecture viewing was monitored by a one-question quiz at the end of each lecture called a lecture quiz.The lecture quiz was administered and handrecorded by the instructor after every f2f lecture, and was administered online and graded by the LMS after students viewed recorded video lectures.
Homework submission rates.In addition to lecture viewing as a determinant of student engagement, the submission of homework assignments outside of the classroom was also examined.While homework assignment grades were previously described in relation to measures of achievement, submission rates for these out-of-class assignments were also calculated and used as a measure of student engagement.

Laboratory submission rates.
Students were required to attend 28 f2f laboratory sessions, two per class week, during the semester and received credit for completing each of 28 lab activities.The completion rate of these laboratory activities was calculated as an indicator of student engagement.
Optional assignment completion.Students were also given two optional assignments that could be completed for bonus points; each assignment required the construction of a simple physical device-a leaf electroscope and an electric motor.The election to complete either or both of these optional assignments was also used as a measure of student engagement.
Note taking.Student note taking behaviors were measured with a self-report survey item immediately after the first and third in-semester examinations.The survey question was answered in the laboratory and asked students to identify the method that best described how lecture notes for the class were obtained.As previously indicated, the instructor's lecture notes for the class were published in the class' LMS site.Therefore, the students could take their own lecture notes, use the instructor's published lecture notes, or use some combination of both methods.All students were given three bonus points for the time required for the survey.Answers to this survey item were used as a measure of student engagement.

Analyses and Results
To provide for appropriate and meaningful analyses, categories representative of the amount of student video lecture use were created.Additionally, to address potentially confounding variables associated with perceptions and lecture video quality, students were asked to self-report on the quality of recorded lecture.

Video Usage Categories
The use of the option to watch the recorded video lectures rather than attending f2f lectures was elected to differing degrees by different students.The number of students choosing different levels of video use is shown in Figure 1.As a reminder, there were eight lectures where the use of the video option was monitored.Many students made no use of the video option with 46.4% (n = 78) of the students attending only f2f lectures; these students were categorized as "Non-User" students.Some students used the option sparingly with 27.4% (n = 46) of the students watching one to two recorded lectures rather than attending f2f; these students were categorized as "Low User" students.Other students made stronger use of the video option with 26.2% (n = 44) of students replacing f2f The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap lecture with video three or more times; these students were categorized as "High User" students.These categorizations were chosen because they represented a natural division of the features of Figure 1, a strong peak at zero, a decay from this peak at video use levels one and two, then relatively constant frequency from three to eight.This choice also balanced the sizes of the Low and High User groups while leaving the groups of sufficient size for meaningful statistical analyses.

Perceived Quality of Video Lectures
The students were surveyed about the quality of the video lectures near the midpoint of the time the video option was available.Students who reported watching at least part of one video were asked to rate video quality on a four-point scale from excellent to poor.Of the 107 students who reported watching at least part of one video, 88% reported the quality of the videos was above average with only one student rating the video as poor.A chi-squared test for independence found the differences between Non-Users and users of some video (Low Users + High Users) was not significant (this test eliminated the single "poor" response because of the inflated influence of small response counts on the chi-squared test).Although video quality was reported as a substantial barrier to the use of recorded lectures in some studies (Lauer et al., 2004), results suggested that the quality of the online videos in this study was not a factor in the choice to watch the lecture on video rather than attending in person.

Achievement
Analyses of student achievement data involved multiple statistical procedures to determine if the provision of the video lecture option impacted measures of achievement for the class as a whole, and in relation to the three levels of student video lecture use (Non-, Low, and High).It should be noted that lab assignments and lecture quizzes were not used as measures of achievement within this study.Students received either full or no credit for complete/incomplete laboratory activities, and were encouraged to work together on f2f lecture quizzes; therefore scores on these assignments were not appropriate measures of individual achievement.One student, who did not take one of the insemester examinations, was eliminated from the achievement analyses.Homework grades.Although 23 multiple-choice homework assignments and 21 openresponse homework assignments were assigned, the first two and the last three homework assignments of the semester were not as mathematically or conceptually challenging as the remainder of the homework assignments.Due to the differences in rigor and focus of the beginning and ending homework assignments, these 5 specific assignments were eliminated from homework grade analyses.The mean score on assignments given before the introduction of the video policy (BV) and after its introduction (AV), as well as the change in assignment average AV-BV (), are presented in Table 1.T-tests were performed for each  in Table 1, and none of the changes in assignment averages were significant at the p = .05level.As indicated in Table 1, assignment scores were approximately constant over the course of the semester.This is contrary to a semester-long decrease in assignment submission rates which will be discussed as part of the investigation of engagement.The degree to which the level of video use affected homework grades before and after implementation of the video option was investigated by performing an analysis of variance.Analysis of variance will be used throughout this study to determine the significance of video use level as a treatment effect.The level of video use was a significant treatment effect in the open-response homework average F(2, 164) = 6.86, p < .00 and the multiple-choice homework average F(2, 164) = 3.32, p = .04before the introduction of the video policy; High-User students scored lower than Non-Users and Low Users on both types of homework assignments before the introduction of the video option.The video policy could not affect students before its introduction; therefore these differences represent pre-existing differences in homework performance for student populations electing different levels of video use before the introduction of the policy.
After the introduction of the video lecture policy, the level of video use was also a significant treatment effect in the open-response homework average F(2, 164) = 3.07, p = .05and the multiplechoice homework average F(2, 164) = 4.46, p = .01.Consistent with the BV findings, students electing a high level of video use scored lower than the other two levels on all homework assignment types after the introduction of the video option.
The level of video use was not a significant treatment effect in the change in open-response homework averages nor the multiple-choice homework average; the video policy did not have a statistically significant impact on the change in student performance on homework assignments for any group.Rather, students who chose to view a high level of recorded video lectures scored lower than students electing low or no video lecture viewing both before the video option was introduced and continued to score lower after its introduction.
Examination scores.Mean scores on examinations are also presented in Table 1.Examination averages increased slightly () for the class as a whole after the introduction of the video policy, but a t-test of the change in exam average showed the change was not statistically significant.As was found with homework grades, the introduction of the video policy had little effect on student achievement overall as measured by examination scores.
Before the introduction of the video policy, the level of video use was a significant treatment effect in the average on the first two tests F(2, 164) = 3.77, p = .03.Post hoc analyses indicated those The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap students electing a High level of video use (M = 69.5,SD = 13.4) scored significantly lower than Non-User students (M = 76.4,SD = 13.6) and lower, but not significantly lower, than Low User (M = 73.1,SD = 12.7) students on exams taken before the introduction of the video option.These differences suggest that weaker performing students before the introduction of the video policy elected higher levels of video lecture replacement.After the introduction of the video policy, an analysis of variance showed the level of video use was not a significant treatment effect for exam scores.Post-hoc analysis found no significant difference between the video use groups AV; while High User students did not surpass Non-User students on exams AV, they did close the gap sufficiently so differences were no longer significant.Table 1 also shows a consistent decrease in AV examination scores with increasing levels of video use.
The level of video use was not a significant treatment effect for the change in exam average (AV-BV), indicating that the video policy did not have a statistically significant impact on exam scores.Students who chose to view a higher level of recorded video lectures scored lower on exams before the video option was introduced; they also scored lower on exams after the video policy was introduced.This observation seems to contradict that of the previous paragraph-the High User group did have a larger change in examination average than other groups as seen in Table 1, but the degree they caught up was not sufficient to change all statistical measures of significance.The primary effect of the introduction of the video option was educationally neutral.

Conceptual evaluation scores. Conceptual learning was monitored with the Conceptual
Survey in Electricity and Magnetism (CSEM).A total of 153 students who took both the pretest and posttest were included in this analysis.Very little difference was found in either the pretest or posttest scores for students with differing levels of video use.The level of video use was not a significant treatment effect for pretest score, posttest score, or the change from pretest to posttest.These results suggest that watching the lectures on video neither damaged nor enhanced the conceptual learning of the students.

Engagement
Analyses of student engagement data examined if the provision of the video lecture option impacted measures of engagement for the class as a whole, and in relation to the three levels of student video lecture use.
Lecture viewing.Lecture viewing was measured for both f2f and video lectures through the submission of a required lecture quiz.As such, lecture viewing was analyzed by examining the submission rates of lecture quizzes.The average lecture quiz submission rates are presented in Table 2.The results were constructed by first finding an average lecture quiz submission rate for each lecture, then an overall average for all lectures BV and AV.The statistical significance of the effect of the introduction of the video policy was evaluated using a t-test on the collection of lecture quiz submission rate averages BV and AV and is presented in the  column which reports the change AV-BV.A t-test showed overall lecture viewing decreased significantly after the introduction of the video policy t(19) = -4.42,p < .00.
The submission rates of the lecture quizzes showed a pattern of continuous decline as the semester progressed.This pattern was well established before the introduction of the video option in the 9 th week of class; student lecture viewing was decreasing prior to the introduction of the video lecture option, and continued to decline for the remainder of the semester.This pattern of declining lecture attendance is evident in data for semesters previous to the studied semester where no video option was available and has been observed in other research (Van Blerkhom, 1992).To further explore this pattern of decline, a linear regression was used to extract the linear equation satisfied by the submission rates BV for lecture quizzes.The linear trend was then subtracted from the lecture quiz averages to produce a more accurate measure of the change in submission rate due to the video policy, as opposed to the naturally occurring decline.The submission rates with the linear trend removed are also presented in  OR = Open Response Homework.MC = Multiple-Choice Homework.a = significant at the p < .05level; b = significant at the p < .01level.() = rate subtracting linear trends.* = Since only one optional assignment was given BV and AV, the significance of the difference for optional assignments could not be tested.
To investigate the change in submission rates by individual students and explore potential differences in submission rates within different levels of video use, individual student averages were examined and paired t-tests performed.Lecture quiz submissions significantly decreased for all three groups of video users (see Table 3) after the introduction of the video option.The High User's lecture quiz submission rate decreased slightly less than the other groups; however, post-hoc analysis showed the differences in the changes of lecture submission rate between the High-User group and the Non-User group (t(120) = .56,p = .58)and the Low User group (t(88) = .75,p = .46)were not significant.Because of the increased variation in the individual student data, a linear trend could not be subtracted from the data.As such, some of the change observed in Table 3 may result from the semester long decrease in lecture quiz submission rates established BV.Before the introduction of the video policy, the level of video use was a significant treatment effect on the rate of lecture quiz submissions F(2, 165) = 8.43, p < .00,with those students electing a High level of video use (M = 84.4,SD = 14.6) submitting fewer lecture quizzes BV than Low (M = 93.8,SD = 11.2) or Non-User (M = 95.1,SD = 15.5)students.Video use was not a significant treatment effect AV nor was it a significant treatment effect in the change in lecture quiz submission rate.While Table 3 shows a consistent decrease in the number of lectures watched both BV and AV, t-tests examining pairs of video use groups found no significant difference between the groups either BV or AV.
Homework submission rates.Homework assignment submission rates were also used as a measure of engagement as shown in Table 2. Submissions rates did not contain a measure of the completeness or correctness of the submitted assignment.Both the open-response homework submission rate and the multiple-choice homework submission rates significantly declined after the introduction of the video policy.Homework submission rates demonstrated a weaker declining evolution than lecture viewing over the course of the semester, but a pattern of ongoing decline was The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap present.As was done with lecture quizzes, the linear trend established BV was subtracted from the data and is presented in parentheses in Table 2.These declines in submission rates changed as the linear trend established BV was removed but remained significant.
Paired t-tests were used to investigate the changes in submission rates for the class overall and within video use groups as presented in Table 3.While all groups declined in the amount of homework submitted, the only significant decline within any group was found in Non-Users, who submitted significantly fewer multiple-choice homework assignments AV than BV.
Before the introduction of the video policy, the level of video use was a significant treatment effect for the rate of homework submissions for open-response homework, F(2, 165) = 4.87, p < .00,but not multiple-choice homework, with those students electing a High level of video use (M = 85.0,SD = 15.4) submitting open-response homework at lower rates before the introduction of the video option than Low (M = 92.5,SD = 11.4) or Non-User (M = 93.0,SD = 15.1)students.After the introduction of the video lecture policy, the level of video use was not a significant treatment effect for the rate of homework submissions of open-response homework or multiple-choice homework.The level of video use was also not a significant treatment effect for either the change in openresponse homework submission rates or the multiple-choice homework submission rate for the class as a whole.Thus, results of homework submission rate analyses indicated that all students decreased slightly in the amount of homework submitted, with the only significant decline in submission rates shown by the Non-User group on multiple-choice homework.As with exam averages, Table 3 shows a generally decreasing pattern of homework submission rates with High Users submitting less homework than Non-Users both BV and AV; however, t-tests comparing pairs of video use groups were not significant.

Laboratory submission rates.
The laboratory submission rate (Table 2) was a measure of whether the students attended and completed the f2f laboratory portion of the class, a primary form of course engagement.The level of video use was a significant treatment effect BV, F(2,165) = 7.36, p < .00,with those students electing a High level of video use (M = 95.3,SD = 7.1) submitting lab assignments at lower rates before the introduction of the video option than Low (M = 98.5, SD = 3.6) or Non-User (M = 98.6,SD = 3.8) students.No significant differences were found between the groups AV.The level of video use was also not a significant treatment effect for the change in laboratory submission rates AV-BV.As with other results, there was a general but slight pattern of decline in laboratory completion with video use level, but t-tests examining pairs of video use groups were not significant.

Optional assignment completion.
A chi-square test of independence was performed to examine the relationship between submitting optional assignments and the level of video use, and found a significant relationship between these variables BV,  2 (2,168) = 6.35, p = .04,but not AV; the change in optional assignment submission rates was also not significant.For this analysis, the four possible combinations of submitting or not submitting assignments BV and AV were coded into a single four level variable.The students using High levels of video (M = 15.9%)turned in fewer optional assignments than Low (M = 39.1%) or Non-video users (M = 33.3%)BV, but increased their rate of turning in optional assignments AV to close the gap with other students, and actually turned in more optional assignments than the Non-User students AV.
Note taking.One of the many ways students interact with a lecture is by taking lecture notes.Instructor-annotated lecture slides for the class under study were published in electronic form at the beginning of the semester providing the students with many options for the taking and use of lecture notes.The students were surveyed about their note taking and use behavior after the first and third insemester examinations with the question "Circle ONE of the following that best describes how you obtain lecture notes."The responses to this question are summarized in Table 4.
Results indicated the pattern of note taking for the class overall changed with the introduction of the video policy.The difference in overall note taking behaviors after the introduction of the video policy was statistically significant  2 (4,319) = 9.66, p = .05,with students taking their own lecture The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap notes or adding to provided notes less frequently after the video option was introduced.Similarly, students printed or read the lectures at the website after a lecture, and printed and followed notes during or after lecture (without adding their own notes) more frequently after the video option was in place than before it was available.In other words, a significant number of students replaced taking their own lecture notes with printing and following printed notes during or after lectures once the video option was provided.1.9 1.9 1.3 0.0 2.3 0.0 2.5 6.8 Notes.BV = before video option.AV = after video option.
A chi-squared test for independence showed that the level of video use had a significant relationship with the way students took notes previous to the introduction of the video policy,  2 (8, N = 158) = 17.29, p = .03.Because some entries were small, Fisher's Exact Test was also used yielding p = .02.Those students classified as High video users were found to be associated with less personal note taking than Low or Non-Users BV.The students most likely to elect to watch the lecture on video were also students least likely to take their own lecture notes before the introduction of the video policy.
Level of video use was also significantly related to note taking after the introduction of the video policy,  2 (8, N = 161) = 27.98,p = .00.Similar to the previous finding, students who chose to view more video lectures were the students least likely to take their own lecture notes, and also more likely to not take or read the lecture notes from the website at all.

Discussion
This study sought to determine the relationship between student lecture choice (recorded vs. f2f) and 1) achievement and 2) forms of course engagement when students in a large introductory science class were given the option to either attend f2f lecture or watch recorded lectures online.
The lecture viewing choice was left to the student's discretion, with neither f2f nor online encouraged.Since the f2f lecture was not encouraged, this study cannot inform the discussion of whether optional recorded lectures negatively impact required f2f attendance (Gysbers et al., 2011;Hove & Corcoran, 2008;Traphagan et al., 2010;von Konsky et al., 2009), but can inform the discussion of the level of recorded lecture use that would be experienced if f2f attendance was not mandatory.
Based on the results of this study, the impact of offering a lecture choice to students in a large lecture course is minimal when considering the class as a whole; offering students the lecture option had a small negative effect on engagement, but no effect on achievement, which supports the findings The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap of Solomon et al. (2004).Those students who elected High use of the video option experienced several minor beneficial impacts on engagement and achievement, corresponding to the findings of Traphagan et al. (2010).The following sections explore these findings in more detail.

Effect of Choice on Overall Lecture Viewing
Students elected the recorded option in varying degrees; 46% of the students continued to exclusively use f2f lecture while 26% replaced three or more f2f lectures with video.Overall average lecture attendance was lower by 3.3% after the introduction of the video lecture option, after a linear trend identified BV was removed.All video use groups experienced a decline in lecture viewing with the High User group declining somewhat less than the other groups.While some of the reduction of lecture viewing is attributable to the natural decline in class attendance from the beginning to the end of a semester as identified in the classic research of Van Blerkom (1992), a statistically significant proportion of the decline resulted from the video option.While significant, the 3.3% decline attributable to the video option is small, and this additional decline did not result in lowered achievement on examinations.
While the introduction of the video option had a small, but significant, negative impact on the number of lectures viewed overall, the level of recorded videos viewed was a significant treatment effect before the introduction of the video policy, but ceased to be significant after the introduction of the policy.A smaller decline in lecture viewing by the High video users contributed to the reduction of the lecture viewing differences between the video-use groups to a non-significant level.Students self-selecting the video option were already watching fewer lectures before the option was made available and continued to watch fewer lectures after the policy was enacted; however, the decline in lecture viewing of High User students was smaller than other students.The election of the video option by less engaged students aligns with assertions by researchers (Albert, 2004;Traphagan et al., 2010) who suggested that lower performing students gravitate toward electronic or web-based instruction, but does not support the warnings from Brecht (2012) who suggested that students may adopt bad lecture viewing habits with the introduction of an online video option.Rather, this finding suggests that the video option was used by the High viewing group to reduce the differences in the number of lectures viewed between themselves and other students; thus supporting researchers such as Ealy (2013) and Gosper et al. (2010) who report on the benefits of video lectures which can afford students who need them more options and opportunities for engagement.

Effect of Choice on Other Forms of Engagement
The level of f2f lecture replacement had a similar effect on most other measures of course engagement.The level of video lecture use was a significant treatment effect for open-response homework submission rates BV, but not multiple-choice homework, with High User students submitting both assignments at lower rates than Non-video users; however, the level of video use was not significant for either assignment type AV.The High User group demonstrated lower rates of decline in the submission rate of open-response homework as compared to other groups, which contributed to closing the gap between assignment submission rates of the three video use groups.Thus, the results of this study suggest the introduction of the video lecture option did have an impact on engagement as measured by open-response homework submission rates.This effect was complicated with the class as a whole experiencing a small, but significant, decline in homework submission rates AV, but the differences in the three video use groups ceasing to be significant AV.
Students watching a High level of video actually increased the number of optional assignments completed where Non-or Low Users decreased their completion of optional assignments.This willingness to invest extra out-of-class effort was a strong indication that High video use students were still engaged in the class, and contradicted suggestions made by Phillips et al. (2007), who predicted less participation in other components of a course by students who do not attend classroom lectures.The pattern of High Users either increasing engagement or showing a lower decline in engagement was consistent across assignment types except for multiple-choice homework where the High User group showed the largest decline of any group.This decline was unusual because students were strongly penalized for not turning in assignments and the multiple-choice homework was the easiest assignment to submit.While many causes are possible, it seems likely that the combination of having the multiple-choice homework due before the f2f lecture and the structure that attending lecture imposed on Non-and Low User students conspired to create this effect.This conceivably supports the work of Phillips et al. (2007) who identify the potential for negative impacts of recorded lecture related to the removal of the fixed schedule imposed by f2f lecture.
A significant change in note taking behavior was observed with the introduction of the video policy for all students, with High User students reporting less engaged note taking behavior both BV and AV.High User students were less likely to take their own lecture notes before the introduction of the video option, further supporting the conclusion that High User students were less engaged than other students before the video option was made available.All student note taking behaviors degraded slightly as the semester progressed, but High User students showed a greater transition to less engaged behaviors by preferring to read and possibly augment published notes, a more passive and less engaged means of interacting with lecture, than taking their own lecture notes.Unlike lecture viewing and assignment submission rates, differences in note taking habits were not reduced by the flexibility, privacy, and replay features of video.This provides evidence that students' note taking interactions with video lectures were different than their note taking interactions within a f2f lecture.It should be observed, however, that despite the deterioration of personal note taking behaviors, this change did not result in a corresponding decrease in achievement as predicted by Reisberg (2000) and Phillips et al. (2007).

Effect of Choice on Achievement
Results from the current investigation on measures of achievement were similar to engagement except that the assignment averages did not show a progressive decline over the course of the semester.The change in assignment average BV to AV was not statistically significant for either the class as a whole or for any video use level; thus, the results of this study align with Solomon et al. (2004) who also found no achievement differences based on f2f versus video lecture viewing, but contradict the findings of Traphagan et al. (2010) who reported improved achievement with video lecture viewing.While changes in assignment average were not significant, the change in examination average BV to AV allowed the Low User and High User groups to catch up sufficiently that the level of video use was not a significant treatment effect AV.This effect was subtle with High Users statistically significantly weaker than Non-Users but not Low Users BV, but with no statistical differences AV.
A statistically significant self-selection effect was identified where students electing high levels of the replacement of f2f lectures with video lectures were performing more weakly in the class before the video option was made available; the level of video use was a significant treatment effect on all assignment averages before the video policy was introduced.This result is consistent with the lower levels of engagement measured for Low and High Users, and supports previous research showing weaker performing students prefer electronic educational options (Albert, 2004;Traphagan et al., 2010) and stronger performing students watch less video if given the choice (Artino, 2010;Gysbers et al., 2011).
The achievement findings also demonstrate that even though the lower performing students were more likely to use web-based lectures (von Konsky et al., 2009), the concern that these students would choose to replace live lectures with web-based lectures to their detriment (Brecht, 2012) was unsupported by this study.The results of this within-subjects study demonstrated that the preferential election of the video lecture option by weaker performing students did not further degrade their performance in the class as suggested by Brecht (2012).Rather, it seemed to slightly mitigate differences in achievement between the different video use groups.The negative effects cited for recorded lecture, the lack of social interaction, inability to ask questions, and the lack of temporal The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap structure, were insufficient to lower student achievement on examinations and open-response homework.Neither, however, did the various positive aspects cited for recorded lectures, such as temporal flexibility, lack of classroom distraction, and pause and replay (Borman, 2010;Ealy, 2013;Gysbers et al., 2011), serve to significantly improve the performance of High video use students; however, the introduction of a lecture option was sufficient to allow the High user group to close the achievement gap such that the level of video use was no longer a significant treatment effect after the introduction of the video option.
Outcomes of students electing to use either of the two modes of lecture delivery were consistent with achievement outcomes that would be expected based on student achievement prior to the provision of lecture viewing options; students who began the study as high achieving students remained high achieving, while those who began as lower achieving continued to struggle.However, the differences between the groups reduced sufficiently AV so that the level of video use was no longer a significant treatment effect.

Conclusion
The within-group research design allowed this study to provide strong support for the efficacy of video lectures, even in the most demanding of introductory science classes.This study was able to control for confounding variables found in previous video lecture research (Harmon, Alpert, & Lambrinos, 2014;Lauer et al., 2004) by ensuring high quality video, as evaluated by student surveys, and video that contained a high fidelity reproduction of the content delivered in the f2f lecture.
The instructional implications of this study show that a direct replacement of lecture with high quality video has a small negative effect on engagement but no effect on achievement overall, with a suggestion of a slight benefit to achievement and engagement for students self-selecting higher levels of use of the video option.Students self-selecting high levels of video lecture were lower achieving and less engaged before the lecture could be replaced by video and somewhat less so after the video option was made available.While the positive aspects cited for video lecture were insufficient to raise High video user achievement to the class average, some narrowing of the gap in performance was evidenced.Thus, even though the delivery of lecture over the internet was largely an instructionally neutral change to the class as a whole, it assisted lower performing students who elected higher uses of the video option to more closely resemble other students at the end of the semester on measures of performance and engagement.

Limitations
This study was performed at a single institution for a single semester.Therefore, the above conclusions may not be representative of all classes or student populations.Unfortunately, once an online option was available, it could not be temporarily discontinued to repeat the experiment.The experiment was performed in a class that had been instructed in the same format by the same instructor for many semesters and the video lectures carefully reflected the presentation delivered in the f2f lectures.As such, this study cannot inform the discussion of the effect of replacing a f2f lecture with video lectures that are either superior or inferior in content to the f2f lecture.

Figure 1 .
Figure 1.The Categorization of Student Levels of Video Use.
Notes.BV = before video option.AV = after video option. change in averages.. LQuiz = Lecture Quiz.OR = Open Response Homework.MC = Multiple-Choice Homework.a = significant at the p < .05level; b = significant at the p < .01level.
The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap course is negatively impacted by the presence of video lectures, while others indicate course engagement by students increases because video lectures allow students to stay connected to the course outside of traditional class time.Measures for student engagement within these studies vary, but in most cases include the completion of out-of-class assignments and readings.Further study is needed that includes multiple indicators of engagement to clarify the impact of the provision of recorded video lectures on student engagement within traditional lecture courses.
The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap

Table 1
Means and Changes in Averages on Measures of Achievement Overall and By Levels of Video Use.

Table 2 .
A t-test using lecture submission rates with the linear trendThe Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap removed showed lecture viewing decreased significantly, but not as significantly, after the introduction of the video policy t(19) = -2.48,p = .02.

Table 2
Class Submission Rates on Changes in Measures of Engagement Before and After the Video Option.Notes.BV = before video option.AV = after video option. change in averages.LQuiz = Lecture Quiz.

Table 3
Means and Changes in Averages on Measures of Engagement by Levels of Video Use.

Table 4
Note-taking Behaviors Reported as Percentages Overall and By Levels of Video Use.
The Impact of Online or F2F Lecture Choice on Student Achievement and Engagement in a Large Lecture-Based Science Course: Closing the Gap