The relationship between perceived learning and student attainment: Lectures and problem-based learning sessions

There has been a paradigm shift in higher education towards the utilisation of problem-based learning sessions instead of the traditional lecture format which is considered obsolete by detractors and simply convenient by its advocates. This move coincides with research highlighting that active learning strategies promote profound rather than superficial learning, encourage critical thinking and increase student attainment. However, there are limited studies quantifying the attainment of undergraduate students presented with varied pedagogical techniques, and still fewer that assess student perceptions of their own learning. Over four different teaching sessions (two lectures and two problem-based learning), n=94 students completed a 10 question questionnaire comprised of Likeart style questions assessing how advantageous they perceived the teaching strategy implemented to be to their own learning. In addition, n=64 of the students also completed short quizzes at the completion of three out of four of the sessions (one lecture, two problem-based learning sessions) assessing their knowledge of the content covered. There was no significant difference in attainment (quiz score) between the two delivery methods (p = 0.113), however there was a significant difference between individual sessions (p<0.05). Student ratings (total questionnaire score) did not differ between the two delivery methods (p=0.487) or between individual sessions (p=0.748). There was a trend for increased attainment in the problem-based sessions and a decreased student rating in these classes. Findings were in agreement with prior work and highlight the need for a varied approach to teaching in the biological sciences alongside the judicious interpretation of student feedback.


Introduction
Small group teaching and problem-based learning style sessions have gained increasing credence in recent years as a beneficial teaching strategy in higher education institutions (Jaques, 2003;Knight & Wood, 2005). Whilst small-group teaching may not always be feasible with increasing student numbers, recommendations have been made as to how active teaching methods can be utilised within a traditional lecture environment (Frederick, 1986;Ebert-May, Brewer & Allred, 1997;Walker, Cotner, Baepler & Decker, 2008). Hence, based on this evidence, lecturers are actively encouraged to decrease their reliance on lectures; now deemed an inefficient pedagogical tool (Knight& Wood, 2005;Allen & Tanner, 2005). However, it is generally accepted that due to their convenience and the established nature of the lecture format, they still endure in the higher education sector (Walker et al., 2008;Cantillon, 2003).

Problem-based learning in the biomedical sciences
Problem-based learning style sessions, particularly focused around the use of clinical case studies, have been repeatedly demonstrated to promote student learning within the biomedical sciences, Spencer & Jordan, 1999). In addition, techniques for integrating active (Van Kampen, Banahan, Kelly, McLoughlin & O'Leary, 2004), learning into large group biology classes have been proposed (Allen & Tanner, 2005). Some studies have even advocated removing lectures completely from the teaching programme (Trelstad & Raskova, 1992). Unsurprisingly, some of these methods have come under criticism from some authors. It has been proposed that whilst evidence for the promotion of profound rather than superficial student learning is present, alongside an increase in critical thinking skills (Garside, 1996;Baeten, Struyven & Dochy, 2013), there may be disadvantages from the perspective of the instructor (Dahle, Brynhildsen, Fallsberg, Rendquist & Hammar, 2002). Furthermore, some studies have now disputed the relevance of the concept that students can only concentrate for around 15 minutes in a didactic lecture style session (Wilson & Korn, 2007). Consequently, Walker et al. (2008) suggest a blended approach, integrating the strengths of both delivery styles into a large lecture course in biology.

Student perceptions of problem-based learning
One of the main hurdles associated with moving away from the lecture style format is student perception and expectation of the session that they should receive. This is particularly topical in light of recent research into the role of students as consumers (Singh, 2002). Some studies have suggested that some students may feel a level of academic entitlement (Singleton-Jackson, Jackson & Reinhart, 2010), and are thus demanding more 'value' from their education (Woodall, Hiller & Resnick, 2014). Interestingly, student feedback on active based learning sessions is varied, and sometimes negative. In a study assessing how confident they felt in their scientific knowledge, students felt less confident when delivered an active learning session as opposed to a traditional lecture (Walker et al., 2008). However, despite this students scored significantly higher in assessments presented after active based learning sessions (Walker et al., 2008). This finding was corroborated in a smaller study assessing student perceptions of small group teaching (Steinert, 2004). Students raised concerns over some aspects of the facilitation of the session, however, as students were not formally assessed, this study cannot make a valid comment on learning gains. The work of Walker et al. 2008 reiterates the current understanding that student perceptions are generally an invalid measure of attainment (Emerson & Taylor, 2004;Gremmen & Potters, 1997).
Why students lack competence in assessing their own learning is unclear, however, student feedback is highly regarded in many institutions. Despite this, uncertainty exists surrounding the benefits of student feedback questionnaires in improving teaching standards (Kember, Leung & Kwan, 2002). This includes concerns pertaining to the validity of the evaluation format (Donovan et al., 2010). However, these findings should be interpreted with caution as in an extensive study of 988 students, student ratings of teaching were still the most accurate predictor of attainment (Wilkerson, Manatt, Rogers & Maughan, 2000). Furthermore, in one study, students did concede to finding active learning beneficial (McClanahan & McClanahan, 2002) and it has been stated that students need to become familiarised with the concept before it can become a valuable experience (Bennett, Hogarth, Lubben, Campbell & Robinson, 2010). Regardless, there seems to arise a problematic situation in which students are inaccurate in their ability to assess their own learning, hence problem-based learning sessions are unfavourably rated in student feedback despite their proven ability to promote profound learning (Jaques, 2003). In addition, lecturers are being placed under constant pressure to respond to student feedback and consequently facilitate sessions that students rate highly.

Student perceptions and attainment in the biological sciences
With the exception of the aforementioned work of Walter et al. 2008, few studies have directly investigated student perceptions of their own learning and attainment in the biological sciences. Therefore, in this study students were assessed on four novel teaching sessions in anatomy and physiology and were then asked to rate the session on how much they perceived they learned and how useful they felt the teaching strategies utilised were. Two of the four sessions were delivered in a traditional lecture format with minimal student engagement and two were problem based learning sessions. As it has been previously demonstrated that first year students are most receptive to active learning techniques, level one students were selected as the sample population (Bennett et al., 2010).
The hypotheses were as follows: (1) Students will achieve higher marks on assessments presented following problem-based learning sessions and (2) students will rate lectures as more beneficial for their learning than problem-based learning sessions.

Participants
The study population were a group of first year Animal Science and Bioveterinary Science students at Hartpury College, Hartpury, Gloucestershire, UK. Students were participating in a first year Anatomy and Physiology compulsory module. Students were assumed to be entering the study with similar background knowledge of the subject matter. The study was approved by the University of the West of England (UWE) ethics committee and all students gave informed consent. Students were allowed to withdraw from the study at any time in line with the Data Protection Act (1998).

Teaching sessions
The study was performed over the course of four different compulsory timetabled teaching sessions. Students were presented with a hard copy questionnaire (including consent form) and quiz at the end of each session. Students completed the quiz first (for which they were given 10 minutes) and then the questionnaire for which they were allowed unlimited time. Students completed quizzes without access to their notes or textbooks in a classroom environment and were supervised by the instructor of the session.
Two of the sessions delivered were lectures and two were problem based learning sessions. All students attended the lecture sessions together (x+y), however, in the problem-based learning sessions the group were split into two smaller sessions (x and y). The tasks presented to these students were identical across the two groups. Due to variation in attendance over the course of the sessions, the sample size varied. Over the course of all sessions n=94 questionnaires were completed (A = 23, B = 24, C = 28, D = 19) and n=64 quizzes (A = 23, B = 24, D = 17). Quiz data were omitted from session C due to students not completing the task as instructed, hence the data were not considered to be valid for inclusion in the study. Two students submitted blank quizzes in session D. It was not deemed necessary for students to have participated in the study for all four sessions for their data to be included.
In the lectures, a Microsoft PowerPoint (Microsoft, Redmond, USA) presentation was used as the main teaching aid and the majority of the session was delivered to students in a didactic fashion. Sessions lasted two hours with a ten minute break in the middle. Lectures were focused on basic veterinary anatomy and physiology, for example, the endocrine system. Problem based learning sessions consisted of the students being set a task at the beginning of the session verbally by the instructor and then students had the remainder of the session (one hour) to work in small groups to complete the task. Some tasks involved additional materials, for example, dissection specimens.

Questionnaire design
The questionnaire consisted of 10 statements with Likert scale answers (strongly disagree to strongly agree, with seven intermediary options). These statements were adapted from similar studies of student perceptions of their own learning (O'Malley et al. 1999;Stepp-Greaney, 2002;Klegeris and Hurren, 2011). These statements were designed to gauge how beneficial students found the session and how much they perceived that they had learned from it. Questionnaires were designed to be quick to complete and easily understood by the respondents.

Quiz design
The quizzes consisted of multiple choice or short answer questions. All questions were worth 1 mark and the quizzes had a total of 15 available marks. Questions were designed to cover the material presented in the session and a couple of harder questions were included to see if students could draw inferences from what they had learned to answer more challenging questions. On completion of the quizzes students were informed of the correct answers; however, the quizzes were collected in and marked by the lecturer. Students had previously been presented with these formative tests at the end of sessions so this was not a novel concept.

Quiz and questionnaire responses
Quiz scores were calculated as a percentage and entered into Microsoft Excel (Microsoft, Redmond, USA). The questionnaire data were coded (1-7). A total score for the questionnaire was calculated to give a value (out of a possible total of 70). Students were asked to identify their questionnaire and quiz by their student number (to keep responses anonymous in line with the Data Protection Act), however many omitted this information.

Statistical analyses
All statistical analyses and graphical representations (package = "ggplot2") were performed in the statistical software R (Version 3.0.3, R Core Team, Vienna). Data were tested for normality (Q-Q Plots) and deemed to be parametric. Due to the unequal samples for the two different delivery methods (omission of session C quiz data and fluctuation in student participation) and lack of data ID (student number) for some cases, an independent samples t-test was performed. The t-test was performed on the test score (%) and session type data (lecture/problem based learning) to test for a difference in the mean scores between the two delivery methods. A further t-test was run on the total questionnaire score and session type data to test for a difference in total questionnaire score between the two different delivery methods. A one-way ANOVA was performed to test for an effect of group (x, y and x+y) on both the quiz score and questionnaire response data. An additional ANOVA was performed to test for an effect of the individual session (A,B,C,D) on questionnaire response and test result and a Tukeys post-hoc test was applied to this data to identify significant differences between individual sessions.

Results
There was some considerable variability in the test scores achieved by the student group across all sessions. However, students did present with a high mean percentage of 71.46. Similarly, there was some distinct variation in how students rated the sessions (in some instances, within the same session; see Figure 4-1). The mean questionnaire score out of a maximum of 70 was 57.91.

Effect of group
There was no significant effect of group (A, B and A+B) on either the questionnaire score (p = 0.143) or the test score (p = 0.116).

Questionnaire results
There was no significant difference in questionnaire total score between the two different delivery methods (p = 0.487). However, there was a trend for students to rate the problem based learning sessions slightly lower than the lectures. There was no significant difference in questionnaire total score between individual sessions (p = 0.748; see Fig 1.). For most questions students agreed that the session was beneficial to their learning.

Test results
There was no significant difference in quiz score (%) between the two different delivery methods (p = 0.1134). However, there was a trend for students to achieve higher marks in the problem based learning sessions (Fig 1), but there was a high level of variability in test scores across all sessions. There was a significant effect of individual session on the quiz results (p<0.05), with significant differences identified between session A and D (p<0.01) and B and D (p<0.05), see

Discussion and Conclusion
This study set out to investigate whether students rate sessions in which they attain more highly as more beneficial for their learning. The literature currently suggests that this is not the case, with students consistently lacking the ability to accurately rate their learning within a session (Gremmen & Potters, 1997;Emerson & Taylor, 2004). Whilst there was not a significant difference observed between student achievement in the problem-based learning and lecture style sessions, there was a trend for students to achieve more highly in the active learning sessions. A larger sample size and greater range of sessions examined might help to prove the hypothesis that this style of teaching leads to increased learning gains. This finding is also consistent with that of previous studies (Knight & Wood, 2005;Walker et al., 2008).
The lack of significance seen in this investigation could be due to the variability in marks achieved across the two delivery modes (and in some instances within sessions). This could be due to differences in ability between students (their general grade profile was not taken into consideration); however, it might be a result of preferential differences between students for a particular learning approach (Chamorro-Premuzic, Furnham, Christopher, Garwood & Neil Martin, 2008, Chamorro-Premuzic, Furnham & Lewis, 2007. It has also been shown that students with lower grade profiles benefit more from active learning techniques which could in itself have been a confounding factor (Walker et al., 2008). The difficulty in controlling for prior knowledge is a common methodological limitation of pedagogical studies. Interestingly, while there was no observed effect of delivery method on attainment, there was a significant effect of the individual session.
A further consideration is that problem based learning sessions can vary considerably in nature, and are highly dependent on the facilitation of the session (Jaques, 2003). In the biomedical sciences, investigations have been performed into a variety of sessions from computer based exercises to clinical reasoning (Popil, 2011, Trelstad & Raskova, 1992. The problem-based sessions utilised within this study were not of the same format and this may have resulted in the differences seen. Standardising the protocol might be of benefit in future investigations, however, it has been highlighted in previous studies that over-use of one teaching strategy is not considered advantageous (Steinert, 2004).
Generally, student feedback on all the sessions was overwhelmingly positive with students rating most of the statements as 6 or 7 indicating that they agreed the session utilised effective strategies and was beneficial for their own learning. There was no significant difference in total questionnaire score between the two different delivery modes or across individual sessions. Therefore, unlike in previous studies, students perceived lectures and problem-based learning sessions to be comparable in terms of learning gain. This disproves the hypothesis that students would rate lecture sessions more highly than problem based sessions which they would perceive to be less useful. However, there was a slight trend for the problem based sessions to be rated lower than the lectures, in contrast to the students' attainment. This reiterates the current understanding that students are not particularly adept at rating their own learning (Emerson & Taylor, 2004;Gremmen & Potters, 1997;Walker et al., 2008).
Interestingly, despite there not being an option for students to provide additional comments within the questionnaire, an approach implemented in some literature, certain students felt the need to justify their responses. Consequently, it has been noted that in future work it may be beneficial to provide a section for qualitative data to be collected. The comments mostly consisted of statements similar to 'this was not the fault of the session' when providing a low score for questions three and eight. This may have been as students felt they did not engage in the session appropriately, which has unsurprisingly been linked to the likelihood of students achieving the learning outcomes of a session (Carini et al., 2006). However, an alternative explanation would be that students lacked confidence in their own ability and thus blamed themselves rather than the teaching strategy for their inability to achieve the desired outcomes. This seems a feasible proposition as multiple studies have reported a lack of confidence in first year undergraduate students (Besterfield-Sacre, Amaya, Y., Shuman, Atman & Porter, 1998;Robson, Francis & Read, 2004).
There were some limitations of this study which should be addressed in future work. Whilst students were supervised in their completion of the quizzes, they were not performed under controlled examination conditions; hence the marks may not have been completely indicative of individual ability. Furthermore, there was unfortunately a missing dataset of quiz scores for session C which might have impacted on the significance of the findings. Evidently, a more extensive study of more students and across a wider range of sessions would result in a more reliable outcome. However, the sample size was comparable to previous literature in terms of the number of responses gained (Walker et al., 2008). This study highlights the need to continue with innovative teaching practice and brings in to question the role of the lecture in teaching in higher education. These findings also reiterate the need to interpret student feedback on teaching practice prudently and in light of the pedagogical literature.