Assessing Scientific Literacy Skill Perceptions and Practical Capabilities in Fourth Year Undergraduate Biological Science Students

In a fourth year undergraduate nutritional toxicology course that included an instructional emphasis on scientific literature critique activities and assessments, we determined the change in students’ (n=144) scientific literacy (SL) skills. The change in students’ perceived and practical SL skills were determined by the completion of two surveys, administered at the start and end of the semester. Additionally, we conducted a follow-up SL survey at the end of the subsequent academic semester (i.e., four months later) to determine if students retained any improvements in their SL skills. Over the semester, students showed improvements in their perceived capabilities of all SL skill parameters assessed (P<0.05); however, the most significant gains were apparent in the areas of i) knowledge application (specifically identifying novel problems or research questions and using new information to address unfamiliar problems or knowledge gaps), and ii) knowledge translation and communication (translating complex information from the scientific literature into clear and understandable terms). There was no change in students TOSLS score between the start and end of the semester (P>0.05). In the follow-up SL survey students showed further improvements in their perceptions of the SL skills for 7 of the 10 parameters assessed compared to the end of the previous semester (P<0.05), however, there remained no change in their practical SL skills assessed using TOSLS. Collectively, this data demonstrates that students’ perceptions of their SL capabilities may not align with their practical capabilities.


Introduction
Following the completion of an undergraduate degree in biological science, it is expected that graduates exhibit strong transferable skills in collaboration, critical thinking and problem-solving to ensure employment attainment and retention (Newton, Bettger, Buchholz, Kulak, & Racey, 2015;Wensing & Grol, 2019;Sibley, Roche, Bell, Temple & Wittmeier, 2017). To support these expectations, undergraduate teaching strategies should directly stimulate high-level skill development while simultaneously meeting course learning objectives . Scientific literacy (SL) is a crucial skill that underlies critical thinking and problem solving skills and is considered an essential component of science education at a post-secondary level (Rahayu, 2017;De Boer, 2000;Norris & Phillips, 2003;Laugksch, 2000). Providing a universal definition of SL is complicated as many skills and activities fall under the domain of SL (De Boer, 2000;Norris & Phillips, 2003;Laugksch, 2000). However, SL is commonly recognized as a developed familiarity with the practice of science, beyond a basic understanding of facts and figures, that can be utilized to make informed decisions in the real-world (National Research Council, 1996; Committee on Science Literacy and Public Perception of Science, 2016). SL is multi-dimensional and involves several components that include, but are not limited to, the ability to i) distinguish between scientific sources (NRC, 1996;Council of Ministers of Education, 1997;Mayer, 1997), ii) independently learn and conceptually understand science (Sutman, 1996), iii) read, evaluate and analyze scientific literature (NRC, 1996;De Boer, 2000;Millar & Osborne, 1998), iv) utilize acquired scientific knowledge in problem solving (American Association for the Advancement of Science, 1993;NRC, 1996), v) extract, interpret and integrate scientific reports, tables and graphs (Anderson, 1999;Phillips, 2002), and vi) critically think about scientific literature to draw evidence-based conclusions (De Boer, 2000;Korpan, Bisanz, Bisanz, & Henderson, 1997;Shamos, 1995; Organisation for Economic Co-operation and Development, 2003). Therefore, literacy, in its most foundational form (i.e. reading and writing) (Norris & Phillips, 2003), in addition to knowledge attainment, comprehension and integration (Klucevsek, 2017) are cross-disciplinary skills central to SL.
The nature and depth of one's individual SL skills are directly related to both the social and educational systems in which they function (CSLPPS, 2016). Without intentional institutional integration of SL skill development into the course curriculum, SL attainment becomes students' individual responsibility (CSLPPS, 2016) and SL competency is often inadequately and indirectly acquired as a by-product of scientific learning, which is difficult to thoroughly assess beyond knowledge of course content and scientific principle evaluations (CSLPPS, 2016;NRC, 1996;Gormally, Brickman & Lutz, 2012). Therefore, it is essential that students can confidently define skillsets that are important in the workplace, such as SL skills, and critically evaluate their relative competency (Yusof, 2012). Success in the development of foundational skills propagates further development (Porter et al., 2010), but requires an accurate perception of individual ability (Yusof, 2012). Therefore, conducting self-assessments of SL skill attainment represents a useful reflective activity for students to conduct throughout their undergraduate program or in pursuit of employment or continued education. In this connection, SL skill self-assessments represent a way of bridging the gap in the acquisition of SL skills between the students' responsibility and the learning environment. Acquiring strong SL skills requires knowledge attainment, evaluation and application (Klucevsek, 2017;Grant & Lapp, 2011); therefore, SL should be a core objective within science coursework at the undergraduate level (Meinwald & Hildebrand, 2010). A learning environment that promotes the development of SL skills, through strategic implementation, and SL skill application, through intentional utilization, provides students with transferable skills that are necessary post-graduation (CSLPPS, 2016;Lapp & Fisher, 2010). Thus, determining the effectiveness of teaching strategies that aim to develop SL skills are critical for creating students that are prepared for either the workplace or continued education (Porter et al., 2010).
Previous research has provided evidence that self-cognizant approaches to teaching and learning, in parallel with self-directed learning assignments, promote higher forms of thinking and knowledge retention, compared to didactic approaches (Borich, 2004). Similarly, a shift from passive to active learning, through diverse teaching activities and assessments at the undergraduate level has been shown to result in a deeper student understanding of course concepts, with simultaneous development of problem solving, critical thinking and SL skills, in particular the retrieval and utilization of scientific information (Salama & Chennaoui, 2016). The integration of literature assessment activities into the development of overall SL skills provides a logical progression of skill development (Krajcik & Sutherland, 2010;Barrows & Tamblyn, 1980). The identified critical components of SL that are encompassed within literature assessment activities include the evaluation of scientific information to assess the credibility and relevance of study results and conclusions, and effective knowledge translation and communication of scientific information (Glick & Greenberg, 2017). Further, problem-based learning (Hussa, 2018;Barrows & Tamblyn, 1980) and case-based learning (Davies, 2004;Harman et al., 2014) have been demonstrated to promote SL; however, these approaches do not necessarily emphasize the critical evaluation of scientific studies and subsequent assessment of study results and conclusions, which are key SL skill components. Therefore, we implemented a scaffolded teaching strategy utilizing both passive and active learning approaches with instructional emphasis on scientific literature critique and assessment to promote the development of key elements of SL in a fourth-year undergraduate nutritional toxicology course. Consequently, we assessed students' approaches to understanding the scientific literature as well as their perceptions of their SL capabilities and their practical SL skill competency. Additionally, we conducted a follow-up SL skill assessment at the end of the subsequent academic semester to determine their degree of SL skill retention.

Participants
Participants were undergraduate students enrolled in the course Toxicology, Nutrition and Food (NUTR*4510) at the University of Guelph in the Fall 2019 semester. The course was presented in a traditional 12-week face-to-face lecture style, with assessments (represented as % of final grade) consisting of individual literature critique assignments (2 x 2.5% each), a group literature critique paper (25%), a midterm exam (35%) and a final exam (35%). Of the 176 students originally enrolled in the course, 144 students completed both Survey 1 and Survey 2 (described below), resulting in 82% participation. The year of study in which participants were enrolled in during the Fall 2019 semester is shown in Figure 1, wherein 79% were in their fourth and final year of their program. All participating students gave informed consent to participate in the study and the project was approved by the University of Guelph Research Ethics Board (REB#19-07-003).

Literature Critique Activities
Throughout the semester, students engaged in a series of scientific literature critique activities in a scaffolded manner including i) instructor-led discussions of assigned studies during lecture, ii) break-out group discussions during lecture, followed by instructor-led discussion, iii) individual literature critique assignments, iv) a group literature critique written assignment, and v) literature critique questions on formal examinations.

SL Surveys
At the start of the academic semester (week 1, Survey 1) and again at the end of the semester (week 12, Survey 2), students were invited to complete an online survey that consisted of i) their perceptions and approaches towards assessing the scientific literature (non-validated researcher generated questions), ii) a self-assessment of their perceptions of their individual SL skill competency that was developed by the research team and published previously (Monk & Newton, 2018), and iii) a practical SL skill assessment using the validated Test of Scientific Literacy (TOSLS) survey (Gormally et al., 2012). TOSLS has been shown to readily access students' SL learning gains resulting from educational efforts and teaching strategies implemented at the undergraduate level (Gormally et al., 2012). The start of semester and end of semester surveys were identical, with the exception of the TOSLS questions, wherein 17 of the 28 questions that comprise the TOSLS survey were randomly selected for inclusion in Survey 1 or 2 (within each of the 9 TOSLS skill categories) to avoid testing bias. Students were asked to answer each question on their approach to scientific literature assessment and perception of their SL skills using a scale of 0 through 10, that progressed as follows: 0 "strongly disagree", 1-2 "disagree", 3-4 "somewhat disagree", 5 "neither agree nor disagree", 6-7 "somewhat agree", 8-9 "agree" and 10 "strongly agree". Students were invited by email (using a private invitation link) to complete the online surveys at the start of the semester (week 1; Survey 1), and again at the end of the semester (week 12; Survey 2), wherein the change over time reflect short-term changes in SL competency. The surveys were administered using the Qualtrics Insight Platform (Provo, UT, USA). Only students who completed both the start of semester and end of semester surveys (n=144) were included in the analysis, which centred on the change in scores of each question over the course of the academic semester. As an incentive for participation, students received a 2% bonus on their midterm exam grade for completing Survey 1 and a 2% bonus on their final exam grade for completing Survey 2.
To determine the longer-term retention of any improvements in students' SL skills, former NUTR*4510 students from the Fall 2019 semester who completed both Survey 1 and Survey 2 (n=144) were invited by email to complete the Follow-up SL survey (Survey 3) during week 12 of the Winter 2020 semester, approximately 4 months after the completion of Survey 2 (week 12 of the Fall 2019 semester). Figure 2 shows the timeline of the surveys administered in this project. Survey 3 was completed online, hosted in Qualtrics and assessed both students' perceptions of their SL skill competency (Monk & Newton, 2018) and their practical SL skills using the same TOSLS questions that appeared on Survey 1 (Gormally et al., 2012). A total of n=68 students completed the Follow-up SL survey in addition to Survey 1 and 2, reflective of 47% participation. Within this smaller cohort of students, 93% of them were in semester 8, and thus, will have completed their undergraduate program. As an incentive for participating in the Follow-up SL survey, students were entered into a draw to be randomly selected to win a $25 iTunes gift card. There were 12 gift cards awarded and the odds of being randomly selected from the participant pool that completed the Follow-up SL survey was one in five.

Statistics
All data were analyzed using Graphpad Prism 8.2.0 (GraphPad Software, Inc., La Jolla, CA, USA). For all data, the predetermined upper limit of probability for statistical significance was P<0.05. Values are expressed as means ± SEM. The assumption of normality was assessed using the Shapiro-Wilk test. Paired t-tests were performed to determine the changes across time (i.e. Survey 1 versus Survey 2) in response to students' scientific literature assessment approaches, SL skill perceptions and TOSLS score. Paired t-tests were also conducted to determine differences in SL skill perceptions between the end of semester survey (Survey 2) and the Follow-up SL survey (Survey 3). A one-way ANOVA followed by Tukey's studentized range test was used to assess the change in students' TOSLS responses over time (i.e. between Survey 1, Survey 2 and Survey 3). Correlation analyses were conducted using least squares regression to determine the association between students' self-assessed SL skill competency and their practical SL skills (i.e. TOSLS score).

Changes in Students' Perceptions and Approaches to Assessing Scientific Literature between the Start and End of the Semester
The results from the start (Survey 1) and end (Survey 2) of the semester surveys that provided a self-assessment of students' perceptions and approaches to interpreting the scientific literature are shown in Table 1. With respect to students' perceptions of their abilities to critically assess research study designs and identify study design strengths and weaknesses, all were significantly increased between the start and end of the semester (i.e. 12 weeks later; P<0.05). Despite the aforementioned improvements in students' perceived comprehension of research study designs, there was no change during the semester in their reported ability to understand the experimental methods that are used in scientific studies. Students demonstrated improvement in their perceived ability to think critically about the overall content in a scientific paper (P<0.05). With respect to their comprehension of components of a scientific paper, students also reported improvements in their ability to interpret results and understand the content of the discussion (P<0.05). They reported improved changes in the way they access information from the scientific literature, indicating an attitude change away from only reading the abstract in favour of a more rigorous approach (i.e. reading the entire scientific paper) (P<0.05). Importantly, students reported an increased agreement with the approach to draw their own conclusions with respect to a study's findings (P<0.05), indicative of improvements in their abilities to interpret the findings from primary scientific literature. Data are presented as means (SEM). (*) denotes statistically significant differences (P≤0.05) between the mean in students responses during the academic semester (end of the semester score -start of the semester score). The survey scale was from 1-10, wherein 1 indicated the lowest level of agreement and 10 indicated the highest level of agreement

Changes in Students SL Perceptions between the Start and End of the Semester
The results from the start (Survey 1) and end (Survey 2) of the semester surveys of students' perceptions of of their capabilities for each of the 10 SL skills assessed (P<0.05). The greatest gains in students' perceived SL skill competency was in their ability to translate complex knowledge obtained from the scientific literature into clear and understandable terms (Question 10) and their ability to use information obtained from the scientific literature to i) address an unfamiliar problem (Question 8), and ii) identify a novel problem or research question (Question 9). In contrast, the SL skill that students reported the least improvement in during the academic semester (although still significantly improving) was in their ability to interpret data from the scientific literature that was presented as a figure (e.g. bar graphs, line graphs or pie charts) (Question 7). Finally, the overall score (sum of all perceived SL skill outcomes combined) significantly improved (i.e. increased) during the academic semester. Data are presented as means (SEM). A maximum score of 10 was possible for each of the 10 questions that constitute the SL skill perceptions scores. A maximum overall score of 100 was possible for both the start of semester and end of semester surveys. (*) denotes statistically significant differences (P≤0.05) between the mean change in SL during the fall 2019 academic semester (end of the semester score -start of the semester score). The survey scale was from 1-10, wherein 1 indicated the lowest level of agreement and 10 indicated the highest level of agreement.

Changes in Students' Practical SL Skills between the Start and End of the Semester using TOSLS
A practical assessment of students' SL skills was assessed using the TOSLS validated survey (Gormally et al., 2012) administered at the start and end of the academic semester (i.e. in Survey 1 and 2, respectively). The average TOSLS scores (expressed as the % of correct answers to the TOSLS questions) at each time point are shown in Figure 3. At the start of the semester, the average TOSLS score was 76.4% correct (13 out of 17 questions correct), whereas at the end of the semester, the average TOSLS score was 70.6% (12 out of 17 questions correct). Although the difference in the average TOSLS score between the start and end of the semester differed by only one correct question (i.e. 13 correct at the start of the semester versus 12 correct answers at the end of the semester), this modest decrease in the average TOSLS score was statistically significant (P<0.05). Values are means ± SEM. Data was analyzed by a paired t-test and bars marked with an asterisk (*) denotes a statistically significant difference (P<0.05).
Additionally, we conducted correlation analyses to determine if there was an association between students' overall SL perceptions and their TOSLS score at both the start and end of the semester. There was a very weak positive association between students' perceptions of and their practical SL skills at the start of the semester (r=0.16, P=0.033; Figure 4A), whereas at the end of the semester, there was a stronger, although still moderate positive association between students' perceptions of and practical SL skills (r=0.41, P<0.001; Figure 4B). The trend line (of best fit) was assigned using least squares regression and is represented by the following formulas: A) Overall SL perception score = (0.1397 x semester start TOSLS score) + 59.87, and B) Overall SL perception score = (0.2874 x semester end TOSLS score) + 56.61.

Longer-Term Retention of SL Perceptions and Practical Skills
Since students' perceptions of their SL skill competency was improved in all of the 10 categories assessed during the Fall 2019 semester ( Table 2), we determined if these gains in SL skills were retained by the end of the following semester (i.e. 4 months later) in the Follow-up survey (Survey 3). The results comparing students' perceptions of their SL skill competency between the end of the Fall semester (Survey 2) and end of the subsequent Winter semester (Survey 3) are shown in Figure 5. All 10 SL skill parameters assessed were increased at the end of the Winter semester with 7 of the 10 reaching statistical significance (P<0.05), reflective of not only a retention in students' perceptions of their SL skills from the previous semester but also further improvement since the previous assessment in Survey 2. Importantly, students' perceptions of their SL skills did not decrease and the overall SL perceptions score significantly increased from 79.74±1.10 at the end of the Fall semester to 84.07±1.12 at the end of the subsequent Winter semester (P=0.003). This reflects students' perceptions of their overall SL skill capabilities at the time of graduation from their undergraduate program for 93% of survey respondents. Importantly, the SL skills wherein students demonstrated additional improvement in their perceptions of competency included their ability to i) distinguish between sources of scientific literature (Question 1), ii) integrate information from multiple sources (Question 5), iii) interpret data presented in a table (Question 6) and graphical figure (Question 7), iv) use information from the scientific literature to address an unfamiliar problem (Question 8) and identify a novel problem or research question (Question 9), and v) translate complex knowledge obtained from the scientific literature into clear and understandable terms (Question 10). Data are presented as means ± SEM (n=68). Data was analyzed by a paired t-test and for each SL skill perception question assessed bars marked with an asterisk (*) denotes a statistically significant difference (P<0.05).
Next, we determined if students' practical SL skills (assessed using TOSLS) changed over time between the start (Survey 1) and end (Survey 2) of the Fall 2019 semester and the end of the subsequent Winter 2020 semester (Survey 3). Within this smaller cohort of students that completed all three surveys there was no difference in the overall TOSLS score between any of the survey time points (P=0.19, Figure 6A). At the end of the Winter semester, there was no association between students' overall SL skill perceptions and overall TOSLS score (P=0.43, r = -0.14; Figure 6B). Values are means ± SEM (n=68). Data was analyzed by one-way ANOVA followed by Tukey's studentized range test. B) Scatterplot depicting the relationship between students' overall SL perceptions score (sum of all 10 SL perceptions questions combined out of 100) and TOSLS score (% of correct answers) assessed in at the end of the Winter 2020 semester. The trend line (of best fit) was assigned using least squares regression and is represented by the following formula: overall SL perception score = (-0.075 x semester start TOSLS score) + 90.95.

Discussion
The current study assessed the changes in perceptions of and practical SL skills of biological science undergraduate students in a fourth year nutritional toxicology course at the beginning and end of the semester (i.e. short-term changes), as well as four months after completion of the course (i.e. longer-term changes) in which scaffolded SL activities were completed. Over the course of the semester (short-term changes), students' demonstrated improvements in both their approaches to critically assessing research studies and their perceptions of their SL skills, although there was no change in their practical SL skills, assessed using TOSLS. The longer-term assessment of changes in SL skill perceptions demonstrated continued improvement, whereas practical skills remained stable. Combined, these data provide insight into both the effectiveness of the scaffolded SL instructional approach and the development of SL skills, which are an essential component of science education (Rahayu, 2017;De Boer, 2000;Norris & Phillips, 2003;Laugksch, 2000).
From an instructional perspective, multiple scientific literature critique activities and assessments throughout the semester were conducted with the aim to promote the development of SL skills and critical thinking about scientific study designs including identification of study strengths and weaknesses, analysis of data and interpretation of results. These activities were conducted in a scaffolded manner that included instructor-led discussions of scientific study designs and analysis of study strengths and weaknesses that impact the interpretation of the results, to demonstrate literature critique approaches. This type of passive learning was followed by utilization of active learning activities during lecture time, in the form of break-out group discussions (Lougheed, Kirkland & Newton, 2012) of assigned scientific studies, followed by summary activities led by the instructor to consolidate break-out group discussion outcomes. Additionally, outside of lecture, students were assessed on i) their written summaries of two separate scientific literature critiques that were completed individually and ii) a written literature critique completed in groups wherein students critiqued the study design and consolidated the findings from multiple papers on a related topic of choice, requiring students to find and interpret the research findings. Therefore, with multiple SL skill building activities embedded into the course curriculum, students were regularly engaging with the scientific literature throughout the academic semester. Additionally, there was an instructional emphasis on assessing the quality of scientific evidence with formal instruction and discussion of concepts including, but not restricted to, Bradford Hills Criteria (i.e., Hill's criteria for causation) (Hill, 1965), hierarchy of scientific evidence (Canadian Task Force on the Periodic Health Examination, 1979;Petrisor & Bhandari, 2007) and evidence-based interpretations of information (Engebretsen, Vollestad, Wahl, Robinson & Heggen, 2015;Lonergan, 1992). Interestingly, in the assessment of students' literature critique capabilities on short answer examination questions that required students to identify the study design strengths and limitations, interpret results and apply their knowledge in designing a subsequent research study, there was a small but statistically significant 3% improvement in their performance between the midterm and final exam, thereby demonstrating that their perceived improvements in these skills aligned with a practical assessment.
Over the course of the academic semester, students reported changes in their approaches to interacting with scientific literature including reducing their reliance on the content in the abstract and reading the content of the entire paper instead, improving their ability to interpret results, understand the content in the discussion section and draw their own independent conclusions about the study results (Table 1). With respect to students' comprehension of research study designs, their confidence in their ability to critically assess the validity of research study designs and identify strengths and weaknesses of various study designs was improved. Collectively, these improvements coincided with an improvement of students' perceptions of their ability to think critically about the content of a scientific paper, a key element in their overall SL capabilities, as critical thinking is a component of SL (Rahayu, 2017). Additionally, in the assessment of students' literature critique capabilities in written answer questions on the final exam, performance was significantly improved compared to their performance on the midterm exam. Despite students' increased confidence in their ability to think critically about research study designs, there was no change in their level of difficulty associated with comprehending the study methods utilized. This highlights a potential knowledge gap in students' understanding of research methodologies and analytical and/or biochemical measurements related to the field of nutritional toxicology. Exposure of undergraduate students to laboratory research has been shown to increase their confidence in working independently and problem solving (Harrison, Dunbar, Ratmansky, Boyd, & Lopatto, 2011), while simultaneously gaining practical lab skills (Kremmer & Bringle, 1990;Kardash, 2000;Seymour, 2004) and their scientific knowledge in the form of understanding of experimental procedures and methodologies (Ryder, Leach & Driver, 1999). However, with larger class sizes, research opportunities can be limited (Hunter, 2007), and a lack of inquiry-based lab programing that parallels traditional lecturing appears to create a gap in students' understanding of research study design, data analysis (Yu & Kuo, 2017) and effective communication of experimental results (Indorf, Weremijewicz, Janos, & Gaines, 2019). Therefore, increasing students' exposure to and comprehension of research methodologies could improve their understanding of the scientific literature, although further study is required.
With respect to the development of students' SL skills over the course of the academic semester, there were improvements in students' perceptions of their SL capabilities for all parameters assessed, ranging from categories such as accessing and identifying types of scientific literature, to interpreting results and applying their knowledge in novel scenarios (Table 2). Interestingly, despite gains in students' confidence in their SL capabilities, there was no change in their overall performance on a practical skill assessment using TOSLS (Gormally et al., 2012). In fact, the average TOSLS score actually decreased between the start and end of the semester, albeit by a modest reduction in the average score of correct answers by one question (13 correct at the start of the semester and 12 correct questions at the end of the semester) (Figure 3). Students' self-assessment of their SL skills was weakly positively correlated with their TOSLS score at the start of the semester and the strength of this positive association increased at the end of the semester (Figure 4). It should be noted that TOSLS represents one validated approach to assessing students' practical SL capabilities, yet SL encompasses multiple components (NRC, 1996;Council of Ministers of Education, 1997;Mayer, 1997;Sutman, 1996;De Boer, 2000;Millar & Osborne, 1998; American Association for the Advancement of Science, 1993;Anderson, 1999;Phillips, 2002;Korpan, Bisanz, Bisanz, & Henderson, 1997;Shamos, 1995;Organisation for Economic Co-operation and Development, 2003) and it is difficult to fully capture the breadth of an individual's SL capabilities through one assessment mechanism. There were some limitations in the use of TOSLS. Despite being a useful tool to determine the effectiveness of a teaching approach to promote SL and/or determine changes in students SL practical skills over an academic semester, some of the SL skills assessed within TOSLS may take a longer amount of time to develop and one semester may be an insufficient duration to see changes (Gormally et al., 2012). In an attempt to reduce the overall length of the start and end of semester surveys to limit the potential effect of survey fatigue (Sharp & Frankel, 1983;Porter, Whitcomb & Weitzer, 2004) and to avoid recall bias by repeating TOSLS questions, we reduced the length of the TOSLS survey from 28 to 17 questions. The TOSLS survey is comprised of 9 categories of individual SL skills, and although all 9 categories were equally represented on both the start and end of semester surveys, the individual TOSLS questions that appeared on each survey administered in this project differed. Therefore, the outcomes from this study may have been affected by a potential difference in the level of difficulty of the TOSLS questions between surveys and/or by the fact that the entire overall TOSLS scale was truncated. Conversely, reducing the length of the TOSLS survey likely helped increase the percentage of students participating in the study, therefore, researchers utilizing longer tests such as TOSLS have to balance concerns for survey fatigue with survey attrition. Future studies utilizing this validated survey would benefit from the use of the all TOSLS questions despite the potential for recall bias, which could still impact the findings. Despite this, students' overall performance on the TOSLS questions demonstrated a level of SL skill competency that would be expected of fourth year undergraduate students (76.4% correct answers at the start of semester and 70.6% at the end of semester survey), indicating that this validated survey is at an appropriate level of difficulty for fourth year undergraduate students.
Interestingly, despite these improvements in students' confidence in their SL capabilities, there was no change in their overall TOSLS score between any time point assessed (Figure 6), thereby indicating that students' SL capabilities did not change during their fourth year of their undergraduate program. It should be noted that a potential limitation was the smaller sample size for Survey 3, with only 47% participation as this survey overlapped with the COVID-19 emergency transfer to online classes. At the University of Guelph, the first year biology curriculum is delivered with an emphasis on evidenced-based approaches that encourage higher-order learning, deeper levels of cognition and the development of scientific skills and attributes, including SL, through the use of problem-based learning (Husband et al., 2015;Murrant et al., 2015). These first-year courses include formal lectures, seminars, practical labs and integrative group work assignments, a structure that is intensive for the development of SL skills. Conversely, as students progress through their four-year undergraduate program, these types of educational experiences may not be as frequent and subsequently the opportunities for developing SL skills may be limited as courses become more focussed on the academic content and less focussed on skill development. Further study is required to map these types of educational experiences across an academic program and assess students' practical skill acquisition over time. Although there was a focus on SL skill practice and development in the course associated with this research project, we cannot attribute the changes in students' SL skills during the Fall 2019 semester, or during the following Winter 2020 semester exclusively to their experience in this course, as other concurrent learning experiences in other courses during the semester may also have contributed to the SL outcomes.
The Follow-up SL survey (Survey 3) was conducted 4 months later and at the time of graduation from their undergraduate program for 93% of students who participated in the survey. There were additional improvements during this 4 month period in students' SL skill perceptions since the completion of the end of semester survey (Survey 2) in the areas of distinguishing between sources of scientific literature, interpreting data and integrating information from multiple sources, applying their knowledge to address and identify unfamiliar problems and research questions, and in knowledge translation (i.e. translating complex information from the scientific literature into clear and understandable terms) (Figure 5). In this connection, acquiring knowledge translation skills in biological science education is critically important, particularly once undergraduates enter the workplace or continue their education (Graham et al., 2006). Moreover, the knowledge translation components of SL are viewed to be as essential as problem solving, critical thinking and collaborative skills (Wensing & Grol, 2019;Sibley et al., 2017). These findings demonstrate that students have attained proficiency in knowledge translation capabilities at the time of completing their undergraduate program.
Collectively, the results from the current study show that students' perceptions of their SL capabilities may not align with their actual capabilities; however, students did retain their practical SL capabilities over an 8 month period, as assessed by the TOSLS score across three time points. Identifying and implementing teaching strategies, course activities (such as laboratory experience) and assessments to improve students' practical SL skills in upper year undergraduate biological science programs may represent an important mechanism to enhance these critical skills.