Understanding and Facilitating Students' Evidentiary Reasoning in Biology Labs: Insights from assessments and Lab Discussions Focused on Evolutionary Tree-thinking

Reform documents have emphasized the importance of thinking with and about evidence in science learning. However, science educators report that students have struggled with understanding, using, and evaluating the evidence underpinning scientific knowledge. Without a comprehensive and practical framework, studies fail to reveal and address students’ difficulties in evidentiary reasoning in a comprehensive manner. The design of instruction and assessments were informed by the Conceptual Analysis of Disciplinary Evidence (CADE) framework, which links biological knowledge with epistemic considerations while considering both domain-general and discipline-specific aspects of evidence. Scaffolds were designed to guide lab class discussions during instruction and assessments were developed and implemented in an introductory undergraduate biological laboratory classroom where students reasoned with and about evidence in the context of evolutionary tree-thinking. The CADE was implemented to inform instructional methods in two ways: 1. Generic Evidence Scaffolds (GES) reminded students of general epistemic considerations; 2. Disciplinary Evidence Scaffolds (DES) explicitly reminded students of the disciplinary knowledge of relevance for considering biological evidence. To better understand and facilitate students' evidentiary reasoning, three studies were conducted for this dissertation. The following research questions were addressed:

• How did CADE influence the development and implementation of scaffolding for GES versus DES treatments during an evolutionary tree-thinking lab task?
• How did different scaffolded discussions differ from baseline in terms of the influence scaffolding had on students' evidentiary reasoning?
• What challenges remain in the implementation of discussions about evidence for evolutionary tree-thinking informed by CADE?
• Based on a comprehensive literature review, what assessments of evidentiary reasoning are being used in post-secondary biology laboratory classrooms where students conduct practical research, and what gaps remain for the development of new and useful assessments of students’ reasoning with and about evidence in undergrad biology labs?
• What difficulties with reasoning about evidence were revealed by the CADE-informed assessments of evolutionary tree-thinking and how did these new assessments address gaps from the literature review?

In the first study, the impact of the CADE as a guide to facilitation of students’ evidentiary reasoning was observed when a laboratory instructor supported biology students in evidentiary reasoning during an evolutionary tree-thinking task. In four successive semesters, instructors participated in a professional development workshop with the CADE where scaffolding questions were developed and modified collaboratively by the instructors, biologists, and learning science professionals. Lesson plans were modified, and classroom discussions of evolutionary tree-thinking were video recorded and analyzed to improve the scaffolded discussion processes in an instructor’s classrooms. Findings show the CADE to be useful as a practical framework for instructional design, which helped the lab instructor facilitate students’ evidentiary reasoning about evolutionary trees. In comparison to baseline, both GES and DES scaffolds were actually implemented to prompt discussions to address more aspects and relationships among types of evidence for evolutionary tree thinking. Several challenges remained in the implementation processes, such as dwelling on disciplinary knowledge without linking to epistemic considerations and asking for opinion vote rather than prompting consideration of the evidence.

A second study examined the different impact on students evidentiary reasoning during lab discussions prompted by GES and DES scaffolding during the video recorded task discussions. To identify the influence each type of scaffolding had on students' evidentiary reasoning, GES and DES scaffolded discussions were compared to baseline when the lesson was taught by the same instructor before and after professional development workshops with the CADE. The same video recordings from the first study were analyzed. Findings from the second study show that students considered more kinds of epistemic considerations and disciplinary knowledge in both GES and the DES scaffolded discussions compared to baseline. In particular, with GES, students brought in their disciplinary knowledge without being prompted to link to what they were previously taught. With DES, students were able to link the specific disciplinary knowledge with the epistemic considerations. With GES and DES, students sometimes reasoned by applying general epistemic considerations without linking to specific disciplinary knowledge.

A third study was conducted to compare established assessment instruments with two new assessments aimed at revealing students’ difficulties in evidentiary reasoning. A comprehensive literature review was done to facilitate the identification of appropriate assessment instruments. Each aspect of the notion of evidence that the instruments have assessed were categorized using the CADE as an analytical framework. First, gaps in terms of parameters that still need to be measured to reveal students’ reasoning about evidence were identified based on findings from the literature review of established assessments. Second, new assessment instruments informed by the CADE framework, using the CADE as the cognitive foundation of the assessment triangle, were designed collaboratively by biologists and learning scientists to address the gaps and assess students’ difficulties in evidentiary reasoning comprehensively. Expert answers were compared with undergraduate biology lab students’ pre- and post-test answers to evaluate the CADE-informed assessments in terms of the observations made of students’ competencies and the interpretation of their reasoning about evidence according to the assessment triangle. Findings from the third study reveal the CADE to be a useful tool to guide both choice and design of assessment instruments. Gaps within the currently used assessments were a lack of emphasis on linking students' epistemic considerations with their disciplinary knowledge, a lack of evaluating how well hypotheses are informed by theory, and a need to measure how well students combine and use convergent or discrepant evidence to support their conclusions. New assessments were found to address these gaps for evaluating students' evidentiary reasoning in a biology context-specific manner.

In conclusion, the CADE framework proved to be a useful guide for facilitating students' evidentiary reasoning and revealing students' difficulties in evidentiary reasoning. The findings of this research will benefit teaching and learning about evolutionary trees by providing educators with a feasible way to deconstruct and unpack the notion of evidence. The findings also provide insight into assessment instrument choices and the design of new assessment tools for use on tests to reveal students’ difficulties with evidentiary reasoning in biology as a discipline.