Types of Data Collected--Advantages and Disadvantages
Research studies can employ a wide range of data collection, quantitative, qualitative, and a mixed methods approach. Each holds particular advantages and disadvantages. Quantitative data analysis allows statistical tests that allows researchers to make statements about the data. Both descriptive statistical analysis as well as inferential statistical analysis can be applied. For example, t-tests, ANOVAs, or MRC allows an individual to derive important facts from the research such as trends, differences between groups, and demographics. The p-value can help indicate the likelihood that research findings were the result of chance. However, there are also disadvantages to quantitative data analysis. Sample sizes matter. Statistical significance may not be achieved if a sample size is too small therefore, it may be a disadvantage because quantitative studies relies on a large sample size. Advantages of qualitative studies include the ability to gain more detailed and rich data along with meaningful context. The study can be observed in a flexible and natural way unlike quantitative studies which are quite structured. Disadvantages are that they are extremely time consuming to observe and code and also subjected to bias because the researcher is usually heavily involved. A mixed-methods approach is a combination of quantitative and qualitative approaches. The combination can be quite effective because qualitative research can inform quantitative studies. Information gathered from the qualitative research can be used to design more effective quantitative research that assess the specific area that is being researched. I will be using a mixed-method approach in my research study because it will give me statistical power to analyze data to support my hypothesis using the t-test. The assessment measures are less bias and less time consuming to collect and analyze. However, I also want to see my students' thinking because my topic is related to metacognitive thinking so I want to be able to see evidence of metacognition and also the types of thinking. Furthermore, this method will allow me to analyze the pre-test and post-test scores to determine statistical significance so I can make a definitive statement about whether my research treatment directly affected the increase in test scores. I hope to see through journal writing, whether my students articulation of the math becomes more sophisticated and whether they will provide evidence of critical thinking skills as a result of my treatment.
0 Comments
Literature Review
Critical Thinking The concept of critical thinking is not a new innovation. Critical thinking has existed as early as the time of Socrates (c. 470—399 BC) more than 2,400 years ago as a deep questioning technique, a method of disciplined and rigorous questioning geared towards the logical analysis and evaluation of the reliability and validity of beliefs (Paul & Elder, 2014). However, scholars such as John Dewey, Robert Ennis, Richard Paul, and many other theorists gradually began to expand on this early notion of critical thinking which has now come to include reflective thinking, creative thinking, problem-solving, and metacognition (Dewey, 1910; Ennis, 1985; Paul & Elder, 2006). Critical Thinking Skills. Critical thinking skills include the ability to think reflectively and reasonably, analyze arguments, challenge assumptions, discern opinion from fact, evaluate issues from various perspectives, problem solve using flexible thinking, and self-regulate or think metacognitively (Ennis, 1985; Paul & Elder 2006). All critical thinking skills are not created equal, more precisely, depending on the discipline, some critical thinking skills may be valued more highly than others. For example, in an English course there may be an emphasis on the ability to analyze and evaluate arguments and assumptions whereas critical thinking skills such as problem solving and self-regulation may be considered more essential in mathematics. In fact, Paul and Elder (2006) frame the entire act of critical thinking as a metacognitive process, suggesting that just becoming aware of and thinking about one’s own thinking can improve a person’s ability to evaluate arguments and challenge assumptions. Within the context of mathematics, critical thinking skills are described as the ability to problem solve, utilizing flexibility of thinking and a broad repertoire of techniques for dealing with novel or non-routine problems along with the ability to reflect on progress as well as analyze and interpret vast amounts of data (Schoenfeld, 1992). Metacognition Metacognition is defined as the awareness and regulation of one’s own thought processes (Flavell, 1979). To elaborate, metacognition is comprised of two components: metacognitive knowledge and metacognitive regulation. Metacognitive knowledge consists of three types of knowledge: declarative knowledge, procedural knowledge, and conditional knowledge. Declarative knowledge is “what” one knows, procedural knowledge is “how” one applies that knowledge, and conditional knowledge is the “when, where, and why” one uses that knowledge. Metacognitive regulation consists of three elements: planning, monitoring, and evaluating (Schraw & Moshman, 1995). Knowledge and regulation are both vital for problem solving. Intersection of Metacognition and Problem Solving. Mathematicians have long since recognized the close relationship that exists between these two critical thinking skills, metacognition and problem solving (Gray, 1991). Metacognition and problem solving often have overlapping elements. During problem solving, metacognitive skills are utilized to analyze information, identify goals within the task, plan, monitor progress, consider alternatives, and evaluate decisions and outcomes of a problem (Garofalo & Lester, 1985, Polya, 1957). Significance of Metacognition. Metacognition can play a crucial role in learning and increase conceptual understanding and procedural fluency (Jbeili, 2012). Because learning shifts from conscious to automatic processing (concept mastery), it is important that increased consciousness is paid during the first stage of learning for it to lead to deeper knowledge (Pammu, Amir, and Maasum, 2011). However, acquiring conceptual understanding and procedural fluency does not ensure that the individual will know how or when to apply it to more complex or unfamiliar problems (Kurfiss, 1988; Mayer, 1993). Studies show that in many cases students have the necessary knowledge base needed to solve a problem but lack the metacognitive knowledge known as conditional knowledge to help them determine how to utilize that knowledge base. Unless the world is a game of Jeopardy, there is no advantage to possessing great amounts of knowledge that cannot be utilized. Moreover, studies indicate that the presence of strong metacognitive skills is a better predictor of student problem solving success than their aptitude and can even compensate for low aptitude (Schraw & Moshman, 1995; Swanson, 1990). Hence, it is important to help students develop strong metacognitive skills. Research-Based Instructional Strategies for Developing Metacognitive Skills Extensive research has been done detailing the value of various instructional strategies for developing metacognitive thinking skills. These strategies include: (a) direct explicit instruction, often in the form of a think aloud, modeling, and whole class discussion, (b) metacognitive regulation strategies that helps students plan, monitor, and evaluate their learning, also accomplished using think alouds as well as journaling in order to discuss and document thinking, and (c) teacher and peer feedback, achieved through teacher questioning and prompting in the form of Socratic Questioning and peer interaction in the form of Socratic Seminars. Several studies show that metacognitive regulation and feedback typically go hand in hand and are directly supported by explicit instruction. In both cases, explicit instruction is the means for teaching these skills. Moreover, it is important to note that numerous studies incorporate similar metacognitive regulation strategies, often, referred to by different names: comprehension monitoring, metacognitive training, and metacognitive scaffolding. Explicit Instruction. Teachers expect students to utilize critical thinking skills to produce high quality work for class projects but how can students or anyone for that matter, be expected to apply skills they have never been effectively or explicitly taught? How can they know what they just learned if they are not told what it is they learned? How can they know how, when, or why to use it if they were not shown? Studies show that explicit instruction of metacognitive thinking is beneficial to students, especially struggling students because it offers them flexibility, greater efficiency, and transferability of skills to unfamiliar situations during problem solving (Lin, 2001; Pintrich, 2002; Schoenfeld, 1992). These skills are not obtained automatically, thus, it is important that teachers help students develop metacognitive thinking with the use of direct and explicit instruction, making the thinking process visible (Conrady, 2015). To achieve this, students must be given opportunities to develop their metacognitive knowledge and regulation skills— these are the skills that allow a student to know when and why a certain procedure should be applied; some scaffolding activities that allow students to practice these skills are whole class discussions, modeling of the problem solving process, a think aloud, [Socratic] questioning, writing about thinking, prompting, using sentence starters, and explicit instruction about thinking and metacognition (Flavell, 1979; Goos, et al., 2002; Pintrich, 2002). Conrady (2015) conducted a naturalistic inquiry study investigating explicit modeling of metacognitive thinking embedded in two university level geometry courses for pre-service education teachers. In the study, 51 students were exposed to explicit instruction of metacognitive thinking strategies such as modeling, think alouds, prompting, and questioning. Observations on the explicit thinking of the students were documented through field notes and transcribed videotapes. Conrady (2015) concluded that the use of think alouds and modeling helped students develop and make their own procedural thinking explicit. However, she also found that students did not demonstrate independent regulatory thoughts during problem solving and that most regulatory thoughts required prompting or questioning by the teacher. Based on the results of Conrady’s (2015) study, it can be concluded that explicit instruction alone is not enough to fully develop all components of metacognitive thinking. Instead, several studies suggest that explicit instruction be strategically taught using various contexts, examples, and applications along with strategies for developing metacognitive regulation skills such as comprehension monitoring, metacognitive training or metacognitive scaffolding (Schurter, 2002; Kramarski Mevarech, 2003; Jbeili, 2012). In a research study, Schurter investigated the effects of using comprehension monitoring, problem solving strategies, and explicit instruction on student problem solving abilities in three university level remedial mathematics courses totaling 60 students. Comprehension monitoring is a metacognitive regulation technique that provides students self-questioning techniques that guide them through problem solving. Findings showed that students who were taught comprehension monitoring alone or in conjunction with problem solving strategies outperformed those that were taught with traditional instruction. However, the results indicated that there was no significant difference between the two treatment groups. In conclusion, the findings of this study support the position that the implementation of comprehension monitoring strategies alone or accompanied by problem solving strategies can improve performance in mathematical problem solving. In a similar study, Kramarski and Meravech (2003) investigated the effects of a metacognitive regulation technique called metacognitive training on mathematical reasoning 12 eighth grade classrooms totaling 384 students. Metacognitive training is a metacognitive regulation strategy like comprehension monitoring. Using metacognitive tranining, students were taught how to use self-questioning techniques during problem solving. Findings from this study echoed Schurter’s findings, suggesting that metacognitive training directly contributed to the improvement of mathematical reasoning. Moreover, the group that was exposed to cooperative learning and metacognitive strategies (COOP+META) outperformed their counterparts, the independent and metacognitive group (IND+META), which in turn outperformed the cooperative group (COOP) and the independent group (IND). No significant difference was found between the two groups, COOP and IND, that were not exposed to metacognitive strategies. In conclusion, evidence from this study showed that the two treatment groups, COOP+META and IND+META, developed fluency and flexible thinking, better knowledge transfer, and the ability to utilize logical-formal arguments as a direct result of using metacognitive regulation strategies. In a similar study to Kramarski and Meravech (2003), Jbeili (2012) examined the effect of metacognitive scaffolding embedded within cooperative learning on 240 fifth grade students’ mathematics conceptual understanding and procedural fluency in problem solving. The result of Jbeili’s study coincided with Schurter (2002) and Kramarski and Mevarech’s (2003) findings that confirmed the importance of learning strategies that used scaffolding techniques to develop metacognitive regulation skills. The data indicated that the group taught using cooperative learning with metacognitive scaffolding (CLMS) significantly outperformed their counterparts, the cooperative learning group (CL) which in turn outperformed their counterparts, the traditional group (T). Jbeili concluded that the CLMS group outperformed both its counterparts in problem solving that required conceptual understanding and procedural fluency because they were provided with various strategies to support this outcome. To elaborate, the CLMS group surpassed their counterparts because they were able to work cooperatively using metacognitive questions. The metacognitive questioning helped prompt students to construct their knowledge and skills by assisting them in retrieving prior knowledge as well as connecting it to new knowledge, building and reinforcing schema. It enabled them to evaluate problems and connect it to similar past problems improving accuracy and efficiency in problem solving. It provided them with flexible thinking and multiple approaches through group discussion and support of the metacognitive questions. It facilitated cooperation and deep learning by requiring them to explain their thinking to group members during discussion rather than relying on rote memory. It can be concluded that all of the skills described above assisted the CLMS group in easily remembering and retrieving math concepts and problem solving strategies and thus, the reason for their high achievement. Reflective Journal Writing. U.S. teachers often communicate that their students struggle with problem solving. During problem solving, instructors observed that students spent very little time planning, quickly chose one strategy to apply and never reflected on the effectiveness of the strategy as they chugged away and eventually gave up. Olson & Johnson (2012) investigated the value of journal writing in mathematics for two groups of eighth grade students totaling 107 students. Their findings indicated that the treatment group showed greater achievement than the group who did not engage in journal writing. Furthermore, the study also indicated that recording thinking steps during problem solving had several benefits. Gray (1991) and Olson and Johnson (2012) reported that journal writing allowed students to reflect, monitor, and evaluate their mathematical, it promoted critical thinking skills such as analyzing data, evaluating and comparing facts, and synthesizing information, and it allowed instructors to assess students’ mathematical thinking and provided regular feedback to deepen the understanding of a concept or correct student thinking. In conclusion, the results of this study showed that regular use of journal writing improved both student academic achievement as well as attitude towards mathematics. Socratic Questioning and Seminars. In Conrady’s (2015) study, students frequently struggled to work through an idea on their own relying heavily on teacher questioning or prompting to develop their remaining thought. Theorists posit that learning is constructed in exactly this way—questioning. Vygotsky’s (1986) theory of cognitive development describes learning as a social process. Likewise, the mathematics learning process is also described as an “inherently social activity” in which students develop metacognitive thinking skills through sharing, comparing, and evaluating mathematical strategies among their peers in order to determine the best approach (Conrady, 2015; Pintrich, 2002; Schoenfeld, 1992). Researchers hypothesized that cooperative learning accompanied by reflective discussion helped students develop critical thinking skills and deepen their understanding of concepts and procedures (Jbeili, 2012; Kramarski & Mevarachi, 2003; Schurter, 2015). Several studies suggest that Socratic Questioning and Socratic Seminars are effective strategies for developing these skills. In a study, Tanner and Casados (1998) examined the contributions of Socratic Seminars in a Trigonometry, Statistics, and Functions class of 17 high school students. Data was collected and analyzed using videotaping, surveys, and research journal notes. The findings of the study concluded that using Socratic Seminars allowed students to talk through ideas and as a result, students became insightful, logical mathematical solvers. Tanner and Casados’ findings coincided with Olson and Johnson’s (2012) findings, illustrating that student mathematics disposition, active participation, and articulation of concepts all increased as a result of Socratic Questioning and Seminars. Yang (2008) conducted a similar study investigating the effects of Socratic Questioning or dialoging in online discussion on critical thinking skills. The results of her study corroborated previous studies. The data suggested that the use and modeling of Socratic Questioning had a positive effect on student critical thinking skills. It aided students in constructing new meaning from content, enabled them to think independently and critically, explore applications to problems, and generate thoughtful questions (Tanner & Casados, 1998; Yang, 2008). The study seeks to determine whether the development of critical thinking skills will have an effect on standardized test performance. Specifically, the study will explore how metacognition as a critical thinking skill will affect concept mastery and retention and in turn how that might impact standardized test performance. The study will be a mixed methods design with both quantitative and qualitative components. It will include a quasi-experimental design or a pretest-posttest control group design as well as a naturalistic design that involves qualitative data.
The treatment will be comprised of explicit direct instruction of metacognitive skills (teacher think aloud), Socratic Questioning (providing feedback), Socratic Seminar (metacognitive practice and peer feedback), and a metacognitive journal (metacognitive practice of self-regulation). At the beginning and the end of the study students from both the experimental and control group will take an pre/post-assessment which is taken from the official CAASPP practice test. The pre- and post-assessment will be identical. In order to produce valid and reliable data, several questions with varying levels of difficulty will be selected from the official CAASPP testing website as the unbiased measurement tool to simulate a standardized test. After each pre/post-assessment, students will complete a short four question self-assessment using a rating scale of 1-5 regarding their perception of the assessment. This will provide data on student progress comparing pre- and post-assessment as well as reflective feedback from the students on their own progress. The students will complete two learning activities that involve problem solving of performance tasks at a higher level of difficulty. The instructor will explicitly model metacognitive thinking skills on the first performance task using a think aloud first then have the students complete the second performance task in groups of three. The experimental group will use a metacognitive journal to document their problem solving which will include a planning section, monitoring section, and evaluating section as well as metacognitive guiding questions in the right margin for each phase. The experimental group will then engage in a Socratic Seminar with the aid of their metacognitive journal in their discussion. Detailed Treatment:The experimental group will complete a short metacognitive awareness inventory self-assessment and scoring guide which will allow the students to evaluate what kind of thinker they are. Next they will form groups of three and read a short article on metacognition explaining metacognition and the importance of metacognition in academics as well as in the real world. They will summarize the information from the article in cornell note format. Thereafter, students will complete their first metacognitive journal entry with pre-selected writing prompts. The next day, students will attempt to complete a CAASPP practice performance task in their groups of three. After, the instructor will explicitly model how to utilize metacognitive skills during problem solving using a think aloud method. The day after, students will work on their own performance task in their same groups with the aid of the metacognitive journal template. The metacognitive journal template mimics a double entry journal commonly used in the mathematics setting but with modifications. Unlike a double entry journal that consists of two columns (steps and explanation/justifications), the metacognitive journal template has a third column which lists metacognitive questions that guide students on how to proceed during the problem solving process. There is also a section for students to jot down questions that come up as they engage in problem solving so they can ask the instructor or another peer. The metacognitive journal is also divided into three sections: planning phase, monitoring phase, and evaluating phase. The planning phase includes guiding questions that are metacognitive in nature which activates prior knowledge and allow students to problem solve independently. The monitoring phase asks the students to document their strategy and steps in the first column and explain and/or justify in the second column. The evaluation phase asks the students to think about their answer and whether their method was effective and how the process could be applied to other contexts as well as in the real world. The study will include 60 participants, 30 in the experimental group and 30 in the control group. The demographics of the participants will include both females and males consisting of diverse ethnic backgrounds (Caucasian, African-American, Hispanic and Asian/Asian-American) with similar socioeconomic backgrounds. Students with disabilities will not be included. The control will be chosen randomly between two classes who have similar demographics (relatively same amount of males and females, English Learners, with similar ability levels). Data that will be collected includes a pre- and post-assessment, a pre- and post- self-assessment, and a metacognitive journal. All data will be collected in a paper format. This is the best and safest way to collect the because it will ensure that data truly belong to that subject. It will also ensure that subjects are not denied access to the treatment or assessment because of their socio-economic backgrounds (i.e. inability to purchase technological devices, etc.) and other various reasons. The pre- and post-assessment are short answer and multiple choice type questions and will be graded against the official CAASPP practice test grading rubric. The data from the two tests will be analyzed and represented as a bar graph. The data from the self-assessment will be counted and represented in the form of a bar graph for both pre- and post- self-assessments. Finally, the metacognitive journals will provide qualitative data that will be analyzed to find the evidence of student metacognitive thinking. Limitations of this study are that the number of participants are small and that it is conducted in a very short period of time, over the course of two weeks, therefore any absences could affect the results. Loss of preparation time and opportunities to practice their metacognitive skills could diminish their progress. Also mentally, the students are also experiencing high stress due to actual state testing in other classes during those two weeks coupled with the fact that the experimental group have math first thing in the morning when they are not fully awake and the control group have math at the end of the day when they can become antsy. Critical Thinking--Theorists and Researchers in Psychology and Instructional Design
As educators, we continually seek to improve our pedagogy and explore ways we can better serve all of our students. How do we go about accomplishing this you might ask? The answer is in action research. Looking at my own classroom, I wondered how I could help my students gain better retention of conceptual and procedural knowledge of mathematics. This gave rise to the driving question to my research on critical thinking. What effects does critical thinking have on retention? What role does metacognitive thinking play in critical thinking? It was obvious that I needed more information so I set out to gain more insight on this topic. Some basic questions I began with were: What is critical thinking? Why is there a need for teaching critical thinking skills? How do we teach critical thinking? I came across an article called A Model for Teaching Critical Thinking by Marnice K. Emerson (2013) which provided the exact answers to my questions. The author provides an extensive list of references of various influential figures in the area of critical thinking—theorists and researchers in psychology and instructional design. Prominent theorists in the area of critical thinking include Joanne Gainen Kurfiss, Diane Halpern, Robert Ennis, and Richard Paul. These theorists developed definitions of critical thinking that evolved over time to include metacognition and problem solving in addition to reflective and reasonable thinking focused on what to believe or do (Emerson, 2013). Focusing on Joanne G. Kurfiss (1988), in her report, “Critical Thinking: Theory, Research, Practice, and Possibilities”, she identifies reasoning skills most critical to success in six disciplines. She also found that these critical thinking skills only partially overlapped across disciplines. In conclusion, different disciplines value different skills. For example, in Science, the ability to draw inferences from observations, critically analyze and evaluate and generate new questions or experiments are valued while in English, the ability to elaborate an argument, develop implications, understand, analyze, evaluate arguments, support assertions, and recognize the central thesis in a work are most valued. She explains that the problem lies with the fact that “Although critical thinking skills are valued, they are seldom explicitly taught to students” (Kurfiss 1988, 21). Furthermore, products of these critical thinking skills are displayed in the form of arguments or interpretations but students rarely get the opportunity to witness the process. In addition to more explicit instruction, Kurfiss also calls for instruction that is well organized so students are able to organize what they are learning into a matrix or hierarchy. She states that knowledge takes different forms which includes declarative knowledge and procedural or strategic knowledge. Declarative knowledge includes concepts, principles, stories, and other proposition knowledge use to make inferences whereas procedural or strategic knowledge is knowing how or when to use declarative knowledge (Kurfiss 1988). She later elaborates that competent problem solvers plan and monitor their work by making plans, setting goals, ask questions, take notes, observe effectiveness of their efforts, and take corrective action when necessary. Likewise, “A third factor influencing problem solving is metacognition, the use of strategies to monitor and control attention and memory to make decisions about how to proceed on a task” (Kurfiss 1988, 59). The big idea to take away from the ideas of these theorists is that critical thinking is a necessary skill that must be taught explicitly which is particular to different disciplines. It is necessary to teach declarative and procedural knowledge as well as metacognitive thinking skills to effectively teach critical thinking skills. Emerson’s article also discusses different approaches to teaching critical thinking, listing four ways: (1) mixed approach, (2) an immersion approach, (3) a general approach, and (4) an infusion approach. Although all four yield positive results, ranging from moderate to high, the mixed method approach in which critical thinking skills were explicitly taught as a separate unit but within a topical course seemed to have the largest impact. Emerson references several figures in instructional design for the basis of her conclusion. A few well-known figures in instructional design and technology include, Robert Gagné and Marriner David Merrill. Robert Gagné developed a theory that stipulates that there are several different types or levels of learning. Different types of learning require different types of instruction. Five major categories of learning include: verbal information, intellectual skills, cognitive strategies, motor skills, and attitudes. Different learning conditions are necessary for each type of learning. For example, for cognitive strategies, there must be opportunities to practice developing new solutions. The theory also outlines nine instructional events and corresponding cognitive processes organized in a hierarchy: gaining attention (reception), informing learners of the objective (expectancy), stimulating recall or prior learning (retrieval), presenting the stimulus (selective perception), providing learning guidance (semantic encoding), eliciting performance (responding), providing feedback (responding), assessing performance (retrieval), and enhancing retention and transfer (generalization). The hierarchy serves to identify prerequisites that should be completed to facilitate learning at each level. These learning conditions serve as a basis for instructional design and selecting appropriate media (Gagné, Briggs & Wager, 1992). As mentioned before, the first level is gaining attention or engaging students. Student disposition and motivation is a factor in teaching critical thinking. The big idea is that we need students to own the learning and after that we need to teach them how and when to use these critical thinking strategies. In order to do this, we need to explicitly teach students how to apply these critical thinking skills by modeling with a think aloud process. Kurfiss also echoes the necessity for explicit teaching of critical thinking by modeling the process by an expert. Research suggests that this is an effective strategy for teaching expert behavior. “Instructional design theorists corroborate this principle by positing that practicing a skill is an integral part of instruction, leading to significantly higher retention and transfer learning” (Emerson 2013, 12). Emerson (2013) also states that an additional strategy shown to be effective for teaching critical thinking skills is the process of offering reflective feedback on learners’ practices of their thinking skills by the instructor. This is also greatly supported by instructional design theorists for maximizing transfer learning. Two strategies for providing feedback include scaffolding and Socratic dialoging. Focusing on Socratic dialoging, this strategy aids students in increasing their awareness of hidden assumptions and helps students question supposed concrete assertions. Students’ critical thinking skills begin to improve as well as their ability to transfer those skills to new contexts (Emerson 2013). Ultimately, I have gained insight on effective strategies for teaching critical thinking skills as well as gained an abundant amount of knowledge on the background of critical thinking as well as support of expert theorists calling for more research in the area of critical thinking skills and metacognition. What is the Educational Context for Your Driving Question--International, National, State, District and School?
The 21st century has changed education as we have known it and it will continue to change so long as time goes on. Although whether we are prepared for it or not is a different matter. More and more jobs in the market are demanding knowledge and skills of our students that our educational system is not well-equipped to deal with. The fastest growing jobs require postsecondary education however high school graduation rates have decreased and only a third receive their college degree according to Linda Darling-Hammond in The Flat World and Education: How America’s Commitment to Equity Will Determine Our Future. In summary, jobs are demanding specialized skills and knowledge in which our current education system is not preparing the new generation for. Therefore, there is a necessity for a shift towards education reform that focuses on students and how to guide them in order to meet new challenges of the 21st century. And one of the most crucial 21st century skills is the development of critical thinking. Critical thinking is defined as problem solving and the ability to take information to put it to use to produce solutions. Specifically, critical thinking in mathematics involves the 8 Mathematical Practices outlined in the Common Core State Standards (CCSS) which includes analysis, interpretation, precision and accuracy, constructing viable arguments, and reasoning quantitatively and abstractly. In the article "Mathematical Teaching Strategies:Pathways to Critical Thinking and Metacognition" in the International Journal of Research in Education and Science (IJRES), 93% of business and non-profit leaders believe that “a demonstrated capacity to think critically, communicate clearly, and solve complex problems is more important than undergraduate major.” and more than 75% stated that they wanted more emphasis on critical thinking, complex problem solving, written and oral communication, and applied knowledge in real-world settings (Su, Ricci, and Mnatsakanian, 2016). In their recommendations, they state that there is a need to replace our current math classes with meaningful mathematical experiences, which teach “how to think through Math” rather than memorizing formulas. Thus, my proposed research question specifically tends towards exploring the relationship between critical thinking and standardized test performance. On an international level, higher order thinking skills or critical thinking skills comprise 65% of the tests as compared to 35% on lower order thinking skills like knowledge according to the TIMSS 2015 Mathematics Framework. According to the National Center for Education Statistics (NCES), The Program for International Student Assessment (PISA) coordinated by the Organization for Economic Cooperation and Development (OECD) has measured the performance of 15-year-old students in mathematics every 3 years since 2000. In 2012, PISA was administered to 65 countries and education systems, including all 34 member countries of the OECD. Proficiency results are presented in terms of the percentages of students reaching proficiency level 5 or above and students performing below proficiency level 2. Students scoring at proficiency levels 5 and above are considered to be top performers since they have demonstrated advanced mathematical thinking and reasoning skills required to solve problems of greater complexity. The percentage of top performers in the United States was lower than the average of the OECD countries’ percentages of top performers (9 vs. 13 percent). Twenty-seven education systems and two U.S. states had higher percentages of top performers than the United States. Massachusetts and Connecticut both had higher percentages of top performers, respectively 19 and 16 percent than the United states at nine percent. Likewise, the data assessment on trends in international mathematics achievement according to TIMSS 2015 shows the gap between the highest achieving countries and the lowest achieving countries was 48 points. Singapore, Korea, Chinese Taipei, Hong Kong SAR, and Japan are among the highest achieving countries. Two case studies presented in Linda Darling-Hammond’s The Flat World and Education: How America’s Commitment to Equity Will Determine Our Future involving two top performing countries, Korea and Singapore along with Finland shows the importance as well as the effects of a curriculum that involves teaching critical thinking skills (21st century learning skills). On the contrary, the United States has become stagnant. Within a span of four years, from 2011 to 2015, the United States has only increased nine points, 509 to 518. In relation to my driving question, there has been a long and debated history surrounding issues on curriculum around the world and within the context of the United States, it is sometimes referred to as the “curriculum war”. Nationally, data assessment from NAEP shows that only 33% of students in the eighth grade are performing at or above the proficient level in mathematics. That is 67% of students (or more than half of the nation’s eighth graders) are not meeting the standards. Data also shows that the average mathematics scores have dropped in 2015 compared to 2013 from a score of 285 to 282. Additionally, since Common Core reforms have been implemented in 2010, scores increased only 1-2 points. In 2009, the average scores were 283 compared to 284 in 2011 and 285 in 2013. With the shift to Common Core and state standardized tests (Smarter Balanced Test or CAASPP), there has also been a shift from testing rote learning to testing both content and cognitive domains. There are four claims (or categories) in the cognitive domains, (1) Concepts and Procedures, (2) Problem Solving, (3) Communicating Reasoning, and (4) Modeling and Data Analysis. Three of the four categories require higher order critical thinking. The CAASPP is comprised of two items, a computer adaptive test (CAT) and a performance task (PT). The CAT test includes 16-20 questions in claim 1 (lower order thinking), and 16-20 questions in claims 2-4 (higher order thinking) while the PT is solely made up of questions only from claims 2-4. Although the majority of test questions (estimated 50%) are still composed of concept and procedural thinking, the other half the test still calls for critical thinking skills. Thus, in response to national and international data assessment, California has adopted the Common Core State Standards and several districts have begun implementing 21st Century Skills and STEM within our educational school systems. However, this by no means indicates that this is a state-wide nor even district-wide reform. Within local contexts of my district, my school is a big supporter of teaching 21st Century Skills (6 C’s). One of the ways in which we incorporate 21st Century learning and teaching is through Project-Based Learning and Problem-Based Learning (PrBL). According to my principal, PBL has not shown enough impact on scores regarding the standardized tests which is why we have implemented shorter real-world problem-based learning (PrBL) activities which are still rigorous and authentic. However, with time restrictions and pacing guides, PrBL activities are limited and sporadic at best. Moreover, critical thinking skills are not explicitly nor systematically being taught within PrBL activities so it is unclear to what extent the development of critical thinking skills may have on standardized test performance. An Experimental or Naturalistic Approach? Developing a Framework for Research and a Foundation--Subquestions
Driving Question: What effects does the development of critical thinking skills have on standardized test performance? The driving question is a big overarching question that provides a framework for our study. “Each element within the study will be related to this overarching question” (Falk & Blumenreich 2005, p. 21). The elements within the study are known as subquestions or "need to knows". Subquestions provides the foundation of our inquiry. Falk & Blumenreich, 2005 also state that it makes our inquiry more manageable by helping us narrow and define what we are going to examine within the context of our question. The subquestion also helps guide us in deciding what tools to use for our study and aid in our analysis of the information we collect. Therefore, in order to address my driving question, the elements within my study will focus on the subquestions below. My curiosity lies in wanting to find out whether developing critical thinking skills will naturally lead to creating autonomous (metacognitive) thinkers which will in term foster concept mastery and retention and lead to improvement in standardized test performance. Critical Thinking => Metacognitive Thinkers => Retention => Improved Standardized Test Performance Subquestions:
Now that the foundation has been laid, I must decide what tools or treatment I will use in this study. First and foremost, I plan on using a positive paradigm or experimental approach (quasi-experimental approach) as my primary method. I will be administering a pre-assessment (CAASPP practice test) to both groups of participants (the experimental and the control group). It is important that any measurement tools being used produces valid and reliable results, hence the use of a standardized test that has been vetted by a trusted and unbiased institution. That also leads into a dilemma that concerns which type of CAASPP test should be used to measure student academic performance. The CAASPP consist of the CAT (summative computer-based adaptive test) as well as the PT (performance task). The CAT is a mix of multiple choice and short answer question types while the PT also involves short answer questions as well as explanation of method and reasoning. Personally, to eliminate any biases or human error, I think it would be best to use the CAT test which does not involve a subjective grader. How to give partial points in itself is already a dilemma I do not wish to revisit. My research will also consist of a naturalistic approach in which I will be collecting qualitative data in the form of a metacognitive journal. Students will be asked to document their thought processes during their critical thinking learning activities which includes: detailed methodology and justifications (initial approach, misunderstandings, self-corrections, new strategies, results, validity of results, other possible methods, etc.). I will provide consistent feedback on their metacognitive journals that will be used to guide them towards deeper thinking. Furthermore, I will also introduce learning activities that involve peer critiquing such as Socratic seminars (adapted to a mathematical setting) that gives them more practice with critical thinking. At the end of the module, I will administer a post-assessment. The post-assessment will be the same as the pre-assessment. Regarding this decision, I too have some concerns. I wonder whether taking the same test multiple times can distort the results. How do I know whether it was the treatment that affected their test performance and not because they have seen the same problem repeatedly? Part of me wants to know that my students can use their critical thinking skills to analyze a problem, interpret the information, and come up a method to solve (using the state standard concepts and skills they have learned thus far). I feel like that is the true meaning of critical thinking. Let us explore the senario, what if the problem changes (in context and/or possibly wording)? I wonder if not changing the context or wording of a problem is the exact same thing as teaching rote memory. If students can only solve a problem if it is introduced the exact same way (in context and wording) as something they have seen before then have they really gained concept mastery and retention? If students are unable come up with an appropriate strategy for solving, then perhaps they have not developed the critical thinking skills that allow for retention of concepts and skills. It might be important to include problems that are different to address this concern or may be not. Similarly, is there a possibility that students may discuss the problems amongst each other and likewise, would that distort the data? Finally, I would also like to include a survey after each of the pre-assessment and post-assessment. The survey would also be quantitative. It will measure students perspective on how they think they did on the pre-assessment (low or high score using a scale from 1-5). I would also like to include other survey questions (quantitative and qualitative) that has yet to be determined. To conclude, there may be other tools I would like to consider for collecting data but for now this is my work in progress. The context and background for my question stems from my own classroom dilemma. I am concerned that my students are too dependent on the aid of notes and practice exams during summative tests. Initially, it was a way to incentivize my students to take proper notes and learn how to utilize them to reap the benefits. Additionally, it was also just a tool for my special populations (my low performers, EL’s, and students with special needs) but I did not want to spotlight those individuals who needed such accommodations. Therefore, I allowed all my students to use notes on the tests. In addition, I also realize that the summative tests at the end of each chapter can sometimes contain up to 9+ different concepts or skills and after 3-5 weeks, that knowledge they once had may have disappeared. Also, being able to physically match up similar problems for the practice test to the summative test to mimic the steps without understanding the application is not a skill they need nor a luxury they can afford when they enter high school (for the most part) nor college and beyond. College entrance (undergraduate/graduate) is partially determined by standardized tests (SAT/ACT/GRE/LSAT/MCAT). I hope that in gaining critical thinking skills, the students will be able to master the concepts and have better retention and in turn, improve their standardized test performance. Driving Question: What effect does the development of critical thinking skills have on standardized test performance?
After reading countless articles regarding driving questions, the general consensus is that quality driving questions are interesting and provocative, open-ended, challenging, captures the heart of the project, provides a purpose, and can arise from a real dilemma. In my classroom, that dilemma concerns students’ ability to attain concept mastery and be autonomous thinkers. To elaborate, I teach at a middle school in the content area of mathematics. To be precise, I teach adolescents who are experiencing a time of great change (physical, cognitive, social, and emotional). They are in a stage of their life where they are just beginning to form a personal identity and peer relationships. Many times, academics is not their first priority. Moreover, math is rarely considered a friend middle school students wish to understand better. Based on my eight years of teaching experience, most students have been conditioned to “plug and chug” numbers into equations and formulas without understanding the purpose or deeper meaning of it. Furthermore, when faced with a test, my students become crippled without the assistance of written notes. This leads directly to the focus of my dilemma. I have noticed that without the aid of written notes or a practice test from which they can reference, many of my students become hopeless. When presented with a word problem or more cognitively complex tasks, I have also noticed that my students will pick out numbers and perform computations that have been programmed into them with no thought to what is being asked of them, what strategies might be most effective, and whether their answers make sense in the given context. Notes, cheat sheets, practice tests—those are not guaranteed luxuries in high school, college, and beyond. I feel it my duty to ween this generation of students who want everything spoon fed to them off of these so-called "scaffolds" that may prove a detriment. Thus, we arrive at a very important 21st century skill—critical thinking. Critical thinking is problem solving, the ability to take information and put it to use to produce solutions. Specifically, critical thinking in mathematics involves the 8 mathematical practices outlined in the common core state standards which includes analysis, interpretation, precision and accuracy, problem solving, and reasoning (not in this order nor in these exact words). The goal of my research is to explore whether there is a relationship between developing critical thinking skills and performance on standardized tests and thus attaining concept mastery and retention. Considering globalization and the advancements in this new generation, critical thinking is even more vital for student success beyond the four walls of my classroom. What will I need to know to answer this question? For the time being, what strategies can I utilize to teach critical thinking? How will I know when the students have shown an increase or improvement in critical thinking? What will I use to assess their critical thinking skills? Is interest and motivation variables that need to be addressed? Would it be best to conduct the research on two separate populations using one as a control group? How do I decide which group will be the control group? How will I collect or document the data? As a teacher, I am passionate about providing my students with equitable access to education and by developing my students' 21st century skills like critical thinking, I hope to provide them with skills that will contribute to leveling the playing field for them. My school district as well as my school are great advocates of cognitively complex tasks that are embedded in PBL/PrBL. Common core state standards also include 8 mathematical standards in which the Smarter Balance standardized test (CAASPP) is based on. To improve this situation, I think that lessons need to be designed with a different perspective in mind. I think lessons should include tasks that help students practice their critical thinking skills. These activities should allow them to collaborate, analyze, discuss (interpret), and reason. These lessons should be consistent rather than sporadic or nonexistent. This is a challenging question. One in which I do not have much experience with. There are a couple of strategies I have in mind that I am interested in learning about more and am eager to explore. |
Nai Saelee
Middle school math teacher preparing the leaders of the future. Inspiring curiosity, creativity, collaboration Archives
December 2017
Categories |