Faculty of Education - Theses
Permanent URI for this collection
Search Results
1 - 1 of 1
-
ItemAssessment of mathematical problem solving: addition and subtraction in arithmetic word problem solving( 2020)Researchers consider mathematical problem solving (MPS) to be a vehicle for teaching and reinforcing mathematical knowledge, solving real life problems, and helping to meet everyday challenges. Like many other countries in the world, Bangladesh has given great importance to MPS in its primary curriculum. The country has achieved well in quantitative parameters of primary education (e.g., net enrolment rate, completion rate, etc.) but there are concerns about the quality of learning. This is particularly the case for primary mathematics, as exhibited in different examinations. One of the issues with mathematics teaching and learning is the lack of formative assessment materials that teachers can utilise to diagnose students’ learning deficiencies and use as a basis for designing appropriate remedial teaching. Furthermore, there is a paucity of research on mathematics assessment at the primary level in Bangladesh that has addressed the issues related to quality of learning. It is the aim of this research to develop and validate assessment materials using the latest developments in psychometrics that can (a) be used for this purpose, and (b) serve as a model for further relevant work. Arithmetic word problem solving (AWPS) is an area within MPS that is often used to help children learn how to apply the formal mathematical knowledge and skills they acquire at school in real-life situations. Although AWPS plays an important role in understanding mathematics, it is an area where students experience great difficulty. It is a complex activity that requires a series of cognitive processes, but current assessment of AWPS in Bangladesh relies exclusively on giving a single score based on the solution being correct or incorrect. There is limited opportunity for understanding students’ difficulties in the component cognitive processes. In the global context, empirical studies exploring the cognitive processes involved in solving arithmetic word problems are scarce. In the Bangladeshi education context, no studies were found that focused on AWPS. This research endeavoured to develop assessment materials that will enable teachers to identify specific difficulties students face while solving AWPS. The research objectives were to (a) develop a theoretical framework for arithmetic word problem solving using an iterative process (as discussed by Wilson, 2005) by (b) constructing tasks aligned with the framework, (c) trying them out in both small- and large-scale studies, (d) investigating the results using qualitative and quantitative methods, and (e) evaluating the success of the process by investigating the validity and reliability of the results. Only addition and subtraction operations for AWPS were considered to limit the scope of the study. Based on review of existing literature, a framework for assessing AWPS was developed. This framework theorises five cognitive elements/strands of AWPS: comprehension of the problem, mapping conceptual relations to a numerical representation, selecting an appropriate strategy for solution, performing the calculations, and verifying the solution and communicating the results. The validity and reliability of the framework were investigated by developing and administering a test (with items representing the different processes). The test was administered in two phases in Bangladesh: a pilot study with 62 students (from one school) and a final study with 808 students (from nine schools). Grade 4 and grade 5 students were selected through convenience sampling for both iterations of data collection. Four different types of validity evidence (based on test content, response processes, internal structure, and relations with other variables) were collected and analysed. In addition to using test design and specification, an expert opinionnaire was used to collect validity evidence based on test content. Analysis of this data showed that overall, the experts’ association of strands with items agreed with what was hypothesised, that is, they nominated items for strands they were hypothesised to be representing in this research. Cognitive interviews were conducted with 20 respondents from three schools to collect validity evidence based on the response processes. Data from these cognitive interviews were analysed using a qualitative method called ‘reduction and synthesis’ (Miller et al., 2014) and showed that the items were mostly working as intended. An Item Response Theory (IRT) approach was adopted to evaluate the internal structure of the test. A dichotomous Rasch analysis was used. The analysis showed that the item difficulties were appropriately distributed, and that the item fit statistics suggested that almost all the items were within a reasonable range (weighted MNSQ 0.88 to 1.14) and did not show any extreme over-fit or under-fit. The fit t statistics for the items also indicated a good fit. Overall, the data showed good adherence to the Rasch model and can be considered to be demonstrating reasonably good evidence of validity in terms of internal structure of the test. Two types of data were collected with the aim of establishing validity arguments based on the AWPS test’s relation with other variables: (a) scores from six textbook items included in the test that was administered and (b) students’ test scores for Bangla and Mathematics from their schools. These scores were positively correlated with students’ performance on the test conducted in this study, and hence showed good evidence of validity in relation to other variables. Test reliability was calculated using both IRT and CTT measures. The WLE person separation reliability was 0.81 and the EAP/PV reliability was 0.82. The Cronbach’s Alpha value was 0.82. The findings of this study can inform the inclusion of appropriate assessment techniques in the primary mathematics curriculum in Bangladesh, and the development of professional development materials for teachers using the assessments. Teachers of mathematics could utilise the framework to design their own tests that would help them to diagnose students’ learning deficiencies and design appropriate remedial teaching. In addition, future research can replicate this study in different settings to refine the AWPS framework and use it in different contexts. Furthermore, the methodology used in this study could be used in future research to develop assessment materials for other areas of mathematics including other areas of problem solving (e.g., arithmetic word problem solving where other operations like multiplication or division are used).