Investigation of cross-country differences in student performance in standardized tests: the role of modern and traditional teaching methods
| Thesis title in Czech: | Analýza rozdílů ve výsledcích žáků ve standardizovaných testech napříč zeměmi: vliv moderních a tradičních metod výuky |
|---|---|
| Thesis title in English: | Investigation of cross-country differences in student performance in standardized tests: the role of modern and traditional teaching methods |
| Key words: | moderní metody výuky, tradiční metody výuky, standardizované testy TIMSS, mezinárodní srovnání, metoda within-student across-subject, výsledky žáků |
| English key words: | modern teaching practices, traditional teaching practices, TIMSS standardized tests, international comparison, within-student across-subject approach, student achievement |
| Academic year of topic announcement: | 2018/2019 |
| Thesis type: | diploma thesis |
| Thesis language: | angličtina |
| Department: | Institute of Economic Studies (23-IES) |
| Supervisor: | Mgr. Barbara Pertold-Gebicka, M.A., Ph.D. |
| Author: | hidden - assigned by the advisor |
| Date of registration: | 19.11.2019 |
| Date of assignment: | 19.11.2019 |
| Date and time of defence: | 03.02.2021 09:00 |
| Date of electronic submission: | 04.01.2021 |
| Date of proceeded defence: | 03.02.2021 |
| Opponents: | doc. PhDr. Michal Bauer, Ph.D. |
| URKUND check: | |
| References |
| Bietenbeck, J. 2014. “Teaching Practices and Cognitive Skills.” Labour Economics 30: 143–153. doi:10.1016/j.labeco.2014.03.002.
Hanushek, E. A., & Woessmann, L. 2011. “The economics of international differences in educational achievement”. Handbook of the Economics of Education, 3: 89-200. IEA 2009. “TIMSS“. https://www.iea.nl/index.php/data-tools/repository/timss. Accessed 17 Dec 2019. Klaveren, C. V. 2011. “Lecturing Style Teaching and Student Performance.” Economics of Education Review 30 (4): 729–739. doi:10.1016/j.econedurev.2010.08.007 Korbel, V., & Paulus, M. 2018. “Do teaching practices impact socio-emotional skills?“ Education Economics, 26(4): 337-355. https://doi.org/10.1080/09645292.2018.1460320. Lavy, V. 2015. “What Makes an Effective Teacher? Quasi-experimental Evidence.” CESifo Economic Studies. Martin MO, Mullis IV, Foy P. TIMSS 2007. “International mathematics report. Findings from IEA's trends in international mathematics and science study at the fourth and eighth grades.“ Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College, 2008. http://timss.bc.edu/TIMSS2007/PDF/TIMSS2007_InternationalMathematicsReport.pdf Woessmann, L. 2016. “The importance of school systems: Evidence from international differences in student achievement.“ Journal of Economic Perspectives, 30(3): 3-32. |
| Preliminary scope of work |
| Motivation:
Children all around the world spend many years in schools. Education can promote or hinder development of various skills. As teachers play a key role in educating pupils, current research has focused on investigation of teaching practices used in the classroom. A recent study by Bienteck (2014) examines modern vs. traditional teaching methods in the US. Similarly, Korbel and Paulus (2018) look at impact of modern teaching methods vs. traditional ones on student test results in the Czech Republic, and more specifically, on socio-emotional skills that are very important in later stages of our life. They found that modern teaching practices have positive effect on socio-emotional skills while having no adverse effect on test scores, but that standard practices have no impact on socio-emotional skills or test scores in the Czech Republic. That can be translated that changing the methods of teaching towards more modern methods can have positive impact on pupils’ socio-emotional skills without affecting the test scores. However, little is known about the impact across countries and the evidence is highly inconclusive. For instance, Klaveren (2011) finds that different teaching methods do not impact student performance in the Netherlands. Moreover, Lavy (2015) suggests that the teaching method impacts differently students from different backgrounds in Israel. Inspired by Klaveren (2011) and Lavy (2015), we can assume significant differences across countries. This thesis aims to contribute to the existing body of literature and rising research on student performance in standardized tests and to identify and potentially explain cross-country differences in the extent to which modern teaching practices are used and their effect on students. Standardized tests such as TIMSS 2007 provide public information about student performance with the aim to improve learning (Martin, Mullis and Foy, 2007). The nature of data allows us to explore cross-country variation in educational policies. We will try to identify factors predicting differences in modern vs. traditional teaching practices across countries and their impact on socio-emotional skills and test results of secondary school students across countries. Study by Woessmann (2016) suggests that these differences may be explained by cultural factors, society setting, education system or institutional structures. Hypotheses: Based on the motivation part, the hypotheses are as follows. Hypothesis 1 is that the causal effect of modern teaching methods on students’ performance differs across countries (H1). And Hypothesis 2 that the causal effect of modern teaching methods on student’s socio-emotional skills differs across countries (H2). We will test these hypotheses using the TIMSS 2007 data. If we find significant cross-country differences in the relationship between teaching methods and students’ outcomes, we will investigate them further. Hypothesis 3 is why modern teaching methods have impact on test scores in some countries while not in other ones (H3). And similarly, Hypothesis 4 for socio-emotional skills (H4). In the regression, the explanatory variables of our interest will be country characteristics. These may include cultural factors, societal setting, educational system or institutional structures (Woessmann, 2016). Also, some societies may valuate achievement differently or have different educational systems with (or without) exit exams, early tracking, mandatory nursery school attendance, with a different starting age, a different proportion of students attending private schools, motivate teachers in a dissimilar way, etc. (Hanushek and Woessmann, 2011). 1. Hypothesis #1: The causal effect of modern teaching methods on student’s performance differ across countries. 2. Hypothesis #2: The causal effect of modern teaching methods on student’s socio-emotional skills differ across countries 3. Hypothesis #3: Why do modern teaching methods have impact on test scores in some countries and not in another countries? 4. Hypothesis #4: Why modern teaching methods promote socio-emotional skills in some countries and not in another countries? Methodology: In this thesis, our analysis will take place in two stages. First, we will use the student-fixed effect model (i.e. within-student between-subject strategy) for each country, in line with other studies of this type, including Bietenbeck (2014) and Korbel and Paulus (2018). The outcome variable (test score and self-confidence) of a student in specific subject taught by specific teacher will be regressed on standard vs. modern teaching method, teachers’ characteristics, and student fixed-effects. The equation will take the following form. Test score_ijtc = beta_jc + beta_1c * standard_ijtc + beta_2c * modern_ijtc + beta_3c * Z_jtc + a_ic + u_ijtc Where index i stands for a student, j for a subject, t for a teacher and index c for a given country. We will estimate betas for each country with an index c which then will enter in the second stage as our dependent variable. In the second stage, we will take a full list of beta_1c and beta_2c estimates and regress these on country characteristics using weights proportional to the precision with which betas are estimated. The goal will be to identify factors explaining differences in student performance in standardized tests among the selected countries, with the use of weighted LS method. Data from the Trends in International Mathematics and Science Study (TIMSS) in 2007 that was collected for fourth and eighth graders in 59 countries will be used. We use the 2007 TIMSS wave because it is the last one including students’ reports on teaching practices. The data is publicly available on the survey’s website. The data provides information on students’ test scores, other students’ and teachers’ characteristics along with teaching methods collected through the self-reported questionnaires. Moreover, the students’ questionnaires contain data on teaching practices based on which we will construct aggregate class indexes on modern vs. traditional teaching practices in line with Bietenbeck (2014) and Korbel and Paulus (2018). Students were asked to answer on a scale whether a particular teaching method was used in the classroom, separately for math and science. Modern methods are defined as working in groups, having discussions, and using what they learn in daily lives unlike traditional methods which include rote learning and memorizing. We will work with econometric models including the Fixed Effect model and the weighted LS estimation method using the R software. Expected Contribution: This study aims to contribute to the existing literature and bring new findings about the factors that predict differences in test results and socio-emotional skills among lower secondary school students across countries. We will especially focus on the role of modern vs. traditional teaching practices. The main contribution of this thesis is international comparison (i.e. across-countries). To my best knowledge, there was no similar study investigating cross-country differences. Differences in educational systems and attitudes towards education are substantial across countries to such an extent that it seems impossible to set and apply one-size-fits-all teaching method all around the world. Therefore, we investigate differences in individual teaching methods and how these methods resonate with educational systems and social standards. Thus, this study would like to help countries to understand to what extent they should apply modern teaching methods given specific (education policies) characteristics. Results from this work can be useful to policymakers who set educational policies that encourage socio-emotional skills on international levels, as well as to better understand the role of modern teaching methods in education systems. Outline: The structure of my thesis will be as follows. 1. Introduction - I will introduce the topic and provide motivation why this topic is relevant to be studied as well as I will mention potential contribution of my work to current research 2. Literature review - In this part, a detailed overview of the literature and existing body of research will be provided, including the most relevant studies critically analysing them including their methods used, results found and limitations 3. Data - I will describe my dataset in detail and explain how the data was collected. Furthermore, I will briefly describe how I transform the data into the format needed for the model (e.g. construct modern vs. traditional teaching method indexes) 4. Empirical analysis – In this part, my model(s) and estimation methods that will be used are introduced. Also, I will explain in detail the fixed effect estimation, its advantages and relevance. 5. Results - I will present and discuss my regressions results and robustness checks. 6. Conclusion - In this section, my findings will be summarized, including possible future research avenues and implications of my results for policymakers and educational sector |
| Preliminary scope of work in English |
| Motivation:
Children all around the world spend many years in schools. Education can promote or hinder development of various skills. As teachers play a key role in educating pupils, current research has focused on investigation of teaching practices used in the classroom. A recent study by Bienteck (2014) examines modern vs. traditional teaching methods in the US. Similarly, Korbel and Paulus (2018) look at impact of modern teaching methods vs. traditional ones on student test results in the Czech Republic, and more specifically, on socio-emotional skills that are very important in later stages of our life. They found that modern teaching practices have positive effect on socio-emotional skills while having no adverse effect on test scores, but that standard practices have no impact on socio-emotional skills or test scores in the Czech Republic. That can be translated that changing the methods of teaching towards more modern methods can have positive impact on pupils’ socio-emotional skills without affecting the test scores. However, little is known about the impact across countries and the evidence is highly inconclusive. For instance, Klaveren (2011) finds that different teaching methods do not impact student performance in the Netherlands. Moreover, Lavy (2015) suggests that the teaching method impacts differently students from different backgrounds in Israel. Inspired by Klaveren (2011) and Lavy (2015), we can assume significant differences across countries. This thesis aims to contribute to the existing body of literature and rising research on student performance in standardized tests and to identify and potentially explain cross-country differences in the extent to which modern teaching practices are used and their effect on students. Standardized tests such as TIMSS 2007 provide public information about student performance with the aim to improve learning (Martin, Mullis and Foy, 2007). The nature of data allows us to explore cross-country variation in educational policies. We will try to identify factors predicting differences in modern vs. traditional teaching practices across countries and their impact on socio-emotional skills and test results of secondary school students across countries. Study by Woessmann (2016) suggests that these differences may be explained by cultural factors, society setting, education system or institutional structures. Hypotheses: Based on the motivation part, the hypotheses are as follows. Hypothesis 1 is that the causal effect of modern teaching methods on students’ performance differs across countries (H1). And Hypothesis 2 that the causal effect of modern teaching methods on student’s socio-emotional skills differs across countries (H2). We will test these hypotheses using the TIMSS 2007 data. If we find significant cross-country differences in the relationship between teaching methods and students’ outcomes, we will investigate them further. Hypothesis 3 is why modern teaching methods have impact on test scores in some countries while not in other ones (H3). And similarly, Hypothesis 4 for socio-emotional skills (H4). In the regression, the explanatory variables of our interest will be country characteristics. These may include cultural factors, societal setting, educational system or institutional structures (Woessmann, 2016). Also, some societies may valuate achievement differently or have different educational systems with (or without) exit exams, early tracking, mandatory nursery school attendance, with a different starting age, a different proportion of students attending private schools, motivate teachers in a dissimilar way, etc. (Hanushek and Woessmann, 2011). 1. Hypothesis #1: The causal effect of modern teaching methods on student’s performance differ across countries. 2. Hypothesis #2: The causal effect of modern teaching methods on student’s socio-emotional skills differ across countries 3. Hypothesis #3: Why do modern teaching methods have impact on test scores in some countries and not in another countries? 4. Hypothesis #4: Why modern teaching methods promote socio-emotional skills in some countries and not in another countries? Methodology: In this thesis, our analysis will take place in two stages. First, we will use the student-fixed effect model (i.e. within-student between-subject strategy) for each country, in line with other studies of this type, including Bietenbeck (2014) and Korbel and Paulus (2018). The outcome variable (test score and self-confidence) of a student in specific subject taught by specific teacher will be regressed on standard vs. modern teaching method, teachers’ characteristics, and student fixed-effects. The equation will take the following form. Test score_ijtc = beta_jc + beta_1c * standard_ijtc + beta_2c * modern_ijtc + beta_3c * Z_jtc + a_ic + u_ijtc Where index i stands for a student, j for a subject, t for a teacher and index c for a given country. We will estimate betas for each country with an index c which then will enter in the second stage as our dependent variable. In the second stage, we will take a full list of beta_1c and beta_2c estimates and regress these on country characteristics using weights proportional to the precision with which betas are estimated. The goal will be to identify factors explaining differences in student performance in standardized tests among the selected countries, with the use of weighted LS method. Data from the Trends in International Mathematics and Science Study (TIMSS) in 2007 that was collected for fourth and eighth graders in 59 countries will be used. We use the 2007 TIMSS wave because it is the last one including students’ reports on teaching practices. The data is publicly available on the survey’s website. The data provides information on students’ test scores, other students’ and teachers’ characteristics along with teaching methods collected through the self-reported questionnaires. Moreover, the students’ questionnaires contain data on teaching practices based on which we will construct aggregate class indexes on modern vs. traditional teaching practices in line with Bietenbeck (2014) and Korbel and Paulus (2018). Students were asked to answer on a scale whether a particular teaching method was used in the classroom, separately for math and science. Modern methods are defined as working in groups, having discussions, and using what they learn in daily lives unlike traditional methods which include rote learning and memorizing. We will work with econometric models including the Fixed Effect model and the weighted LS estimation method using the R software. Expected Contribution: This study aims to contribute to the existing literature and bring new findings about the factors that predict differences in test results and socio-emotional skills among lower secondary school students across countries. We will especially focus on the role of modern vs. traditional teaching practices. The main contribution of this thesis is international comparison (i.e. across-countries). To my best knowledge, there was no similar study investigating cross-country differences. Differences in educational systems and attitudes towards education are substantial across countries to such an extent that it seems impossible to set and apply one-size-fits-all teaching method all around the world. Therefore, we investigate differences in individual teaching methods and how these methods resonate with educational systems and social standards. Thus, this study would like to help countries to understand to what extent they should apply modern teaching methods given specific (education policies) characteristics. Results from this work can be useful to policymakers who set educational policies that encourage socio-emotional skills on international levels, as well as to better understand the role of modern teaching methods in education systems. Outline: The structure of my thesis will be as follows. 1. Introduction - I will introduce the topic and provide motivation why this topic is relevant to be studied as well as I will mention potential contribution of my work to current research 2. Literature review - In this part, a detailed overview of the literature and existing body of research will be provided, including the most relevant studies critically analysing them including their methods used, results found and limitations 3. Data - I will describe my dataset in detail and explain how the data was collected. Furthermore, I will briefly describe how I transform the data into the format needed for the model (e.g. construct modern vs. traditional teaching method indexes) 4. Empirical analysis – In this part, my model(s) and estimation methods that will be used are introduced. Also, I will explain in detail the fixed effect estimation, its advantages and relevance. 5. Results - I will present and discuss my regressions results and robustness checks. 6. Conclusion - In this section, my findings will be summarized, including possible future research avenues and implications of my results for policymakers and educational sector |