This post is also available in: Spanish
The Latin American Network for Education (REDUCA), a group of non-governmental organizations aimed at improving education quality, along with its Argentinian chapter Proyecto Educar 2050, has recently released a new report on school policies, classroom practices, and student behavior factors associated with student achievement.
The report, titled “Clues made in Latin America”, uses the Third Regional Comparative and Explanatory Study (TERCE) database, which was published in 2013. TERCE was coordinated by the Latin American Laboratory of Assessment of Education Quality (LLECE), a group of assessment agencies in the region, based in the regional UNESCO office. This study, in which 15 Latin-American countries participated, assessed the performance of third and six graders in the areas of mathematics, reading, and writing.
This is the sixth report in a series of publications sponsored by Proyecto Educar 2050 to make the results of international, regional, and national student assessments available for the Argentinian audience. A summary of TERCE results can be found in the previous report “Improvements and challenges”.
Over the last few years, it may seem that student assessment results have been published almost every month with similar conclusions. Hence, it is key to highlight what makes this report different from others.
This is the first report to report the results of the TERCE surveys of principals, teachers, and students of the region. This is important because, although LLECE collected these data, the results of these surveys were not published in a user-friendly format and, thus, were not accessible to the everyday citizen (who usually has neither the time nor the technical ability to analyze them).
This is also important, because although LLECE published a report on the relation between school policies, classroom practices and student behaviors, and school performance within every country (e.g., “do schools with more experienced principals in country X have better results?”), it did not analyze the relation between factors and performance between countries (if country X has more experienced principals on average than country Y, does country X have better results?).
This publication is key because it is the only source of comparable data on principals, teachers and students between countries in the region. The data from the Program for International Student Assessment (PISA) data only includes eight Latin American countries. In addition, many countries in the region do not collect such data, others do not publish them, and out of those that do publish them, their data may not be comparable with those of other countries. That said, TERCE provides us with a unique opportunity to inform policy debates that would otherwise typically rely solely on the opinion of experts.
What did we find?
The report includes two types of findings—those that describe the school policies, classroom practices, and student behaviors of the region’s education systems, and those that analyze whether these factors are associated with better performance.
First, the report shows that Chile, the best performing country in the region, is also the most successful in providing basic inputs for student learning. It is among the countries with the highest proportion of students in full time schools, computers with Internet, textbooks, and principals and teachers with teaching degrees. Yet, in general, countries with more of these basic inputs do not necessarily perform better. The relationship between these inputs and students’ scores tends to be weak and countries with better basic services tend to have better economic performance.
Second, the report reveals that the profiles of principals and teachers vary considerably across countries. With the exception of Argentina and Panama, nearly all principals have a teaching degree, but the percentage that attained a tertiary or higher education varies greatly, from 10% in Uruguay, to almost everyone in Colombia. Similarly, the years of experience of teachers in the Dominican Republic and Guatemala are almost half of those in Chile and Peru. Yet, these indicators that are typically seen as proxies of principal and teacher “quality” do not show a strong relationship with better performance.
Third, most students in the region attend schools with low work flexibility. Most have tenured principals and teachers. With the exception of Chile, Paraguay, Brazil and Peru, most students attend schools in which the principal has no control over the hiring or firing teachers. And, with the exception of Chile, these students go to schools in which the principal does not determine the wages of teachers. This flexibility that some consider necessary to improve schools, however, is not associated with better results.
Fourth, only a few countries in the region assess almost all their students and teachers. Only in Colombia, Mexico, Chile and Peru, more than 90% of students attend schools that participate in standardized tests. Similarly, only in Paraguay, Nicaragua, Ecuador and Mexico, more than 80% of students attend schools where teachers are assessed. Nonetheless, countries vary considerably in the use of these assessments. For example, while 83% of third grade Mexican students attend schools where results are used for teacher assessments, only 51% of their peers in Peru are in the same situation. Similarly, 56% of third graders in the region attend schools where teacher assessments have no consequences.
Finally, a large percentage of the region’s teachers are frequently late to work or absent. More than half of third and six graders in Panama and Argentina state that their teachers miss school regularly. Similarly, one third of the same group of students in the region report that their teachers either come late to class or leave early. The percentage of students admitting to miss class at least once a week is lower (around 10% third graders and 8% six graders); however, there is a close relation between the percentage of students admitting to miss class and country performance.
What can we conclude?
The data of TERCE surveys on principals, teachers, and students provide valuable information that helps us gain a better understanding of schools in the region. Yet, the data is solely based on what these actors say; not necessarily on what is happening in schools or on education policies. This way of gathering information is useful in understanding how the implementation of education policies varies within countries, but it is a complement to, not a substitute for, studies of policy implementation.
Secondly, the data allows us to describe the practices of the best-performing countries in the region; but they cannot tell us whether these practices are the reason for these countries’ performance. This may come as an obvious statement, but it is often ignored. In fact, TERCE’s own report on “associated factors” recommends to end grade repetition based on the negative relation between the percentage of repeating students and their performance. Yet, students who repeat grades are different from those who do not in many ways not accounted for in this assessment; the association we observe does not mean grade repetition causes poor performance. TERCE was not designed to make these types of policy recommendations and using it for this purpose is wrong.
Third, it is important to conduct impact evaluations to explore more deeply the “clues” TERCE offers. For example, why are school inputs, teacher degrees, and work flexibility not associated with better performance? It is fundamental to design rigorous impact evaluations that allow us to analyze the causal relationships between these (and other) factors and student learning.
Forth, this report invites us to reconsider the pervasive notion that there is only one way to achieve good results in the region. As the report shows, top-performing countries in TERCE like Chile and Costa Rica, or rapidly-improving systems like Peru and Ecuador have different education policies. These results should help us question the popular notions of “what good education systems do” and should prompt us to consider the possibility that there are multiple paths towards education improvement.
Finally, we must remember that TERCE offers a regional picture of the performance of Latin American systems; it is not a substitute for an international perspective. Countries like Chile and Costa Rica, which have the best scores in TERCE, still rank in the bottom third of countries participating in PISA. It is essential not to confuse “better performance in the region” with “good performance” or to assume that the characteristics of the best-performing education systems in TERCE are those of successful systems.
Manuel Álvarez Trongé is president of Proyecto Educar 2050. Alejandro J. Ganimian is an advisory member of Proyecto Educar 2050 board and a non-resident fellow of the Inter-American Dialogue.