This post is also available in: Spanish
The following article by Ariel Fiszbein was originally written for Bastión Digital, which has given the Inter-American Dialogue permission to reproduce the text on the PREAL blog. To read the original article in Spanish, click here.
A basic principle of good management is that if clear objectives are not set and we do not measure to what extent they are being met, it is very difficult to obtain results. In the case of education this entails the existence of learning goals and assessment systems.
These issues have been central in discussions of educational policy in Latin America for quite some time. In 2006 PREAL published a report on learning assessment systems in Latin America, covering both the setting of clear learning goals (standards) as well as the measurement of learning achievements (assessment). Beginning in the nineties, almost all countries in the region began to establish national learning assessment systems – in some cases moving very quickly as part of reform programs, and in others more slowly as part of specific projects.
The report recognized that significant progress had been made in establishing assessment systems, but also showed major differences in the level of development of those systems, depending on the characteristics of the units responsible and the political context in which they operated. The regional perspective suggested that the most stable institutional arrangements were those operating outside of the organizational structure of the ministries of education through institutes with greater administrative and technical autonomy. The main weaknesses were found with regard to the dissemination and use of results. In particular, the report showed that “the general public’s limited participation in assessment processes has not been conducive to any significant rise in demand for better quality.”
A decade after the report was published, we do not have a current overview of the state of assessment systems in the region. For example, although the 2021 Education Goals agreed upon by members of the Organization of Ibero-American States (OEI) include a specific goal to ‘Strengthen systems of evaluation in each of the countries,’ the annual tracking report of OEI explicitly excludes it from its measurement efforts.
This analytical gap contrasts with the presence of a number of significant changes in the institutional structure of the entities in charge of assessment in countries such as Ecuador (where the INEVAL was created in 2012) or Mexico (where the INEE began operating as an autonomous entity 2013), or in their operation in countries such as Brazil (where the index of basic education development (IDEB), created in 2007 is published annually at the municipal level) or Colombia (where you can access online the results of the SABER tests at the establishment level). It is likely that a more updated report would show significant progress in some countries—and stagnation in others—while offering us new lessons about the challenges identified a decade ago.
In particular, what should be the process by which standards and competencies that govern the education system are set, and who should participate in it? For example, what role can and should business organizations have in the definition of standards and competencies in technical education? How can technocratic views and expectations of citizens be reconciled in the formulation of standards and competencies? How can standards be defined in ways that are understandable for users of services and yield the social mobilization necessary to achieve them?
Setting standards and learning goals is critical not only for the design and monitoring of policies, but also to achieve the commitment, involvement, and motivation of all stakeholders in the educational process: students and their families, teachers and principals, administrators and policymakers at national, regional and local levels, and employers.
Recent years have seen a growing number of participatory experiences in setting goals and objectives for the education system, of agreements/pacts on educational policy, and of schemes for monitoring compliance with them (see Table 1). Initiatives by civil society organizations have been instrumental in this process—for example, “Todos pela Educação” in Brazil. In general, these include learning goals together with targets for coverage and public financing. These experiences vary in the degree of formality and official participation. Some of these generate a high level of social energy, although we do not know if that energy is maintained over time and whether these participatory schemes serve in the medium and long term as a mechanism to influence policies and their implementation.
Table 1: Education Pacts and Goals
|Brazil||Plano Nacional de Educaçao||Official||Yes|
|Colombia||Gran Acuerdo por la Educación||Multisectoral||Yes|
|Ecuador||Plan Decenal de Educación||Multisectoral||No|
|Guatemala||Juntos por la Educación||Civic||Yes|
|Panama||Unidos por la Educación||Civic||Yes|
|Dominican Republic||Pacto Educativo||Multisectoral||No|
How can we ensure the credibility of the evaluation function and the practical usefulness of the information generated for stakeholders in the education system (teachers, administrators, students, parents, etc.)? In the last decade, particularly in recent years, there was a trend in this direction (e.g. in the case of Ecuador and Mexico above, or the very recent case of the City of Buenos Aires). While these processes are not exempt from conflicts and difficulties, the logic of separating the evaluation functions from the management of the system and to protect them from political influence seems to be prevailing.
The practical utility of the information generated by assessment systems strongly depends on the rules governing access to it and approach followed to communicate results. Regarding learning standards, it is critical that they are communicated in such a way as to convene and mobilize the efforts of teachers, families and society as a whole. For example, overcoming the strong deficiencies in reading in the early years of primary school—perhaps the most significant cause of school failure later on—requires that teachers and parents have clear expectations about what a second grader should be able to read. Similarly, communication of results on achievement tests is a key factor to improving both teacher classroom practices and the management at the school level, as well as empowering parents in their efforts to achieve better education for their children.
Finally, what should be the role of schemes of internationalization of standards, competencies and evaluation of results? Participation in international tests has the huge benefit of giving a broader perspective of educational assessment. While the purpose of these is not to create a ‘horse race,’ the power to compare oneself to one’s neighbors, more advanced countries one aspires to emulate, or countries with which one competes in international markets is undoubtedly a very attractive aspect of these tests. While Latin American countries participate in international tests (Table 2), this participation is still not massive or consistent over time. Despite the increasing attention that civil society and the media lend to the results of these tests, as evidenced by the intense analysis of the results of PISA in late 2013 – there still exists resistance of many governments to participate systematically.
Table 2: Participation in International Tests
|TERCE||Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Paraguay, Dominican Republic, and Uruguay|
|PISA 2012||Argentina, Brazil, Chile, Colombia, Costa Rica, Mexico, Peru, and Uruguay|
|PIRLS 2011||Colombia and Honduras|
|ICCS 2009||Chile, Colombia, Guatemala, Mexico, Nicaragua, Paraguay, and Dominican Republic|
|Pisa for Development||Ecuador, Guatemala, and Paraguay|
Finding effective participatory mechanisms to define standards and competencies, strengthening institutions in charge of evaluation, and advancing the internationalization of standards and results measurement, are potentially key elements of a reform strategy. However, resistance to these ideas remains strong, if not at a rhetorical level, then in practice. In part this resistance may be caused by the inertia of old practices. Sharing and comparing how different countries have been breaking the inertia (e.g. increasingly participating in international tests) should weaken this resistance.
i Guillermo Ferrer: Educational Assessment Systems in Latin America: Current Practice and Future Challenges, PREAL (2006).
ii Ferrer (2006) p.44
iii OEI, Miradas sobre la educación en Iberoamérica: Avances en las metas educativas 2021 (2014)