Evaluating Learning in Times of Covid-19 – ICAN, a Civil Society Initiative

This post is also available in: Español 

Main Photo: Damián Buenvarón / Flickr / CC BY-NC 2.0

On November 10, the Education Program at the Inter-American Dialogue, Medición Independiente de Aprendizajes (MIA), and the Regional Office of Education for Latin America and the Caribbean of UNESCO Santiago, convened a virtual seminar to present ICAN (Common International Assessment of Basic Numeracy) of the Citizen Action Network for Learning (PAL͕), and discuss how to effectively, rapidly, and practically assess learning in a pandemic and long-term. The panelists shared their experiences on the current challenges and opportunities to evaluate learning in on-site contexts and during the pandemic in Latin America and the Caribbean.

This digital seminar featured Felipe Hevia de la Jara, researcher at CIESAS, co-responsible for the MIA-Mexico project and part of the PAL network; Carlos Henríquez Calderón, coordinator of UNESCO's Latin American Laboratory for the Evaluation of the Quality of Education (LLECE); and Blanca Heredia, director and professor-researcher of the Interdisciplinary Program on Educational Policy and Practice at the Center for Economic Research and Teaching (CIDE). The seminar was moderated by Ariel Fiszbein, director of the Education Program at the Inter-American Dialogue, and included the participation of Claudia Uribe Salazar, director of UNESCO Santiago's Regional Office for Education in Latin America. The event was attended by 292 participants.

First, Felipe Hevia presented ICAN, an open source assessment developed through collaboration among countries in the Southern Hemisphere and aligned with Sustainable Development Goals (SDG 4.1.1 (a)), serving to establish quick and replicable diagnostics for the entire region. Taking into account the importance of evaluating learning under different lenses that complement each other, Hevia explained the function of ICAN as an effective and easy-to-apply diagnostic tool, originating within civil society. Created before the pandemic to reduce lags in basic learning from the primary level, ICAN is an accessible, robust, flexible, user-friendly tool available in indigenous languages that measures mathematical learning through simple tasks organized into thematic blocks. The evaluation includes spatial orientation questions, simple data visualization, basic calculations, number recognition, and daily problem solving, among others. This diagnostic also has contextual surveys focused on children, parents, and members of the school community.

The Citizen Action Network for Learning (PAL), creator of ICAN, was established in 2015 as a Southern Hemisphere alliance of organizations from three continents. It currently has members in 14 countries and seeks to determine the basic learning of children and adolescents in reading and mathematics. In 2019, ICAN covered 13 countries in 3 continents, evaluating more than 20,000 children. The results indicated significant dropout levels and lags in basic math learning outcomes. In countries such as Mexico, children entering secondary school are lagging behind in fundamental knowledge and skills in mathematics. According to Hevia, some of the main uses of ICAN are: monitoring of sustainable development objectives in municipalities, states, and countries; monitoring of children, adolescents, and youth inside and outside of schools; guidelines for educational policies in arithmetic; and research on equity and factors associated with educational achievement. Hevia detailed the benefits of having a diagnosis that can be applied individually in an oral format, as well as how its results maximize the performance of each student. In addition, in the context of a pandemic, it is much safer to apply this protocol from a safe distance in the home than to ask children to go out for testing.

Carlos Henríquez highlighted the importance of creating public-private alliances and expanding national and international networks so that different types of diagnostics can work in synergy towards the same goal. At the Latin American Laboratory for Assessment of the Quality of Education (LLECE), Henríquez has focused on measuring learning achievements through the Regional Comparative and Explanatory Study, which presented its fourth version, ERCE, in 2019. The main goal of these evaluations is to monitor the quality of education in the region in order to inform debate and guide decision-making. LLECE evaluations showed that there are significant gaps in basic learning in mathematics and literacy, as well as gender and indigenous gaps. Henríquez agreed with Hevia on the importance of focusing not only on access but also on students' learning trajectories. To achieve this assessment, the role of summative, diagnostic, and formative evaluation must be highlighted as a key complement to national and international tests.

Henríquez recognized the inclusive nature of ICAN as one of its best attributes, since its personalized and flexible evaluation methods allow for a deep understanding of the magnitude of learning gaps, and create opportunities to develop basic skills. He also welcomed the unprecedented inclusion of children outside of the formal school system. Henríquez highlighted the need to measure trajectories of early learning levels through early diagnostics such as ICAN, since identifying learning lags in the initial stages of education can have significant effects in the future of millions of students in higher levels. "It is not possible to improve what is unknown," said Henríquez, who maintained that evaluation should not be the end, but a means to connect teaching with learning. Finally, he emphasized the value of collaborations between civil society, governments, and non-governmental organizations to fulfill the 2020-2030 agenda for sustainable development.

Blanca Heredia discussed the efforts of both Hevia and Henríquez in enriching assessment with a focus on students' well-being and learning trajectories, rather than only focus on enrollment. Heredia highlighted the value of ICAN as a diagnostic tool that integrates the community, including teachers and parents, directly. Despite the undeniable importance of national and international tests, the panelist highlighted the importance of assessment complements that involve communities in identifying learning trajectories and acting in relation to where each child is in terms of fundamental knowledge and skills. Finally, Heredia spoke of the potential of a test such as ICAN in a pandemic context, since most countries in Latin America do not yet know how extensive the learning lags will be for millions of students receiving remote education.

The event concluded with a brief round of questions from the audience about the role evaluations can play in recovering dramatic learning and human capital losses in the region, one of the most real and difficult to measure effects in the current context. The panelists agreed on the importance of having fast and effective feedback that allows linking evaluations with concrete improvements. They also emphasized the importance of complementing national and international assessments with innovative instruments such as ICAN, a diagnosis that adds value to the current educational evaluation landscape. Finally, the need to make visible and help students who are clearly falling behind in and out of the educational system was highlighted.

WATCH THE RECORDING OF THE EVENT: