Disruptions to Schooling and the Need for Recovery

Silvia Montoya, Director UNESCO Institute for Statistics and Martin Gustafssson, Research on Socio-Economic Policy (ReSEP), University of Stellenbosch

The 2020 learning losses equal the gains made in the last 20 years

Some numbers to retain:

  1. In many developing countries the percentage of children considered proficient was increasing by two percentage points a year before COVID-19
  2. COVID-19 school disruptions have caused learning losses equal to all of the learning gains in the last two decades
  3. 101 million children and youth from Grades 1 to 8 will fall below the minimum proficiency level after 2020 [note]
  4. For each month of no or little contact between teacher and learner two months of learning were lost
  5. Recovery could occur by 2024, but only if exceptional efforts are devoted to the task through remedial and catch-up strategies

The effects of the pandemic on children have been multifaceted and for too many, devastating. Before the pandemic, the data suggested that learning proficiency was gradually improving, though not fast enough to reach the Sustainable Development Goal 4 (SDG 4) objective of all children and young people being minimally proficient in reading and mathematics by 2030. Yet was progress was occurring. A new report by the UNESCO Institute for Statistics (UIS) examines the learning losses associated with school closures and considers how interventions may help children catch up.

In many developing countries the percentage of children considered proficient was increasing by two percentage points a year[1]. To provide an idea of the global trend, among the 130 million children of the age corresponding to Grade 3, only 73 million could read proficiently before the pandemic, but this was increasing by 700,000 a year. Each year roughly 700,000 additional children were proficient, compared to the previous year[2]. And this is counting just one school grade. This was mainly because learning improved in schools, but to some extent better school participation rates played a role.

Recovering from COVID-19 learning losses will depend on catch-up strategies

The pandemic brought with it school closures of a scale never previously seen. At the most serious point, around March and April 2020, around 95% of children were not attending school. It has been estimated that by early November, the world’s learners had lost between 41% and 68% of the contact schooling they should have received in 2020. It has been difficult to establish precise figures mainly because once schools re-opened many only allowed learners to attend on a rotational basis[3].

Continue reading

Building Back Better After COVID-19: The Importance of Tracking Learning Inequality

João Pedro Azevedo, Lead Economist, Education Global Practice, World Bank Group and Silvia Montoya, Director, UNESCO Institute for Statistics (UIS)

Our choice of a measure shapes our understanding of the size and nature of a problem. In a recent blog we discuss why the learning poverty measure is well suited to monitor the educational impacts of COVID-19. In this blog, we discuss two complementary concepts: the learning poverty gap and learning poverty severity, to look at distributions among the learning poor and measure how these changes affect learning inequality. The UNESCO Institute for Statistics (UIS) and the World Bank provide data for both of these concepts. Data for many key SDG 4 indicators were updated in the March data refresh. The UIS is also collecting data on the national response to COVID-19 on education, equity and inclusion.

Understanding changes in learning inequality through the learning poverty gap, learning poverty severity and minimum proficiency

While learning poverty is a simple concept to grasp, this indicator alone does not provide a picture of the learning level and distribution of learning among those below the minimum proficiency level (MPL). Because learning poverty is a headcount ratio, estimates treat all students below the minimum proficiency level as being equally learning deprived. It also does not reflect improvements in learning below the MPL threshold, which can fail to create compatible incentives as it may miss progress in foundational subskills critical for developing reading proficiency, for example, knowledge of spoken words and how to use them, hearing and making the sounds of words, mapping sounds to letters and letters to sounds while learning letter names, among others, as described in the reading rainbow (Figure 1). Understanding the heterogeneity among the learning poor is critical to combat learning poverty as children who do not master these subskills in early primary grades remain unable to read with comprehension.

Figure 1:  SDG 4.1 framework and reading rainbow of literacy subskills

Figure 2:  Distributional sensitive measure of learning deprivation

Continue reading

Cómo el marco del ODS 4.1.1 y la pobreza en el aprendizaje pueden ayudar a los países a dirigir su respuesta a la COVID-19 en materia de política educativa

João Pedro Azevedo, economista jefe, Education Global Practice, Grupo Banco Mundial y Silvia Montoya, Directora, Instituto de Estadística de la UNESCO (UIS)

La mayoría de los gobiernos y asociados para el desarrollo están trabajando en la identificación, protección y apoyo del aprendizaje de los miembros más vulnerables de la generación COVID-19. En este blog, analizamos cómo el marco del ODS 4.1.1 y el concepto de pobreza en el aprendizaje son herramientas útiles para ayudar a los países a entender y corregir los efectos de la COVID-19 en la escolarización y el aprendizaje.

Del nivel mínimo de competencias a una medición de la privación del aprendizaje

En octubre de 2018, la comunidad internacional acordó ser prudente en la utilización de un estándar global para el seguimiento del progreso en el aprendizaje de los estudiantes. El nivel mínimo de competencias (NMC)­ acordado a través de la Alianza Global para el Seguimiento del Aprendizaje (AGSA) ofrece un punto de referencia único para ayudar a los países y los asociados para el desarrollo a trabajar juntos para monitorear y mejorar el aprendizaje de aquellos estudiantes que se están quedando atrás. La pantalla de visualización interactiva que se muestra a continuación (Figura 1) permite explorar con el control deslizante los datos utilizados para monitorear este ODS, utilizando tanto el NMC de la AGSA como diferentes niveles mínimos de competencia.

Figura 1. Ejemplo de cómo el ODS 4.1.1 puede utilizarse para poner el foco en los estudiantes que están por debajo del nivel mínimo de competencias (NMC)

La pobreza en el aprendizaje: un indicador multidimensional para el sector educativo

En octubre de 2019, el Banco Mundial y el Instituto de Estadística de la UNESCO (UIS) lanzaron un nuevo indicador multidimensional llamado pobreza en el aprendizaje. Se basa en la idea de que todos los niños deberían estar escolarizados y ser capaces de leer un texto apropiado para su edad a los 10 años[1]. Esta formulación refleja cuál es su aspiración y sirve como indicador de alerta temprana del Objetivo de Desarrollo Sostenible 4 (ODS 4), de que todos los niños deben estar escolarizados y aprendiendo[2], y se basa en dos privaciones.

Continue reading

Comment le cadre de l’ODD 4.1.1 et le concept de pauvreté des apprentissages peuvent-ils aider les pays à orienter leur politique d’éducation en réponse à la COVID-19

João Pedro Azevedo, économiste principal, Pôle mondial d’expertise en éducation, Groupe de la Banque mondiale et Silvia Montoya, directrice, Institut de statistique de l’UNESCO (ISU)

La plupart des gouvernements et des partenaires de développement s’emploient à connaître, à protéger et à soutenir l’apprentissage des membres les plus vulnérables de la génération COVID-19. Dans ce blog, nous examinons de quelle manière le cadre de l’ODD 4.1.1 et le concept de pauvreté des apprentissages sont en mesure d’aider les pays à comprendre les impacts de la COVID-19 sur la scolarité et l’apprentissage, et à prendre les mesures appropriées pour les atténuer.

Du seuil minimal de compétences à la mesure de la pauvreté des apprentissages

En octobre 2018, la communauté internationale a convenu de réfléchir sur le suivi des progrès de l’apprentissage des élèves à l’aide d’une norme mondiale. Le Seuil Minimal de Compétences (SMC), approuvé par l’Alliance mondiale pour la mesure de l’apprentissage, fournit une valeur de référence incomparable pour aider les pays et les partenaires de développement à travailler de concert pour suivre et pour améliorer l’apprentissage des élèves qui ont du retard. La visualisation interactive ci-dessous (figure 1) vous permet d’explorer les données utilisées pour le suivi de cet ODD à l’aide du SMC-GAML et des différents seuils minimaux de compétence en déplaçant le curseur.

La figure 1 montre comment utiliser l’ODD 4.1.1 pour mettre l’accent sur les élèves en dessous du Seuil Minimal de Compétences (SMC).

La pauvreté des apprentissages : un facteur multidimensionnel pour le secteur éducatif

En octobre 2019, la Banque mondiale et l’Institut de statistique de l’UNESCO (ISU) ont lancé le nouvel indicateur multidimensionnel appelé « pauvreté des apprentissages ». Il est fondé sur la notion selon laquelle chaque enfant devrait aller à l’école et savoir lire un texte adapté à son âge à 10 ans.[1] Cette formulation, qui traduit l’aspiration de l’Objectif de Développement Durable (ODD) 4 qui stipule que tous les enfants doivent aller à l’école et apprendre[2] et qui tient lieu d’indicateur d’alerte précoce, repose sur deux privations. 

Continue reading

How the SDG 4.1.1 Framework and Learning Poverty Can Help Countries Focus Their Education Policy Response to COVID-19

João Pedro Azevedo, Lead Economist, Education Global Practice, World Bank Group and Silvia Montoya, Director, UNESCO Institute for Statistics (UIS)

Most governments and development partners are working on identifying, protecting, and supporting learning of the most vulnerable members of the COVID-19 generation. In this blog, we examine how the SDG 4.1.1 framework and the concept of learning poverty are well positioned to help countries understand and act on the impacts of COVID-19 on schooling and learning.

From the minimum proficiency level to a measure of learning deprivation

In October 2018, the international community agreed to be deliberate about tracking progress in learning of students using a global standard. The minimum proficiency level (MPL)­ agreed through the Global Alliance to Monitor Learning offers a unique benchmark to help countries and development partners work together to monitor and improve learning for these students that are falling behind. The interactive visualization linked to in the image below (Figure 1) allows you to explore the data used to monitor this SDG, using both the GAML MPL as well as different minimum proficiency levels by interacting with the slider.

Figure 1 shows how SDG 4.1.1. can be used to generate focus on students below the minimum proficiency level (MPL)

Learning poverty: a multidimensional indicator for the education sector

In October 2019, the World Bank and the UNESCO Institute for Statistics (UIS) launched a new multidimensional indicator called learning poverty. It is based on the notion that every child should be in school and be able to read an age-appropriate text by age 10.[1] This formulation reflects the aspiration and serves as an early warning indicator of Sustainable Development Goal (SDG) 4 that all children must be in school and learning[2] and builds on two deprivations.

Learning poverty (and Indicator 4.1.1 on learning deprivation) has many desirable characteristics, including simplicity and focus on those in the bottom of the learning distribution (for a longer discussion on some of the properties of the learning poverty measure please see a recent paper). It brings together schooling and learning indicators, as it combines the share of primary-aged children out-of-school who are schooling deprived (SD), and the share of pupils below a minimum proficiency in reading, who are learning deprived (LD). This measure implies that both “more schooling”, which by itself serves a variety of critical societal functions, as well as “better learning” which is important to ensure that time spent in school translates into acquisition of skills and capabilities.

Figure 2 provides an animation which numerically and visually illustrates the concept of learning poverty

The learning poverty indicator is calculated as follows:

LP = [LD x (1-SD)] + [1 x SD]

LP = Learning poverty

LD = Learning deprivation, defined as share of children at the end of primary who read at below the minimum proficiency level, as defined by the Global Alliance to Monitor Learning (GAML) in the context of the SDG 4.1.1 monitoring

SD = Schooling deprivation, defined as the share of primary aged children who are out-of-school. All out-of-school children are assumed to be below the minimum proficiency level in reading.

By construction, learning poverty can be affected by changes in its two dimensions: (i) learning deprivation as the share of students below the minimum proficiency threshold improves or worsens, or (ii) schooling deprivation as access or age-grade distortion changes due to shocks or policies.

While schooling deprivation can be directly observed depending on whether the child is enrolled or not enrolled in school, learning deprivation cannot be directly observed, and is measured through standardized learning assessments using SDG’s definition of minimum proficiency level, where reading proficiency is defined as:

“Students independently and fluently read simple, short narrative and expository texts. They locate explicitly stated information. They interpret and give some explanations about the key ideas in these texts. They provide simple, personal opinions or judgements about the information, events and characters in a text.” (UIS and GAML 2019)

 SDG 4.1.1, learning poverty and COVID-19

The SDG 4.1.1 framework and the learning poverty measure can help monitor and guide national conversations on the impacts and education policy response to COVID-19 by:

Reaching agreement and clarity on a minimum proficiency level: The GAML process, through the Global Proficiency Framework, has produced detailed documentation about the competences expected to be mastered at the minimum proficiency level (MPL). All this material can be used to inform a national conversation on what elements of the curriculum could be prioritized as the system reopens.

Focus on children falling behind: SDG 4.1.1 uses the MPL to measure the share of students above this threshold reflecting the aspiration that all children must be performing above the MPL. However, during a time of crisis and shock, such as COVID-19, countries might want to pay special attention to those students left behind. The latter is precisely what the measure of learning deprivation used in the learning poverty measure does.

Monitoring multiple dimensions of education:  As schools close, students will lose learning. However, for certain sub-populations, COVID-19 might push students out of the educational system, increasing drop-out rates; and in certain countries, due to a choice of policies and practices, might increase repetition and age-grade distortion. Moreover, if school deprivation increases, through an increase of drop-out or of the age-grade distortion of previously low-performing students, it is statistically possible that average learning scores might increase or at least not fall as much after COVID-19. This misleading result can be avoided if effects are monitored using a measure which is simultaneously sensitive to changes in learning and access to schooling.

COVID-19 has led to an unprecedented crisis within an already existing global crisis of the education system in the developing world. The ability to use it as an opportunity to build back better will depend on the quality of our understanding of its effects. For that, both data and our choice of measures will be equally critical.

[1] World Bank 2019.

[2] SDG 4 makes this commitment: by 2030, signatories will “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.” The goal’s target 4.1 is to “ensure that all girls and boys complete free, equitable, and quality primary and secondary education leading to relevant and effective learning outcomes.”

Bridging CESA 16-25 and SDG 4: Using Regional Benchmarks to Meet Education Objectives

By Silvia Montoya, Director of the UNESCO Institute for Statistics (UIS) and Daniel Capistrano, University College Dublin

This International Day of Education, the impact of COVID-19 on education is top of mind and finding solutions to revitalize learning is a priority, now more than ever. The UNESCO Institute for Statistics (UIS) is working with regional organizations and education partners to expand the global focus on benchmarking for the Sustainable Development Goals for education (SDG 4) so that regions and countries have more manageable, annual objectives.

With just a decade remaining to achieve SDG 4, it is imperative that all countries have the means to monitor progress and to plan necessary changes for the future. As the custodian of SDG 4 data and the lead agency providing internationally comparable and quality education data, the UIS has been working to help countries deal with this challenge.

One of the most effective ways of achieving the Agenda 2030 is by connecting existing efforts. The Africa Regional Report is a product of this collaborative strategy. Worldwide, there are several regional or sub-regional organizations that produce data and follow the progress of education policies based on common goals. Their transnational commitments require national and regional coordination and monitoring mechanisms to identify progress and obstacles. At the same time, they have articulated – or begun to articulate – their regional objectives with the Education 2030 Agenda.

Continue reading

Using Household Surveys to Achieve the SDG 4 Objectives of Inclusivity and Equity

By Friedrich Huebler, Head of Education Standards and Methodology at the UNESCO Institute for Statistics (UIS)

It is not enough to simply collect data. Data that are useful for monitoring progress towards Sustainable Development Goal 4 on education must be of high quality and comparable across countries. But collecting the data across a wide range of indicators has strained the data collection capacity of many Member States. At the same time, additional reporting needs brought on by COVID-19 have added further pressure to produce data as evidence for remedial action once schools fully re-open.

As the custodian agency for SDG 4, the UNESCO Institute for Statistics (UIS) works with countries to build their capacity to collect, produce and disseminate the data for monitoring progress towards international goals and for designing appropriate interventions, all while trying to mitigate the demands that this entails.

With this in mind, today the UIS is launching the 2020 SDG 4 Data Digest. This year, the Data Digest focuses on using household surveys to improve the scope of data collection while filling some of the gaps in administrative data.

To do this, the Data Digest explains the need for more and better data, serving as a “how-to” manual for ministries of education, national statistical offices and other education sector stakeholders. Readers will find information on everything from planning and design considerations for a household survey, to tips for writing compelling and effective questions, an interviewer’s check list of do’s and don’ts, the pros and cons of various modes of survey administration, along with implementation details like the most appropriate kind of field materials. The Data Digest also makes suggestions on how to communicate data findings.

In short, the 2020 SDG 4 Data Digest is the go-to source for a succinct overview of creating and implementing a household survey.

Continue reading

Prepared for the Future: A New Indicator That Combines Completion with Learning

By Silvia Montoya, Director, UNESCO Institute for Statistics (UIS) and Manos Antoninis, Director, Global Education Monitoring Report

New global indicator will provide a simple, comprehensive measure of progress towards the education goal, SDG4.

The Sustainable Development Goal (SDG 4) for education recognizes that all children deserve, and have the right to, a quality education. Over the last three decades, enrollment has risen to historic highs, though school disruptions and the economic implications of COVID-19 will offset some of these gains. But enrollment is only a part of what children need. For children to be fully prepared for the future, they need to complete their education, and emerge having learnt at least the basics. The new global indicator will combine all these critical factors to provide a snapshot of progress towards SDG 4.

Completing and learning are critical elements of a quality education

Unfortunately, in some of the poorest regions where children are most in need of a high-quality education to get ahead, poor learning outcomes often result in higher drop-out rates with large numbers of children not completing school at all – or completing it when more than five years older than the intended graduation age for that level. In sub-Saharan Africa, for example, where 82% of primary aged children are enrolled in school at the right grade for their age, just 62% graduate from this level on time. When children don’t finish school, it is hard, if not impossible, for any more learning to happen.

Even where education is free, poor families still pay for books and uniforms, and there is the perceived “opportunity cost” of lost income or help with household chores, while children are in school. If parents don’t see a pay-off from their investment in education, children can be pulled out of school before completing a level, or when transitioning between levels, from say, primary school, to lower secondary.

Introducing the new indicator: Prepared for the Future

To underline the need for countries to prioritize school completion, the UNESCO Institute for Statistics (UIS), working through the Global Alliance to Monitor Learning (GAML), is proposing a new, holistic, indicator that will track both completion and learning combined. The aim is to ensure that global leaders and education policy makers have the evidence they need to zero in on where they stand on their SDG 4 commitments. With ten targets and 42 indicators in the monitoring framework for SDG 4, some might argue that it is hard to quickly grasp where countries stand in their progress towards the goal. With so many touch-points, it risks calls for change being watered down. This new indicator will hopefully answer these concerns, providing a simple rallying reference point for all education actors to lobby for improvement, ensuring all children are prepared for the future.

Continue reading

Recalculating Proficiency in Schools: COVID-19 Related Learning Losses and Dropping Out

Traditional ways of measuring educational proficiency, based on assessment results and the percentage of children enrolled in schooling, may not be as effective a measurement while COVID-19 impacts upon attendance rates.

Analysts working with SDG indicators on learning proficiency, in particular Indicator 4.1.1, would in many cases be familiar with a formula such as the following:

The percentage of enrolled learners who are proficient, S, is typically known from assessment results. The percentage of children of the age of the grade in question who are not enrolled, N, is typically known from household data. is what planners are ultimately interested in: the percentage of the relevant age cohort in the population who are proficient. While there is some ambiguity over whether Indicator 4.1.1 refers to or Pis clearly important and must be monitored.

The above formula assumes that non-enrolled children do not reach minimum proficiency levels, following the approach taken in UIS (2020) and UIS (2017). The relationships can be illustrated as in the following graph, which is based loosely on levels of learning proficiency and the out-of-school problem in developing countries. Here is a function of and N, according to the above formula. 

In the context of COVID-19, sudden declines in enrolment, meaning rises, and declines in proficiency in the population due to disruptions in schools, meaning drops, are possible. In examining the effects on the three statistics Pand S, it is important to remember that and influence each other. Assuming that children who drop out of the schooling system are the academically weakest learners, an assumption that is likely to hold for a number of reasons, one can expect S either to rise or decline, depending on the magnitudes of the two effects: dropping out of school and learning losses. If one has estimates of and N, then the new is calculated according to the following transformation of the previous formula:

To illustrate the ambiguity, we can imagine learning losses resulting in a drop in from the 35% seen in the graph to 33%. If out-of-school increases from 5% to 9%, declines from 37% to 36%. However, if displays a larger increase, from 5% to 13%, then displays an increase from 37% to 38%. Even with learning losses in the population of a specific age, if enough learners drop out, and we assume that these learners are those who struggled most academically, then assessment systems may in fact detect an increase in the percentage of proficient learners.

Clearly, planners need to be fully aware of these relationships. Above all, the first formula above should not be used to conclude that more out-of-school children on its own produces a decline in P. If was 37% (as in the graph) and then out-of-school increases from 5% to 13%, one cannot conclude that drops to 32%. This would ignore the fact that if changes, automatically changes too. 

Projections For Learning Proficiency Can Inform Post COVID-19 Educational Strategies

The UIS will soon be releasing its projections of the impact of COVID-19 on the learning proficiency of children and youth, building upon previous UIS projections published in early 2020, before the pandemic. Projections of this kind are vital and inform the planning conducted by national governments, global bodies, and development assistance organizations. 

The data suggest that the impact of the pandemic on learning occurs through two distinct channels:

  1. Disruption in schooling has denied learners their usual classroom experience with teachers and other children. Disruptions have been particularly serious for younger children, who are most in need of contact teaching, and often receive meals through the school. These needs are felt especially in developing countries.
  2. The economic shock of the pandemic has put education budgets under pressure, and worsened household poverty. These factors also impact negatively on learning. 

The findings suggest that the effects of school disruptions on learning proficiency could be felt for many years, even after the pandemic is declared over. In future years, however, as children who did not directly experience the disruptions enter the education system, the longer-term budgetary and poverty effects will be of greatest significance. It is important for planners both to mitigate the effects of the disruptions, and to plan for the educational well-being of future age cohorts of children not affected directly by the disruptions. 

Current focus

The following diagram outlines the variety of factors influencing any schooling system’s ability to minimize learning losses: 

This illustrates how future priorities depend to a large degree on the strengths and weaknesses of the schooling system that existed before the pandemic. Systems with effective support and accountability structures, and which had been experiencing improvements in learning proficiency before 2020, are likely to be the most resilient to the shock of the pandemic. 

Continue reading