Recalculating Proficiency in Schools: COVID-19 Related Learning Losses and Dropping Out

Traditional ways of measuring educational proficiency, based on assessment results and the percentage of children enrolled in schooling, may not be as effective a measurement while COVID-19 impacts upon attendance rates.

Analysts working with SDG indicators on learning proficiency, in particular Indicator 4.1.1, would in many cases be familiar with a formula such as the following:

The percentage of enrolled learners who are proficient, S, is typically known from assessment results. The percentage of children of the age of the grade in question who are not enrolled, N, is typically known from household data. is what planners are ultimately interested in: the percentage of the relevant age cohort in the population who are proficient. While there is some ambiguity over whether Indicator 4.1.1 refers to or Pis clearly important and must be monitored.

The above formula assumes that non-enrolled children do not reach minimum proficiency levels, following the approach taken in UIS (2020) and UIS (2017). The relationships can be illustrated as in the following graph, which is based loosely on levels of learning proficiency and the out-of-school problem in developing countries. Here is a function of and N, according to the above formula. 

In the context of COVID-19, sudden declines in enrolment, meaning rises, and declines in proficiency in the population due to disruptions in schools, meaning drops, are possible. In examining the effects on the three statistics Pand S, it is important to remember that and influence each other. Assuming that children who drop out of the schooling system are the academically weakest learners, an assumption that is likely to hold for a number of reasons, one can expect S either to rise or decline, depending on the magnitudes of the two effects: dropping out of school and learning losses. If one has estimates of and N, then the new is calculated according to the following transformation of the previous formula:

To illustrate the ambiguity, we can imagine learning losses resulting in a drop in from the 35% seen in the graph to 33%. If out-of-school increases from 5% to 9%, declines from 37% to 36%. However, if displays a larger increase, from 5% to 13%, then displays an increase from 37% to 38%. Even with learning losses in the population of a specific age, if enough learners drop out, and we assume that these learners are those who struggled most academically, then assessment systems may in fact detect an increase in the percentage of proficient learners.

Clearly, planners need to be fully aware of these relationships. Above all, the first formula above should not be used to conclude that more out-of-school children on its own produces a decline in P. If was 37% (as in the graph) and then out-of-school increases from 5% to 13%, one cannot conclude that drops to 32%. This would ignore the fact that if changes, automatically changes too. 

Projections For Learning Proficiency Can Inform Post COVID-19 Educational Strategies

The UIS will soon be releasing its projections of the impact of COVID-19 on the learning proficiency of children and youth, building upon previous UIS projections published in early 2020, before the pandemic. Projections of this kind are vital and inform the planning conducted by national governments, global bodies, and development assistance organizations. 

The data suggest that the impact of the pandemic on learning occurs through two distinct channels:

  1. Disruption in schooling has denied learners their usual classroom experience with teachers and other children. Disruptions have been particularly serious for younger children, who are most in need of contact teaching, and often receive meals through the school. These needs are felt especially in developing countries.
  2. The economic shock of the pandemic has put education budgets under pressure, and worsened household poverty. These factors also impact negatively on learning. 

The findings suggest that the effects of school disruptions on learning proficiency could be felt for many years, even after the pandemic is declared over. In future years, however, as children who did not directly experience the disruptions enter the education system, the longer-term budgetary and poverty effects will be of greatest significance. It is important for planners both to mitigate the effects of the disruptions, and to plan for the educational well-being of future age cohorts of children not affected directly by the disruptions. 

Current focus

The following diagram outlines the variety of factors influencing any schooling system’s ability to minimize learning losses: 

This illustrates how future priorities depend to a large degree on the strengths and weaknesses of the schooling system that existed before the pandemic. Systems with effective support and accountability structures, and which had been experiencing improvements in learning proficiency before 2020, are likely to be the most resilient to the shock of the pandemic. 

Good national assessment programmes have become more important than ever. Countries with such programmes will soon have an idea of the magnitude of the pandemic-related learning losses, information which is vital in planning the recovery process. Initial studies of the actual (not projected) magnitudes of these losses have started to emerge. Countries without good national assessment programmes, need to invest in this now. This will enable them to gauge how proficient children are relative to children in other, similar countries, and to gauge whether improvement occurs.The following map is based on UIS.Stat data and points to clear regional and school calendar differences: 

Note: ‘A’ means school year starts in January to April, ‘B’ that it starts in July to October.

Categories of school year

The data show that having a school year starting in January to April (‘A’ in the map) is associated with 11% more of the school year having been disrupted. Being in Latin America and the Caribbean is associated with a further 11% increase. 

To illustrate the effects, in grade 3 reading the pandemic-induced disruptions lead to an immediate decline in global learning proficiency, from 59% proficient in 2019, to 50% in 2020. This takes into account the fact that if a learner loses 50% of the school year due to disruptions, the learner loses more than 50% of a year’s learning. Given a global learner-weighted average of 50.3% of the year lost, counting the situation up to the end of September 2020, around a year’s worth of learning lost globally is the estimated result. Further, the data suggest that the more disadvantaged learners experience both greater school disruption and poverty effects.

Beyond 2020

Two key factors influence the future trajectory:

  1. The 2020 disruptions experienced by learners entering grade 3 in future years – linked to the extent of the disruption to pre-schooling. 
  2. The ability of schooling systems to catch up to where they should be in terms of fundamental skills, particularly reading and mathematics. 

Analysts have warned of the risk of a scenario which is even worse than zero catching up. This would be a scenario where beyond 2020 children fall even further behind the pre-pandemic trajectory because teachers fail to adapt their teaching – essentially, continuing to teach as if the pandemic had not happened. Such a scenario should be avoided, through appropriate messaging and support to teachers, with a focus on the learning outcomes of learners in each school, and in the system as a whole.

Disruptions to pre-schooling would affect, roughly, those children taking grade 3 in 2023 and during a number of subsequent years. Evidence suggests institutions offering education to children below the primary level were disrupted as severely as primary schools. While there is little data on the quality of pre-schooling across the world, UIS.Stat provides statistics on national coverage of the population at this level. 

In general, countries with more pre-schooling have displayed better grade 3 learning proficiency, though the correlation is relatively weak. Better performing countries are thus more likely to experience learning losses relating to the disruption of pre-schooling. By having greater pre-school coverage, these countries have more to lose.The following graph illustrates the estimated impact of the pandemic on global grade 3 reading proficiency. 

Global projections for Grade 3 reading

The model predicts a relatively large impact on cognitive development for those children who were in utero during 2020. This rests on evidence that in utero impacts of socio-economic shocks can in fact be even larger than impacts felt by infants who are already born. 

In the model, a return to the original trajectory occurs in 2030, for grade 3, or possibly earlier, depending in part on the success of catch-up programmes. This return can be expected to occur later for the upper primary or lower secondary levels, given those levels will have to deal for more years with children who in some way experienced a 2020 disruption. While a return to the pre-pandemic learning trajectory when children born after 2020 enter grade 3 may seem optimistic, it is not that unrealistic. But such an outcome depends on fundamental elements of the schooling system being preserved, and improved upon. 

Maintaining and enhancing the quality of the teaching profession is especially important, given that it is the capacity and motivation of teachers which lies at the core of improving learning proficiency. Teachers, like many other workers, are likely to see their real income decline as a result of the pandemic. This process must be well-managed and must be seen as fair by teachers. The entry of new and young teachers into the profession, as older teachers retire, is a key mechanism through which the quality of teaching is raised. The pandemic should not disrupt innovations in teacher training. Moreover, there should not be an interruption in the hiring of young teachers joining the profession.  

The regional picture

The next graph illustrates differences by world region. The Latin America and Caribbean region is indeed expected to see the largest 2020 losses, amounting a population-weighted loss of 1.21 years of learning, compared to a global average of 1.01. The second-highest 2020 loss is that of Central and Southern Asia, at 1.18 years of learning. The longer duration of proficiency deficits in Latin America and the Caribbean beyond 2020 is in part driven by the exceptionally large primary school disruptions in 2020, meaning catching up takes longer, but also by the fact that this region’s pre-primary coverage has been relatively good, meaning disruptions at this level affect more learners entering primary schooling. Eastern and South-eastern Asia also displays high levels of pre-school coverage, but here the smaller magnitude of the 2020 disruptions to primary schooling assists in bringing about a return to the pre-pandemic trajectory by 2025. Sub-Saharan Africa’s pre-school coverage is less than half of that in Latin America and the Caribbean, which translates to a relatively early return to the pre-pandemic trajectory. It is particularly important for sub-Saharan Africa’s schooling systems to recover, and to improve, given how low learning proficiency has historically been in this region.

Projections for Grade 3 reading with 10% catching up by world region

Closing remarks

The United Nations Secretary-General, António Guterres, echoed the concerns of people and organizations around the world when he recently referred to the impact of the COVID-19 pandemic on schooling as a ‘generational catastrophe’. Children and youth are falling behind in their learning, and this is expected to have an impact lasting decades. This said, the data suggest that there are actions that can be taken now to limit this impact and national governments, global bodies, and development assistance organizations have a role to play in implementing this through evidence-based policy choices and support. Equally, schools and teachers themselves have to adapt to the ‘new normal’ and focus on ensuring that current and future generations are taught in ways that minimize losses – essential if the SDG 4 2030 target is to be achieved.

The Importance of Monitoring and Improving ICT Use in Education Post-Confinement

By Silvia Montoya, Director, UNESCO Institute for Statistics; and Alexandre Barbosa, Head of the Regional Center for Studies on the Development of the Information Society, under the auspices of UNESCO, Brazil (Cetic.br).

The global provision of schooling is facing unprecedented challenges as a result of the COVID-19 crisis. Within the span of a few months, 191 countries had closed their schools to deploy social distancing measures in accordance with the World Health Organisation (WHO) recommendations. More than 1.5 billion students from pre-primary to university-level have been affected by these closures, with classroom-based learning interrupted for indefinite periods of time. While some education systems, teachers, students and parents were somewhat prepared to adapt to existing distance learning programmes and platforms, millions were not.

In the context of COVID-19 school closures, paper-based and digital distance education platforms have become essential to the continued provision of education for all. After more than a month of school closures across the world, many students are still struggling with remote learning. Global estimates suggest that 826 million students are without a household computer, 706 million lack internet access at home and another 56 million lack coverage by mobile 3G/4G networks. To better gauge the scope of the impact of school closures and of the ensuing national education responses, a survey of ministries of education developed jointly by UNESCO, UNICEF and the World Bank was recently launched to more accurately inform a collaborative global education response.

Without adequate information and communication technology (ICT) devices, internet/mobile network access, educational resources and teachers’ training, students simply cannot partake in distance education to continue on their learning trajectories. At most risk of being left behind are students from resource-poor areas, remote rural areas and low-income households. In addition, learners with disabilities or those who use a different language in the home than in school will require more individualised support.

Multiple delivery channels are an essential component to reach all children and youth during this pandemic. A recent UNICEF survey found that 68% of the 127 countries were using a combination of digital and non-digital delivery of remote education (i.e. TV, radio and take-home packages). Even before the COVID-19 related school closures, the use of radio, video and television for remote learning has proven to be strong components of well-designed numeracy, literacy and financial education programmes for children, youth and adults living in remote and rural communities. However, the implementation and reach of such programmes require the monitoring and support of trained educators.

Distance learning also requires that school systems consider the needs of parents and guardians who have to step in to facilitate learning to ensure the pedagogical continuity of their children, especially for the children in earlier grades (Grades 1-3) who need more one-on-one support. The ability for parents and guardians to effectively facilitate home-based learning depends on a variety of interacting factors, including their education level, native language and time availability. Understanding parental digital literacy – which could be estimated from SDG 4 Indicator 4.4.1 that assesses ICT skills among youth and adults – is essential for targeting skill support and development for parents. Without ICT skills support for the adults in the home, children from families with poor digital literacy are likely to fall even further behind.

Developing ICT skills to ensure education weathers the storm of future crises

Reports of parents, teachers, communities and networks that have developed innovative and makeshift interventions, such as mobile-based Wi-Fi networks as well as on-demand content and textbooks available in clouds — to broaden digital capacities have certainly sparked optimism. However, these grassroots efforts largely serve as a short-term band-aid solution. Although they are inspiring, more fundamental developments to bolster access to and use of ICT are required – both at home and in schools, and especially for younger learners at the primary and secondary levels where gaps are largest. Hastily put-together remote teaching approaches have not proven to be optimal learning experiences and could be off-putting to some students.

School closures such as those currently experienced by the more than 1.5 billion students worldwide are commonplace in some countries due to natural emergencies, conflict as well as budgetary or labour negotiations. Once schools reopen, building skills and support for distance education in schools so learners can continue learning in the home can help minimise learning interruptions as well as deter learners from leaving school early or dropping out in the event of future crises. In addition, there remains a possibility that the COVID-19 crisis and its ensuing confinement measures may not be short-lived as flare-ups of cases may spark future school closures in certain countries. As some countries begin to reopen their schools, they will need to select innovative remote teaching modalities that blend with face-to-face teaching to ensure that learners are better prepared for future school closures. Thus, given the importance of distance education in the current context and in anticipation of future crises, countries need to take responsibility for monitoring, facilitating and enabling access to ICT in schools as well as in the homes of all learners.

Current measures of ICT availability fall short of capturing the needs in certain countries and regions as they fail to report on factors, such as the availability of electricity (grid- or solar based) and access to computers for pedagogical purposes, which are primary necessities. At a global level, these indicators are needed to monitor ICT use and detect national trends. However, they are not sufficiently detailed or policy-oriented to provide governments with adequate information to improve access to and use of ICT in education as well as sufficient information on teacher training and digital skills. For instance, counting the number of computers per school or per student poorly reflects the use of computers, which may in fact be minimal if these devices are locked in computer labs.

Monitoring ICT use in schools to better inform education policies post-confinement

Reliable data from school-based surveys can provide the quality ICT use data required to better inform education policy and practices, especially in developing countries. Capturing the complex set of factors involved will paint a more accurate picture of what is available and used by both students and teachers. This includes information, such as availability of digital infrastructure; internet connection speed; school activities in which teachers use ICT; training received by teachers to empower them to integrate ICT into their practices; strategies implemented by schools to develop digital skills; and perceptions by principals and teachers on ICT use in education and its barriers. Furthermore, the presence of qualified technical staff (e.g. technicians, librarians) is required to support the use of ICT in schools, including ensuring digital access and ICT learning among teachers.

These indicators and more are proposed in the Practical Guide to Implement Surveys on ICT Use in Primary and Secondary Schools – a joint publication by the UIS and Cetic.br (Regional Center for Studies on the Development of the Information Society). The guide discusses the relevance of survey data on ICT use in schools to inform policymaking and underscores the need for robust data to understand factors that determine equal access to and use of technologies by the teachers, principals, students and their families.

The guide serves as an especially useful reference for government agencies, school authorities and other stakeholders looking to measure ICT access and use in education as it outlines the methodology and steps required to conduct a successful survey (i.e. planning, fieldwork, data processing, reporting and dissemination). This comprehensive document also examines the practical aspects of developing school-based surveys explicitly related to ICT and includes methodological datasheets for 26 core and optional indicators.

As some countries begin to reopen their schools, promoting equity in ICT access and use will continue to be an important factor to consider when addressing educational challenges for disadvantaged schools and learners from vulnerable households. In addition, the availability of computers, tablets, mobile phones and other potential learning devices, along with the provision of internet access in the home, will ultimately determine which children will be able to participate in distance learning and be more likely to complete their education in the event of future school closures.

Bridging the digital divide at home and in schools

Evidence indicates that there is a substantial ‘digital divide’ in access to ICT between countries. For example, according to estimates from the International Telecommunication Union (ITU), 21% of learners in Africa cannot be reached by 3G mobile networks. In terms of internet access, 82.2% of households in Africa lack access in the home (see Figure 1). To bridge the divide and encourage mobile-based education, and in addition to infrastructure investment, lowering the cost to consumers to gain access to online data needs to be considered as these are prohibitive in many countries.

Source: International Telecommunication Union (ITU) 

Under Sustainable Development Goal (SDG) 4 to ensure inclusive and equitable education and promote life-long learning opportunities for all, governments have committed both to increase digital skills and expand ICT infrastructure in schools. To support distance education, schools will need to better equip learners with the skills to migrate onto these online learning platforms. Moreover, closing the ‘digital divide’ will require governments to invest in supporting learners in the early grades of school. In this endeavour, the first step is to map within and between countries where investment is most needed. This requires better measures of access to and use of digital technologies in schools.

The latest UIS data for SDG Indicator 4.a.1 on the availability of electricity, computers and internet in schools for pedagogical purposes indicate that certain regions are behind in their capabilities to support learners. Although sub-Saharan Africa as well as Central and Southern Asia do not have sufficient data for this indicator in lower education levels, there is adequate data for upper-secondary schools. Only about one quarter of upper-secondary schools in sub-Saharan Africa and one-half in Central and Southern Asia are equipped with internet access (see Figure 2). Electricity – another necessity – is also not available equitably across regions and school levels. In sub-Saharan Africa, only 33.8% of primary schools have access to electricity while the same holds true for 57.2% of upper-secondary schools in the region. The situation is bleaker still in the Democratic Republic of the Congo where only 13.7% of upper-secondary schools have access to electricity.

Source: UNESCO Institute for Statistics, SDG 4 Data Explorer (data for the latest available year used). 

Teacher training as part of the solution to closing the ICT skill gap

As noted, beyond the provision of internet access and ICT equipment in education, there is also a need to support learners by building their ICT skills. But what about teachers? During this period of school closures, teachers require training in the use of distance learning platforms to ensure teaching and learning can continue. While some of the 63 million primary and secondary school teachers who were displaced by COVID-19-related school closures have managed to reach students with their existing set of skills and equipment, many have not received basic teacher training. It is therefore disconcerting that most teacher training programmes do not include the use of ICT in education to develop appropriate learning and teaching strategies. In sub-Saharan Africa, only 64% of primary and 50% of secondary teachers have received minimum training. Indicators recommended in the Practical Guide to Implement Surveys on ICT Use in Primary and Secondary Schools can point to specific areas in which teacher training needs to be reinforced to improve ICT skills.

Use of ICTs for education during the COVID-19 crisis and beyond is a reality for which teachers and learners must be better prepared. This crisis marks the time for government-led initiatives for schools to test innovative methods to reach out to students, learn from other countries and incorporate effective approaches into the regular provision of education. Digital technologies need to be integrated within sound learning programmes to create effective student learning experiences. Countries need to make the necessary preparations to better map teaching and learning needs for future crises in education. As a necessary step, this requires collecting robust survey data on ICT use in schools to adequately guide policymaking.

Rethinking Education Post-Coronavirus: Lessons from Spain to Avoid Widening the Socioeconomic Achievement Gap

By Ana Capilla, Organización de Estados Iberoamericanos (OEI) and UFV;
Jorge Sainz, Universidad Rey Juan Carlos (URJC), Madrid (Spain) and IPR (University of Bath, UK); and Ismael Sanz, URJC, Madrid (Spain)

In a recent post, UNESCO reminded us of the similarity between the learning challenges that fourteenth century societies confronted during the Black Death and the current COVID-19 pandemic. Back then, as William Courtenay remembers, the plague helped develop new ways of teaching and the beginning of the substitution of Latin with popular languages as vehicles to communicate science. Just as that global pandemic marked the beginning of a new world in education, so could the current one if policymakers respond thoughtfully to this education crisis.

Continue reading

Nurturing Education at Home in the Midst of a Health Crisis: How SDG Indicator 4.2.3 can Help Guide Where More Support is Needed

By Silvia Montoya, Director, UNESCO Institute for Statistics

In the midst of this global health crisis that threatens lives and containment measures that threaten our ways of living, we are faced with the stark reality that the world we return to will be forever altered. The far-reaching consequences of the COVID-19 pandemic may also jeopardize the hard won gains made in improving global education. 

The country statistics data collected and housed at the UIS has proved crucial in this time of crisis to help us estimate the global scope of the pandemic’s impact on education due to massive school closures. Given the importance of education as the foundation for all development, we must strive to safeguard learning at all ages. Thus, as a response, UNESCO has launched the Global Education Coalition in an attempt to support learning in the home as this becomes the new normal.

Continue reading

Where Are All the Women in Science and Research?

By Rohan Pathirage, R&D statistician for the UNESCO Institute for Statistics (UIS)

On the International Day of Women and Girls in Science, you could be forgiven for asking: just where are all these women and girls? They are there if you look but they remain the exception at the very highest echelons in science and research. Their rarity may even – on occasion – obscure the critical value of their work. From Marie Curie to Rosalind Franklin, women scientists have often been viewed in terms of their relationships to their male counterparts.

A simple web search on female scientists will yield lists of the ‘17 Famous Female Scientists Who Helped Change the World’ or ‘10 Women in Science and Tech Who Should be Household Names’. But search for ‘the most famous male scientists today and the lists talk about the most ‘influential’ scientists, ‘the top 10 greatest scientists’ – with no mention of the word ‘male’. It simply isn’t noteworthy. Continue reading

Data to Fuel an Education Revolution: Countries Answer the Funding Call

By Julia Dicum, Deputy Director of Education at Global Affairs Canada, and Silvia Montoya, Director of the UNESCO Institute for Statistics.

It is time for all donor countries to invest more heavily in education data. This matters because we can’t solve a problem we don’t understand. And it is only too clear that we have some big problems that must be solved right away, with 617 million children and adolescents who are not reaching even minimum proficiency levels in reading and mathematics, and 262 million children – one in every five –  who are out of school and half of whom are girls. Continue reading

Focus on our SDG 4 Data Digest Partners: Conférence des Ministres de l’Éducation des États et Gouvernements de la Francophonie (CONFEMEN)

By Hilaire Hounkpodoté, PASEC Coordinator

The recent SDG 4 Data Digest illustrates the range of partners working with the UNESCO Institute for Statistics (UIS) to help countries produce and use assessment data to strengthen lifelong learning. This blog highlights the work of one of these vital partners: the Conférence des Ministres de l’Éducation des États et Gouvernements de la Francophonie (CONFEMEN). CONFEMEN works with the world’s French-speaking countries to implement the Programme for the Analysis of Education Systems (PASEC), a renowned regional learning assessment.

Continue reading

Data to Celebrate 50 Years of Progress on Girls’ Education

By Silvia Montoya, Director of the UNESCO Institute for Statistics (UIS)

This blog was also published by Norrag.

On International Women’s Day, we can look back at 50 years of extraordinary efforts to get girls into the classroom. Thanks to these efforts, the gender gap in education is closing and has been closing for decades. And today, the majority of girls worldwide complete primary school. Girls are also in school for longer than ever before. As 50 years of data produced by the UNESCO Institute for Statistics (UIS) show, girls’ school life expectancy is on the rise.

School life expectancy is the number of years of education a child can expect to receive. Overall the trends look good, even though school life expectancy also includes the years that girls repeat grades. Back in 1970, the global average for girls’ school life expectancy, spanning primary, secondary and tertiary education, was around 6.7 years. Today, it tops 12 years. Continue reading

Digital Literacy Skills: From a Framework to a Measure

By Manos Antoninis, Director of the Global Education Monitoring Report

The latest SDG 4 Data Digest 2018, Data to Nurture Learning, summarized, among other things, the progress made by a range of partners working with the UNESCO Institute for Statistics (UIS) towards establishing a framework and identifying assessment tools to monitor digital literacy skills.

 Against the backdrop of Mobile Learning Week, this blog highlights the challenges for operationalizing the framework in a way that is formative, informative and cost efficient.   Continue reading