From Concept to Practice: Five Steps to Measure Education Equity

By Carina Omoeva, Wael Moussa and Rachel Hatch, FHI 360

This blog was also published by the Global Partnership for Education (GPE)

We know it is critical to measure equity in education: the fairness of education provision and quality of learning outcomes, as well as access to schooling, if we are to meet the global goals and targets for education.

We also know there are many different ways to conceptualize the measurement of equity, with different perspectives, start points and aims – often driven by the current political climate.

But now it is time to take a deep breath and make that all-important leap from the conceptual to the practical measurement of education equity. In the Handbook on Measuring Equity in Education, published earlier this year by the UNESCO Institute for Statistics (UIS), we zoom in on two key concepts – equality of condition and impartiality – and map out a five-step process for their measurement. We hope that it will help researchers navigate their way towards a fairly reliable measure of educational equity.

An equality of condition approach looks at how variables are distributed across all children, regardless of their circumstances. The advantage of this approach is that you only have to measure one variable, such as the number of years of education, to generate equity indicators that are comparable over time and between countries. The results can, for example, be mapped out on a Lorenz curve, and the flatter the line, the greater the equality of condition.

An impartiality approach measures whether quality and access to educational resources, as well as education outcomes, are equal across different equity dimensions, aiming to guard against any unfair distribution of opportunities that works against those who are already at a disadvantage. Above all, impartiality measures direct us to the most disadvantaged groups, who can then be targeted by policy (and hopefully resources).

Our proposed process links these two key approaches, creating a five-step checklist for the analysis of equity, based on research questions that can be studied separately or, ideally, combined into a single study. The full process quantifies and visualizes equality of condition, before moving on to describe impartiality, and finally testing program impacts for their impartiality – in other words, their equitable impact on a child’s chances for a good quality education.

1. Identify the equity dimensions to be examined

It seems self-evident that we must know who and what we are studying. So the first step is to define the demographic categories of the population we are studying and the key dimensions that so often influence equity, based on who and where they are. These include gender, ethnicity, location, poverty, disability and immigration or migration status.

Once these categories are identified, a simple descriptive analysis to summarize the make-up of the sample helps to contextualize pre-existing disparities between individuals and between groups. From a statistical standpoint, a summary of these ‘equity groups’ reveals not only the size of each group, but also its size relative to other groups. It is important to note, however, that while all metrics of inequality at the group level are a function of the size of that group, some groups are so small that inequality metrics cannot pinpoint the true extent of disparities between them and other groups.

2. Summarize observable characteristics by equity dimensions

Next, we propose a summary of the observable characteristics of each group according to the equity dimensions that have been mapped out. This step helps researchers to construct statistical profiles for each equity group that show discrepancies not only in their educational outcomes but also in their baseline characteristics. This is a research win-win: confirming not only any gaps in educational outcomes across different groups, but also suggesting the probable drivers of these gaps. While this step does not substitute more rigorous analysis of these drivers, it gives researchers a foothold when exploring the reasons for the gaps.

3. Analyze the overall distribution of outcomes

Thanks to this overview of the research sample and general state of inequality at the individual and group level, we can now take the next step: examining the full set of available information to focus on distributional analyses of educational outcomes. Here, we recommend the use of visualization tools, such as a histogram (Figures 1 and 2) to capture the degrees of disparity for the outcomes we are examining.

Figure 1. Hypothetical test score distributions with varying levels of inequality

Figure1-equity-chapter-3-blog

Figure 2. Empirical distributions of PIRLS test scores for Canada and Oman

Figure2-equity-chapter-3-blog

4. Analyze outcomes by equity dimension

Now it is time to augment our analysis of the overall distribution of educational outcomes by overlaying the distributions of the outcome for each group of interest. This will show both the absolute degree of inequality that exists between the groups by contrasting the spatial location of the distributions, and the degree of inequality within groups, allowing researchers to examine the range of disparity for each group separately. Researchers can also assess the degree of inequality within each group.

5. Estimate the main effects, overall and stratified by equity dimensions

The final step is to implement more rigorous methods to identify the plausible impact of policies, interventions and other ‘treatment’ on educational outcomes. From an equity perspective, however, it is not enough to identify the overall impact of a policy or any other solution on an entire population, as it may not have a similar effect across all individuals and groups. And as we know, such ‘blanket’ impact may not be enough to compensate for the serious disadvantages experienced by some groups. What is needed is an approach that goes deeper, testing whether the impact really is evenly distributed, or whether it has a greater impact for some groups than others.

Researchers can test this in a regression analysis framework by combining the independent variable (such as school enrollment) with the indicators for group membership, using interaction effects to stratify the overall impact and show whether the policy ‘treatment’ does in fact affect different groups in a way that is systematically different. Post-estimation testing on the interaction of equity and treatment can then be conducted to determine whether the estimated impact really is statistically heterogeneous.

These steps are familiar to any researcher and data analyst working in the field of international education.  We hope that with the publication of the Handbook, following a simple equity-oriented framework will become a routine process – bringing equity considerations into the mainstream, rather than adding them as an afterthought.

By fully integrating the accountabilities and processes that come with examining equity into our operations, we can move the development community closer towards achieving equitable quality education for all.

Strengthening Citizen-Led Assessment Data

By Hannah-May Wilson, Program Manager, PAL Network Secretariat

As delegates gather for the World Bank South Asia Regional Workshop on Learning Assessment in New Delhi to share knowledge and learning on assessment practices in basic education in the South Asia region, the PAL Network of citizen-led assessment organizations spanning South Asia, Africa and Central America have just released their newly-created network-wide Data Quality Standards Framework. Continue reading

A Sound Investment: The Benefits of Large-Scale Learning Assessments

By Silvia Montoya, Director of the UNESCO Institute for Statistics (UIS), and David Coleman, Senior Education Advisor at Australia’s Department of Foreign Affairs and Trade (DFAT) and Head of the Strategic Planning Committee of the Global Alliance to Monitor Learning (GAML)

This blog was also published by the Global Partnership for Education (GPE). 

As delegates gather in New Delhi for the South Asia Regional Conference on Using Large-Scale Assessments to Improve Teaching and Learning, a new synthesis paper from the UIS makes the case for greater investment. Continue reading

The View from Madagascar: Data to Build Evidence-Based Policy

By Rolland Rabeson, Secretary-General of the National Education Ministry, Georges Solay Rakotonirainy, Secretary General of the Ministry of Employment, Technical and Vocational Education and Training, and Christian Guy Ralijaona, Secretary-General of the Ministry of Higher Education and Scientific Research, Madagascar

Reinforcing and deepening regional synergies in education will be at the forefront of the Pan-African High-Level Conference on Education (PACE 2018) in Nairobi from 25-27 April. The UNESCO Institute for Statistics (UIS), a key partner of countries across the region, will give a series of presentations on the importance of data for national education planning and for monitoring international commitments enshrined in the Sustainable Development Goal for education (SDG 4).

The UIS is working side-by-side with country partners in a UNESCO-sponsored pilot project called Capacity Development for Education (CapED). The participating countries are: Afghanistan, Cambodia, Democratic Republic of Congo, Haiti, Mali, Madagascar, Mozambique, Myanmar, Nepal and Senegal. The aim of the project is to help these countries develop and strengthen their own abilities to produce quality data.

To this end, since September 2017, our joint team of education ministries, the National Institute for Statistics and other national institutions involved in education data production in Madagascar has been working with the UIS to fulfill these objectives. Continue reading

Follow the Money: Tracking Education Spending to Reinforce Accountability

By Sonia Ilie, Pauline Rose and Asma Zubairi, Research for Equitable Access and Learning (REAL) Centre, University Of Cambridge

This blog was also published by the Global Partnership for Education (GPE)

It’s Global Action Week for Education, with the focus firmly on accountability. As we all know, if we want to hold our decisionmakers to account, we must have good data. Without it, we have little evidence of whether they are keeping their promises or not.

In the case of education, this certainly means knowing how many children are in school, how many are out of the classroom, and whether they are making good progress in their learning. But there is another critical area that is crucial not only for accountability on education, and that is the money. Who pays for education? How much do they pay? Where does the money go? And very importantly, who is benefiting from public spending by governments? Continue reading

Why We Need Effective Education Management Information Systems

By Silvia Montoya, Director of the UNESCO Institute for Statistics (UIS)

It may sound dry and dusty, but an education management information system (EMIS) lies at the very heart of efforts to monitor progress towards the world’s education goals, particularly Sustainable Development Goal 4 (SDG 4). It is a vital instrument that has, perhaps, had less attention than it deserves, given that an EMIS should be, in essence, in the core of the planning and policy implementation processes in a country’s education ‘machine’. Continue reading

Priorities and Challenges for Education Data in Sweden

Lotta Larsson, Senior Advisor in the Department for Population and Welfare Statistics, Statistics Sweden

This blog was also published by Norrag.

As the Inter-agency and Expert Group on SDG Indicators (IAEG-SDGs) meets in Vienna from 9-12 April 2018, a perspective from Sweden illustrates the challenges even the world’s most advanced statistical systems face in producing the education data needed to monitor and achieve the global education goal.  

The Scandinavian countries are often held as a model for other countries to follow on almost any area of development you can name, from poverty reduction to health and well-being. From an international perspective, Sweden is a country with a high quality education system.

In 2013, however, the PISA results showed that the average scores had declined from previous heights to below the average for OECD countries. This started discussions on the quality of the education system at the primary and lower secondary levels in Sweden. Since 2013 the country’s PISA results have improved and it is now – once again – at or above the OECD average for mathematics, reading and science. Continue reading