By Carina Omoeva, Wael Moussa and Rachel Hatch, FHI 360

This blog was also published by the Global Partnership for Education (GPE)

We know it is critical to measure equity in education: the fairness of education provision and quality of learning outcomes, as well as access to schooling, if we are to meet the global goals and targets for education.

We also know there are many different ways to conceptualize the measurement of equity, with different perspectives, start points and aims – often driven by the current political climate.

But now it is time to take a deep breath and make that all-important leap from the conceptual to the practical measurement of education equity. In the Handbook on Measuring Equity in Education, published earlier this year by the UNESCO Institute for Statistics (UIS), we zoom in on two key concepts – equality of condition and impartiality – and map out a five-step process for their measurement. We hope that it will help researchers navigate their way towards a fairly reliable measure of educational equity.

An equality of condition approach looks at how variables are distributed across all children, regardless of their circumstances. The advantage of this approach is that you only have to measure one variable, such as the number of years of education, to generate equity indicators that are comparable over time and between countries. The results can, for example, be mapped out on a Lorenz curve, and the flatter the line, the greater the equality of condition.

An impartiality approach measures whether quality and access to educational resources, as well as education outcomes, are equal across different equity dimensions, aiming to guard against any unfair distribution of opportunities that works against those who are already at a disadvantage. Above all, impartiality measures direct us to the most disadvantaged groups, who can then be targeted by policy (and hopefully resources).

Our proposed process links these two key approaches, creating a five-step checklist for the analysis of equity, based on research questions that can be studied separately or, ideally, combined into a single study. The full process quantifies and visualizes equality of condition, before moving on to describe impartiality, and finally testing program impacts for their impartiality – in other words, their equitable impact on a child’s chances for a good quality education.

1. Identify the equity dimensions to be examined

It seems self-evident that we must know who and what we are studying. So the first step is to define the demographic categories of the population we are studying and the key dimensions that so often influence equity, based on who and where they are. These include gender, ethnicity, location, poverty, disability and immigration or migration status.

Once these categories are identified, a simple descriptive analysis to summarize the make-up of the sample helps to contextualize pre-existing disparities between individuals and between groups. From a statistical standpoint, a summary of these ‘equity groups’ reveals not only the size of each group, but also its size relative to other groups. It is important to note, however, that while all metrics of inequality at the group level are a function of the size of that group, some groups are so small that inequality metrics cannot pinpoint the true extent of disparities between them and other groups.

2. Summarize observable characteristics by equity dimensions

Next, we propose a summary of the observable characteristics of each group according to the equity dimensions that have been mapped out. This step helps researchers to construct statistical profiles for each equity group that show discrepancies not only in their educational outcomes but also in their baseline characteristics. This is a research win-win: confirming not only any gaps in educational outcomes across different groups, but also suggesting the probable drivers of these gaps. While this step does not substitute more rigorous analysis of these drivers, it gives researchers a foothold when exploring the reasons for the gaps.

3. Analyze the overall distribution of outcomes

Thanks to this overview of the research sample and general state of inequality at the individual and group level, we can now take the next step: examining the full set of available information to focus on distributional analyses of educational outcomes. Here, we recommend the use of visualization tools, such as a histogram (Figures 1 and 2) to capture the degrees of disparity for the outcomes we are examining.

Figure 1. Hypothetical test score distributions with varying levels of inequality

Figure 2. Empirical distributions of PIRLS test scores for Canada and Oman

4. Analyze outcomes by equity dimension

Now it is time to augment our analysis of the overall distribution of educational outcomes by overlaying the distributions of the outcome for each group of interest. This will show both the absolute degree of inequality that exists between the groups by contrasting the spatial location of the distributions, and the degree of inequality within groups, allowing researchers to examine the range of disparity for each group separately. Researchers can also assess the degree of inequality within each group.

5. Estimate the main effects, overall and stratified by equity dimensions

The final step is to implement more rigorous methods to identify the plausible impact of policies, interventions and other ‘treatment’ on educational outcomes. From an equity perspective, however, it is not enough to identify the overall impact of a policy or any other solution on an entire population, as it may not have a similar effect across all individuals and groups. And as we know, such ‘blanket’ impact may not be enough to compensate for the serious disadvantages experienced by some groups. What is needed is an approach that goes deeper, testing whether the impact really is evenly distributed, or whether it has a greater impact for some groups than others.

Researchers can test this in a regression analysis framework by combining the independent variable (such as school enrollment) with the indicators for group membership, using interaction effects to stratify the overall impact and show whether the policy ‘treatment’ does in fact affect different groups in a way that is systematically different. Post-estimation testing on the interaction of equity and treatment can then be conducted to determine whether the estimated impact really is statistically heterogeneous.

These steps are familiar to any researcher and data analyst working in the field of international education.  We hope that with the publication of the Handbook, following a simple equity-oriented framework will become a routine process – bringing equity considerations into the mainstream, rather than adding them as an afterthought.

By fully integrating the accountabilities and processes that come with examining equity into our operations, we can move the development community closer towards achieving equitable quality education for all.