By Hannah-May Wilson, Program Manager, PAL Network Secretariat

As delegates gather for the World Bank South Asia Regional Workshop on Learning Assessment in New Delhi to share knowledge and learning on assessment practices in basic education in the South Asia region, the PAL Network of citizen-led assessment organizations spanning South Asia, Africa and Central America have just released their newly-created network-wide Data Quality Standards Framework.

In the past, citizen-led assessments have been too easily dismissed as lacking the necessary rigour or quality to be compared with other school-based assessments – an issue highlighted during a recent gathering of senior data analysts and leaders from the PAL Network. Speaking at the opening plenary, Dr. Wilima Wadhwa from the ASER Centre, India stressed the importance of PAL Network members establishing systems to ensure robust and reliable data, saying: “When you release data highlighting government failure, the first thing that will be attacked is your methodology. If someone can cast doubt on your methodology or say your data is unreliable or that the sampling hasn’t been done properly and isn’t representative, then people will find a reason not to engage with the findings.”

Catering to the realities of countries in the global South

The fact is that the citizen-led assessment model was designed carefully and deliberately to cater for the realities of countries in the global South. The design reflects a philosophy that differs from that of standard school-based assessments, as citizen-led assessments train volunteers to assess children regardless of their schooling status, using simple tests and tools, sitting one-on-one with the child, in their own homes.

What may appear to be a very simple assessment is actually backed up by highly-sophisticated processes to ensure the reliability of the data generated. These include systematic processes for sampling, partner and volunteer selection, training, monitoring and re-check. In addition, careful data cleaning and other methods are used to validate the data.

As Dr Wadhwa explained: “Documenting and strengthening these processes and sharing best practices is one of the reasons the network exists. As more countries start to conduct citizen-led assessments and the network grows, network members want to learn from each other. It was this shared focus that brought us together a year ago to draft this standards framework. Now we are back here to say – OK, we have this document. How can we support each other to implement it?”

 Making the case for citizen-led assessment data to monitor progress in education

For the past three years, the PAL Network has advocated for the inclusion of an early grade indicator in Sustainable Development Goal 4 (SDG 4). For a clear picture of progress, data used to measure this indicator must include all children, regardless of their schooling status. In the wake of the World Education Forum 2015, the PAL Network released a public statement to the Technical Advisory Group on SDG 4, demonstrating the value of citizen-led assessments to track the acquisition of foundational skills for all children. This was followed by an Open Letter to the Inter-Agency and Expert Group (IEAG) in March 2016 with an urgent appeal to retain the draft indicator for the percentage of children in Grades 2/3 who have learned the basics, and a position statement on SDG4 in July 2016.

Since May 2016, network members have been active participants in the Global Alliance to Monitor Learning (GAML), ensuring that citizen-led assessment data are included as a complementary source for reporting against SDG 4 Indicator 4.1.1. Data from citizen-led assessments are also included in the Catalogue of Learning Assessments (CLA) developed by the UNESCO Institute for Statistics (UIS): an important step towards the inclusion of learning data for children in developing countries who are less likely to be found in traditional educational settings.

Ensuring no child is left behind

To ensure that no child is left behind, active efforts are needed to identify the most disadvantaged children. And that, in turn, means  understanding where they are most likely to be found. Across the developing world, the children who are the hardest to reach are often found in the areas that are hardest to reach, and are unlikely to be in school. In places where significant numbers of children are out of school or attending sporadically, citizen-led assessments have significantly improved knowledge of the inequalities that persist in educational access and the acquisition of foundational reading and numeracy skills.

The new PAL Network Data Quality Standards Framework will help member countries improve and ensure technical rigour, with enough flexibility to accommodate the diversity of processes and adaptations to local context that are central to the citizen-led assessment model. The DQSF will be accompanied by implementation and monitoring plans, with member countries supporting each other to meet the minimum required standards, and ensuring that the PAL Network continues to make an important and robust contribution to understanding learning progress for all children.

By Rolland Rabeson, Secretary-General of the National Education Ministry, Georges Solay Rakotonirainy, Secretary General of the Ministry of Employment, Technical and Vocational Education and Training, and Christian Guy Ralijaona, Secretary-General of the Ministry of Higher Education and Scientific Research, Madagascar

Reinforcing and deepening regional synergies in education will be at the forefront of the Pan-African High-Level Conference on Education (PACE 2018) in Nairobi from 25-27 April. The UNESCO Institute for Statistics (UIS), a key partner of countries across the region, will give a series of presentations on the importance of data for national education planning and for monitoring international commitments enshrined in the Sustainable Development Goal for education (SDG 4).

The UIS is working side-by-side with country partners in a UNESCO-sponsored pilot project called Capacity Development for Education (CapED). The participating countries are: Afghanistan, Cambodia, Democratic Republic of Congo, Haiti, Mali, Madagascar, Mozambique, Myanmar, Nepal and Senegal. The aim of the project is to help these countries develop and strengthen their own abilities to produce quality data.

To this end, since September 2017, our joint team of education ministries, the National Institute for Statistics and other national institutions involved in education data production in Madagascar has been working with the UIS to fulfill these objectives.

We need data to revitalize education policy and planning to build a prosperous future

As a part of the CapED initiative, we have reviewed our education sector plans to find any gaps between our national policy objectives, and our international and regional commitments.

We have also looked at our existing data sources, comparing them to our needs. Based on our particular national context, we have focused on improving several key areas such as age-related data for basic education and school readiness, and the data coverage for technical and vocational education and training (TVET) and higher education.

It has been important to find synergies between our own national data needs and our commitments to SDG 4. So, in consolidating all of the data available we have created a national indicator framework that is relevant to us while meeting our international goals.

Finally, we worked with the UIS to undertake a data quality assessment, which enabled us to look at all of our data sources and ensure that they meet international standards.

We are now entering the final phase of building our education data roadmap: a National Strategy for the Development of Education Statistics (NSDES).

To get there, our strategic objectives are based on a set of recommendations in each of the following areas:

• Institutional framework for data collection and analysis,
• Process and methodologies for developing relevant educational statistics and;
• Quality of the final data product.

Financial limitations hampering data collection and development

A strong institutional framework is a key element in making sure that our team of education ministries in Madagascar collects and produces quality data with the scientific guidance of our National Institute for Statistics, along with development partners. We know that our technology needs remain high and all three ministries have ongoing requirements for qualified personnel to analyze the information. Moreover, the responsibilities along the data production cycle are not always clear, therefore we need to review our governance framework in this area.

But most crucially, this evaluation revealed that limited financial resources is hampering our ability to produce enough data in a timely fashion. Underfunding of our institutional statistical framework can make planning our educational programs more difficult. It also impacts our ability to meet our SDG 4 commitments.

Addressing this issue would involve creating a separate budget line item for data. This would ensure that funds are available for statistical needs and it would reduce our reliance on external sources of funding. Allocating funds for data in this way would enhance our national sovereignty as quality data – produced independently – should form the basis for all strategic education policy planning decisions.

Consultation with end user key to data uptake

In terms of the collection and classification of data, our partners at UIS have recommended regular, annual consultations with data users in the public and private sectors so that we can identify their needs. This would lead to data creation that is more relevant to the end user with the aim of increasing the rate of uptake.

Greater coordination would help us to put data at the heart of our policy decisions for education. This in turn would lead to more effective policies that target the education needs in-line with national priorities and SDG 4.

Availability of quality education data necessary for evidence-based policy

Finally, the quality of education data is a critical element of our data roadmap, the NSDES. Important elements for this are the accessibility and clarity of the data, as well as their coherence and timeliness. Our evaluation indicates that there are discrepancies between the different types of data produced; bridging this gap – and adhering to international standards – will be key to reinforcing the data quality and ensuring its cross-country comparability.

Asserting the role of the national statistics office would also help. It is also important to implement a national census every ten years to form a baseline reference point. The last time Madagascar conducted a census was 1993. Such a long time-lag can make data projections difficult.

We will consider all of these objectives as we document our NSDES to create a strong data environment for education planning purposes.

In the run-up to PACE 2018, we are proud to be a part of UNESCO’s innovative CapED initiative to improve education data both for national education planning and to meet the objectives of SDG 4.

As African countries meet in Nairobi to work towards our shared agenda for more prosperous, people-centered development, let’s remember that education remains at the heart of building a better future. To achieve the Africa we want, we need quality data to target education policies and to monitor progress.

By Sonia Ilie, Pauline Rose and Asma Zubairi, Research for Equitable Access and Learning (REAL) Centre, University Of Cambridge

This blog was also published by the Global Partnership for Education (GPE)

It’s Global Action Week for Education, with the focus firmly on accountability. As we all know, if we want to hold our decisionmakers to account, we must have good data. Without it, we have little evidence of whether they are keeping their promises or not.

In the case of education, this certainly means knowing how many children are in school, how many are out of the classroom, and whether they are making good progress in their learning. But there is another critical area that is crucial not only for accountability on education, and that is the money. Who pays for education? How much do they pay? Where does the money go? And very importantly, who is benefiting from public spending by governments?

In the Handbook on Measuring Equity in Education, published by the UNESCO Institute for Statistics (UIS) in March, we scrutinized government spending as a way to increase equity in education. We looked at who benefits most from government education expenditure and highlighted ‘formula’ funding as a way to redistribute resources to those with the greatest need.

While tracking progress towards education goals signals political will to tackle inequality gaps, such as those based on gender or location, it is not enough, on its own, to help close those gaps. Another way to assess true commitment is to look at how government resources are allocated.

We know that reaching the most disadvantaged children is likely to cost far more, per head, than the spending needed to reach those who do not face the same challenges. It costs more to tackle the costs of inequities that are deeply embedded within societies.

Equity demands more than spending the same amount on rich and poor students

Such problems can’t be solved by spending exactly the same on both a rich and poor child. While this would be an equal spread of spending, it would not, however, be equitable. Equitable funding provides additional resources to those who face inherited disadvantages. And it is the only way to achieve the global ambition of ensuring that every child enjoys the same opportunity to achieve their potential in education.

So, how close – or far – are countries to this ambition at present? Our findings are pretty stark: in many countries, children and young people who are the hardest to reach are often the last to benefit from government investment in education.

There are still large inequalities in the distribution of government spending on education within countries. As the figure below shows, there are vast wealth gaps in government spending within 31 low- and lower-middle-income countries in sub-Saharan Africa and South Asia for which data were available, with government spending benefiting the richest groups in all 31 countries disproportionately. In Congo, Guinea, Liberia and Malawi, children from the poorest 10% of households benefit from less than $10 for every $100 spent on children living in the richest 10% of households.

In most of 31 selected low- and lower-middle-income countries, the poorest children receive just a fraction of government expenditure on education

Note: As far as possible, the data have been matched for the years available for expenditure data from the UIS with the years for the DHS school attendance data, using the most recent year available. The years of data available ranged from 2005-2014. Data on government expenditure for all levels of education were analysed.
Source: Ilie and Rose (2017) based on calculations with UIS and DHS data.

We have also looked more closely at who, exactly, pays for education. The assumption may be that it is governments, but in many low-income countries, households contribute significant amounts of money to send their children to school. In several countries, they contribute 30% or more of the combined household and government funding for primary education.

Such contributions hit the poorest households the hardest, especially in areas where there is low investment by governments. A survey of 12 African countries found that household spending on learning materials swallowed up 56% of household expenditure on education for the poorest households, compared with 27% for the richest households.

Formula funding for equitable education

To ensure more equitable spending on education, we argue for the use of ‘formula’ funding that distributes a larger proportion of resources to those in greatest need. Formula funding is already underway in some countries but is not always tackling inequities as well as it could.

For example, formulae are often based on the principle of equality of funding, with funding allocated on the basis of the numbers of children in school. But this doesn’t factor in the need to provide education to harder-to-reach children who are more likely to be out of school. By failing to differentiate between the backgrounds of students in different locations, equality of funding approaches can be highly regressive. Where they exist, formulae usually focus on access, failing to overcome wide inequalities in learning between better-resourced and the poorly-resourced schools that are more likely to educate children from the most disadvantaged backgrounds.

There are, however, successful examples. In the Netherlands, a school funding formula is weighted in favour of the number of disadvantaged children in a primary school, mostly native Dutch children whose parents have little education, and disadvantaged immigrant children. Schools with large numbers of disadvantaged children have, on average, 58% more teachers per student and more support staff.

In Pakistan, education funding to provinces was disbursed according to their share of the population until 2009. This was a regressive funding formula, as it failed to take into account the relative deprivation of different provinces and their mixed ability to raise their own resources. To address this, four criteria have been used since 2009 to determine provincial allocations: population size, ‘poverty backwardness’, provincial revenue collection and low population density. These changes are intended to favour smaller, sparsely-populated and less-developed provinces.

Six steps for equitable funding

Our review of current funding formulae reveals six pointers for their effective use:

1. Formula funding needs to take account of both access and quality of learning.
2. Formula funding needs to take account of regional inequalities within decentralised systems.
3. Redistributive formulae need to include teacher salaries.
4. Schools need autonomy over the spending of resources, with guidance for using it in ways that address education quality for disadvantaged groups.
5. Even where quality issues are addressed at school level, schools need to ensure the resources reach the most disadvantaged groups.
6. The effects of redistribution on narrowing learning gaps take time. Governments need to be prepared to go the distance.

National Education Accounts offer a useful way to track education finance, particularly for vulnerable and marginalised groups. For example, a USAID-funded State Education Accounts project in Nigeria found a strong bias in Kano and Zamfara states among both public and private providers to fund schools in urban areas. As a result, state planners reassigned teachers from urban to rural areas, channelled more funds to girls’ schools in Zamfara state and allocated more funds for textbooks and maintenance.

Looking ahead, there will be little progress towards SDG 4 – inclusive and equitable quality education and lifelong learning opportunities for all – unless resources are redistributed equitably, with the most disadvantaged receiving the largest share of government resources and paying the smallest share from their own pockets. While these are still early days for such approaches, there are already valuable lessons to be learned to ensure that no one is left behind in education by 2030.