Inside the Digital Society: Measuring the impact of ICTs on development

I’ve been discussing, with colleagues, the challenge of measuring the impact of ICTs on the Sustainable Development Goals (SDGs) – the goals and targets the United Nations agreed in 2015. What data do we have? What data can we get? What can those data tell us?

The SDGs and ICTs

The SDGs are highly complex. They’re comprehensive, ranging from poverty reduction to empowerment, employment to environment. Most targets are for 2030 (a few for sooner). Most are ambitious, and some are looking difficult. It’s always hard to balance what’s desirable with what’s achievable in shortish timescales.

ICTs don’t feature much within them. One target calls for the world community to ‘significantly increase access to information and communications technology and strive to provide universal and affordable access to the internet in least developed countries by 2020.’ It’s 2020 now, and we’re a long way still from that.

Measuring the SDGs

Measuring progress (or lack of it) towards achieving SDGs is difficult.  

  • They’re intended to apply to all countries (not just developing countries), so starting points are very various.

  • Data availability and data quality are variable too – with the biggest deficits in countries with the biggest developmental challenges.

  • Data need to be disaggregated if they’re to be meaningful – not just between countries but within them. Between women and men, between rural and urban areas, between ethnicities, between age and social groups.  

  • We need data about trends, not just data about now.

  • And data aren’t enough to measure everything in any case. Availability and use are only part of the necessary picture: we also need to know about quality of services, about perceptions.  And qualitative measures that complement the numbers. It’s not enough that a policy exists, for example; it needs to be appropriate and implemented.  

  • It’s far easier to measure inputs than to measure outputs – yet it’s outputs, or even better outcomes and impacts – that matter most. Impacts are hard to measure because they’re not immediate and many of them aren’t expected.

  • Few things are binary. What looks like ‘progress’ from one angle, with one data set, can look like quite the opposite with different data from another angle. For instance, greater access to something deemed desirable – mobile phones, let’s say – may be accompanied by greater inequality. So there's gain, but also not so gain.

Data asymmetry

One final challenge that’s worth mentioning is asymmetry in data. It’s hard for governments to measure impacts because it’s hard for them to gather data. Look at any UN data set and you’ll see the difficulties: inconsistencies, for instance, in the way that different countries go about it; data that are out of date, incomplete or unreliable.

Yet at the same time global data corporations have enormous reservoirs of data that are central to their business plans.  Their data are not perfect for analysis – they’re based on customers, not citizens, which skews them to the better-off and certain age groups, for example – but they’re locked away behind walls of commercial confidentiality, serving business rather than public policy requirements.

Open data activists have concentrated on gaining access for everyone (including data corporations) to data sources generated by government departments. But much more developmental value would arise – for governments and civil society – from wider access to data held by private corporations. Time for a rethink?

The impact of ICTs on SDGs

It’s been difficult, therefore for the United Nations to establish data sets that measure SDGs or measure the impact of ICTs on SDGs.  A new set of indicators for the latter’s been developed and will be adopted later in the year, but it’s limited in scope and constrained by variable data quality.

What data do we need to measure the impact of ICTs on SDGs?

Overall, we need data that meet the challenges that I’ve identified in bullet points above.  

  • Data that are accurate, up to date, comprehensive and can be disaggregated.  

  • Data that enable trends to be identified – especially the direction and the pace of change.  

  • Data that address the downsides, rather than just upsides – for example, counting not just the number of new devices but also what happens to the old.

  • Data that consider outputs and outcomes, rather than just inputs.

And we need data analysis that’s concerned with understanding rather than with ticking boxes. That requires nuance – qualitative corroboration as well as sophisticated number-crunching.  

It requires analysts to look for unexpected outcomes rather than focusing only on targets and expectations.  

It requires policymakers to understand that the purpose of the exercise is to learn not spin: to use the findings of analysis to improve policy and practice rather than to boast or moan about their standing in league tables.

Four types of data

So what types of data do we need to gather, analyse and reflect upon when we try to learn about and from the impact of ICTs on SDGs?  I’ll put them in four sets, briefly today, with more to come in blogs later in the year.  All four, I’d suggest, need to move on from where they’ve been.

Data about ICTs

The first concerns the ICTs themselves.  Mostly input data.

In the past these have been better covered than data on impact – for example in data sets gathered and published by the International Telecommunication Union (ITU).

Data concerned with infrastructure and devices – how much bandwidth, for example, how many computers or how many mobile phones.

These need to be expanded now to cover a wider range of products and services – including software and apps, data traffic volumes, access to the cloud, numbers of skilled technicians and computer scientists.  

It would be useful, too, to measure a country’s or a sector’s dependence on imported software, hardware and expertise.

Data about usage

The second set concerns the use of ICTs. Here, too, the ITU and others have gathered data, though what they have’s a tiny fraction of what’s gathered, analysed and leveraged by IT businesses.

To date these data sets have focused on how many people use their devices and, more recently, some aspects of equality (the gender digital divide especially).  

Much more of this is needed, with much more complexity and disaggregation - but also more analysis of types of use (including social media and e-commerce), and more understanding of the integration of ICT products and services in daily lives. Data on how people use devices, apps and services, not just on whether they are using them.  

More understanding too of how changes in usage are impacting on older, more traditional products and services. Digitalisation is part of wider social and economic change, but how?

Data about impact

The third set, and the most important now, is the interface between ICTs and policy, practice and the SDGs.  

  • What impact is digitalisation having now on different aspects of economy, society and culture; on each and every SDG and target?  

  • What trends are evident and which are positive or negative (from an SDG perspective)?

  • Who is gaining, who is losing, and how might that be changed (if change is needed)?

The ITU and others are working hard to identify effective ways of measuring these impacts (and I am pleased to be involved in the work they’re doing).  

It’s crucial (and very difficult) to find indicators that are achievable, reliable and relevant.  Crucial, too, to avoid assuming that inputs lead to impacts or that impacts at the level of whole populations are consistent across different social groups. Crucial to look for unexpected impacts. And crucial, also, to measure negatives such as e-waste and cybercrime.

Indicators of what’s happening in the longer term

My final set is concerned with what’s happening in the longer term – ‘third order impacts’ as they’re sometimes called, beyond the timescale (but not the themes) of SDGs.

We know that digitalisation is having long-term impacts on economies, societies and cultures. We know that these are complex and likely to be profound. Some will be positive but some will not.

The world thirty years from now will be as different from today, in terms of its technology, as today’s world is from that of the early twentieth century. Digitalisation’s changes will interact with others, especially with climate change and with the shifting sands of geopolitics.

I’ll pick out four areas in which I think it’s critical that we build our understanding here of likely impacts and how we can shape them:

  • equality/inequality and changing (economic, social, political) power structures;

  • patterns of economic production and consumption;

  • urbanisation; 

  • and environmental change.

Understanding (and shaping) these will require sophisticated data and foresight analysis.  We’re very far from either.

Next week: on the risk that digitalisation leads to digital exclusion

David Souter writes a weekly column for APC, looking at different aspects of the information society, development and rights. David’s pieces take a fresh look at many of the issues that concern APC and its members, with the aim of provoking discussion and debate. Issues covered include internet governance and sustainable development, human rights and the environment, policy, practice and the use of ICTs by individuals and communities. More about David Souter.
APC-wide activities: 
« Go back