Data for good: RSS 2025, Edinburgh

Every time we tap a loyalty card, navigate using GPS, or scroll through social media, we leave digital breadcrumbs that collectively paint a picture of human behaviour. This ‘smart data’, generated through our everyday digital interactions, offers researchers and policymakers new opportunities to understand social challenges, from poverty and inequality to population movements and economic shifts.

But with this treasure trove of information comes profound questions: How do we ensure this data truly represents all of society? How do we protect privacy while maximising public benefit? And most importantly, how do we translate these insights into policies that genuinely improve people’s lives?

These were the central themes explored during a compelling panel session I had the privilege of chairing earlier this month at the prestigious RSS International Conference in Edinburgh.

The event brought together nearly 1,000 members of the statistics and data science community, and I was delighted to facilitate a discussion on “Doing good things with data”, a mission that sits at the heart of our work at Smart Data Foundry.

Four transformative projects

Our panel brought together four researchers from Smart Data Research UK’s accelerator programme, each grappling with how to ensure insights from the digital world can benefit society ethically and efficiently.

Revealing hidden health needs through shopping data

Anya Skatova, Associate Professor at the University of Bristol, is pioneering work that could revolutionise how we understand health and wellbeing. Her project explores linking shopping data from loyalty cards with longitudinal population studies. The potential is striking. For example, women self-treating menstrual pain won’t appear in NHS records, but their purchasing patterns for pain relief and menstrual products tell a different story. Similarly, people managing chronic pain often buy over-the-counter medications rather than visit doctors, creating invisible health needs that smart data can illuminate.

The challenge lies in creating robust frameworks that protect anonymity while enabling researchers to access this enriched data. As Anya explained, “The value for the research community of these digital traces is undeniable, but how do you create the right framework so everyone can benefit effectively?”

Understanding political behaviour in the digital age

Political scientist Professor Rachel Gibson from the University of Manchester is investigating how to link smart data with traditional survey responses. Rachel and her team are focusing on recent elections in the UK and the US – they want to find out whether it’s possible to get a “360-degree perspective of people’s attitudes” during particular periods in time.

Currently, there’s no accepted standard for this work. Rachel Gibson’s research aims to establish the methodological foundations and ethical guardrails needed to safely combine browser data and social media information with survey responses, all while meeting GDPR requirements and FAIR principles (Findable, Accessible, Interoperable, and Reusable).

Mapping the modern economy

Professor Emmanouil Tranos from the University of Bristol is addressing an equally important challenge: how do we track the emergence of new industries and jobs in our rapidly changing economy? The official Standard Industrial Classification (SIC) codes, used to categorise companies, haven’t been updated since 2007, which has created gaps for researchers wanting to understand modern economic activity.

Emmanouil’s project analyses ‘web archives’ – the descriptions companies write about themselves online – to identify and classify new types of businesses and employment. This work promises to give the research community much more timely and open-source insights into economic transformation, from the gig economy to green technology sectors.

Tracking population movements for better crisis response

Professor Francisco Rowe from the University of Liverpool is tackling a fundamental limitation in understanding population change. Traditional census data, collected every decade, provides only static snapshots of where people live. His DEBIAS (de-bias) project gathers mobility data from smartphones and digital platforms to create dynamic, real-time pictures of population movement.

This research has immediate policy applications. Francisco already advises the UN on using location data to track migration patterns and populations during crises. As he pointed out, during the early days of the war in Ukraine, emergency organisations knew people were crossing into Poland, but they didn’t know what happened to them afterwards. Smart data might have helped to track their subsequent movements and identify where displaced populations settled, crucial information for delivering humanitarian aid and support services.

The promise of smart data and the challenge

Three game-changing advantages

What emerged clearly from our discussion is that smart data research offers three transformative advantages: speed, detail and cost-effectiveness. As Emmanouil Tranos noted, “We can now capture ever more detailed insights far more effectively – time, granularity and cost are making smart data research much more attractive to the research community.”

The bias challenge

But this potential comes with significant challenges. The most pressing is bias. Smart data isn’t collected randomly from across society – it’s generated by people who use smartphones, have loyalty cards, and engage with digital platforms. This means it often over-represents younger, tech-savvy populations while potentially missing older residents or deprived communities with limited technology access.

Innovative solutions in development

The researchers are developing innovative solutions to address these limitations. Rachel Gibson’s team is looking at the potential for generating a ‘traffic light system’ for data reliability: green for data they consider reliable, amber for less reliable sources, and red for “use at your discretion.”

Francisco Rowe’s DEBIAS project is establishing standards for identifying and correcting mobility data biases. These methodological advances will be crucial for ensuring smart data research produces insights that truly represent all of society.

From research to real-world impact

Perhaps the most exciting aspect of this work is its potential for policy application. Unlike traditional research that can take years to influence practice, smart data insights can inform decision-making in near real-time.

Consider the policy implications of Anya Skatova’s shopping data research: understanding previously invisible health needs could help the NHS better allocate resources and develop targeted public health interventions. Rachel Gibson’s work on digital political behaviour could enhance our understanding of democratic processes and help identify and counter misinformation campaigns.

Francisco Rowe’s mobility research has already proven its value in emergency response—the UN has struggled for years to get timely statistics on population movements during crises, and this work could help fill that critical gap. Emmanouil Tranos’s economic mapping might help policymakers identify emerging industries earlier, enabling more targeted support for innovation and job creation.

Building ethical frameworks for the future

As our panel discussion revealed, the smart data research field stands at a crucial juncture. The technical capabilities exist to extract unprecedented insights from our digital footprints, but we need robust frameworks to ensure this work benefits everyone while protecting individual privacy and rights.

This requires bringing policymakers into research projects earlier in the process, as Emmanouil Tranos emphasised. But as Anya Skatova cautioned, we must balance the urgency of policy needs with the rigorous scrutiny that ensures research quality, “Policymaking happens fast, but the research process is slower, especially after it’s been through the rigours of academic scrutiny.”

The researchers are already building bridges between academia and practice. Anya Skatova has established the interdisciplinary Digital Footprints Conference to foster collaboration across fields. Meanwhile, Rachel Gibson is focused on establishing the ‘plumbing’, the fundamental standards and guardrails needed for safe smart data research.

A digital future that benefits everyone

As I reflected on our panel discussion, what struck me most was the researchers’ commitment to ensuring their work genuinely improves people’s lives. Smart data research isn’t just about technical innovation – it’s about creating a more equitable and responsive society.

The digital footprints we leave behind are becoming society’s memory, capturing patterns of behaviour, movement, and need that traditional data sources can miss. By learning to read these traces responsibly and inclusively, we can develop policies that respond more quickly and accurately to citizens’ actual experiences and needs.

The work showcased at our RSS panel represents just the beginning. As these methodological foundations are established and ethical frameworks developed, smart data research will increasingly inform everything from healthcare delivery to urban planning and economic development.

The challenge – and opportunity – lies in ensuring this new capability serves everyone. As we continue to develop this field, the principle of “doing good things with data” must remain our guiding mission.

Gordon Baggott is Chief Data Scientist at Smart Data Foundry, home to Smart Data Research UK’s Financial Data Service (FINDS)