CIBAR Conference 2018
15th – 18th April 2018
Deutsche Welle, Bonn, Germany
IMPACT, INFLUENCE AND ENGAGEMENT
From reach to impact: how the DW achieves its audience impact goal
Kim Schroeter, Leah Winnett (Deutsche Welle)
The uninhibited access to information and the ability to express oneself freely are key to a peaceful and democratic society in a globalised world. DW helps to promote freedom of opinion and freedom of expression globally through its journalism.
By providing independent and reliable information, DW strives to fulfill its primary impact objective: to empower people around the world to form their own opinions and participate in public discourse.
An evaluation was carried out in order to verify whether or not this goal was achieved. The impact evaluation was based on a multi-method approach. The core of the analysis was an impact model: it sets out suspected linear correlations according to the “cause and effect” logic. These were tested using the appropriate methodology, in most cases, through surveys and interviews.
In the presentation we will share our definition of impact in relation to media. We will discuss the benefits and challenges of using a model for measuring impact, in particular looking at the DW’s design as a case study.
How deep is your impact? Is your content “AAA”?
Alena Ivánková & Martin Vejvoda (RFE/RL)
The mission of Radio Free Europe/Radio Liberty is to promote democratic values and institutions by reporting the news in countries where a free press is banned by the government or not fully established, based on the conviction, that the first requirement of democracy is a well-informed citizenry. To measure our success, we need to look at impact of our work on three levels – our three As:
- A1 – Awareness: Does our content reach people?
- A2 – Attention: Is it meaningful to them?
- A3 – actions: Which changes does it bring?
Each of the three levels is important to assess our effort. And each of them requires specific metrics to evaluate how successful our work is. And knowing what to measure is also crucial for all teams in order to learn what works and what does not so they can develop better and more successful strategies the next time. In the presentation, we will share how we measure all three layers of “AAA” success, show user cases and discuss benefits as well as challenges of the research strategy.
Measuring Networks of Influence on Twitter. Why, How and What Then?
Alistair Willis (Open University)
Measuring media influence is difficult, inconclusive, unreliable and, for some, even impossible. Yet it remains one of the most significant and persistently stubborn questions across the history of audience research – and no less so since the advent of social media. Twitter, according to some, lends itself to measuring networks of influence, but how robust are the tools of measurement and the findings? This presentation describes our exploratory attempts to analyse networks of influence around RT using tools of computational science. We offer some preliminary findings setting out a typology of followers and we reflect on clusterings of followers. We explore the implications of RT’s appeal for international broadcasters and the strategies they adopt to influence audiences. But we also aim to provoke wider discussions: Who is RT trying to influence and why? Are their goals and strategies of influence so very different to other international broadcasters? If influencers can be identified, what then?
Understanding how foreign music broadcasting changes behaviour inside authoritarian governments
Karl G Feld (Vice President, Social and Strategic Research, GfK and PhD Candidate, North Carolina State University)
In January of 1983, Sonderzug nach Pankow was broadcast on Radio in the American Sector (RIAS). Its content was one of a series of reputational challenges posed by RIAS against the government of the DDR. This case study uses declassified documents of the DDR’s secret police (Stasi) to understand how and why the behavior of DDR government changed to accommodate following this influential performance. Comparisons are drawn to similar accommodating behavior by the government of today’s Iran to provide generalizable conclusions which can be used to understand how foreign musical broadcasting influences and authoritarian governments decide how to respond.
A bigger bang for your public diplomacy buck: market research and international communications
Josh Miller & Paul Kruchoski (U.S. Department of State)
“Our comms budget is too small to reach EVERYONE in this country about EVERYTHING – so WHICH AUDIENCES should we be engaging and WHAT ABOUT” Every American public diplomacy officer asks that same question, and to answer it, the U.S. State Department has adapted private sector market research techniques. In this presentation, the Public Diplomacy Research and Evaluation Unit will present select country reports and examples of how this technique is producing concrete, actionable insights on targeting, message options, and media choice for U.S. communications to foreign publics.
Experimenting with audience engagement strategies via real-time reconstructions of historical events: What can we learn from RT’s social media re-enactment of the Russian Revolution?
Rhys Crilley (Open University)
International broadcasters constantly experiment with audience engagement strategies. Recently, novel uses of real-time reconstructions of historical events via participatory social media projects have gained a certain popularity – perhaps as an antidote to the instantaneity and present-day preoccupations of social media? Reconstructions of the 1914 Irish Easter Uprisings on Irish social media, for example, preceded and influenced RT’s commemoration of the centenary of the 1917 Russian Revolution in November last year. Over 90 revolution-era characters tweeted the revolution in ‘real time’ ‘as if’ it were happening today. Based on an analysis of a strategically selected corpus of tweets and focus group interviews with user-participants, we assess the extent to which RTs audience engagement strategy was successful and we reflect on what counts as success beyond ratings. We show how RT’s 1917LIVE project engaged users in unique ways through its participatory and immersive experiences and highlight the implications of such projects for international broadcasters.
RESEARCH AROUND THE WORLD
RT and the shifting sands of international broadcasting in the Middle East
Deena Dejani & Marie Gillespie (Open University)
RT is a hot topic in recent weeks in the UK with key government figures calling for its closure in the aftermath of the Skirpal affair. RT in Europe remains widely rebuked, resented and distrusted. Its TV audiences in Europe are fairly miniscule, even if its reach and engagement with social media users fares better. The only region in the world where RTs audiences appear to be on a steady rise is in the Middle East – Syria, Iraq and Libya especially. The crisis within Al Jazeera Arabic, and the pulling back of the BBC Arabic Service’s operations in the Middle East, have arguably increased the significance of RT Arabic’s presence in the region, particularly given its ‘alternative’, anti-American coverage of the latest developments in Syria, Iraq and elsewhere. This presentation develops insights gleaned at CIBAR’s 2017 panel on RT and presents preliminary findings of our investigations into RT Arabic. It will contribute to our understanding of RTs shifting significance in the region as the geopolitical landscape undergoes convulsive shocks. It will examine how diverse audiences and users of RT Arabic respond to its coverage in general and to key moments in the war in and on Syria.
Innovative longitudinal CATI surveys across multiple countries: trials, tribulations and success
Edward Duckhorn & Suteera Nagavajara (ICF, Rockville Maryland)
Utilizing a meta-data driven application allows data to drive the features and functionality of the system, resulting in more efficient creation, reuse and editing of survey components. The removal of all images and the reliance on a text only user interface creates a straightforward economic design that renders efficiently under poor bandwidth conditions. Google’s streamlined user interface, Material Design Lite, allows for the user experience to be based on colors and animations which provide visual cues to the user. Further refinements are made by relying on content delivery networks (CDN) to accelerate the delivery of content by utilizing servers across wide geographic locations. The meta-data database architecture allows for the deployment of new questionnaires, questions, response options, skip patterns, and validations through database scripts and not application code updates. An application parser is employed to read the meta-data and display the content dynamically. This allows for in the field data collection to continue while a new questionnaire, or some part of a questionnaire, is being deployed.
A recent study ICF conducted for a client in the social media industry utilized this meta-data driven application approach to coordinate data collection across four countries into one centralized relational database. This approach ensured the consistent formatting and storage of data, significantly shortened deployment time and streamlined analysis and reporting. This study with one centralized online survey system used a computer-assisted telephone interviewing (CATI) technique with varying capacities in the four countries. Many technical challenges were encountered and adjustments were made to accommodate the local capacities to successfully complete the study.
From on the ground research to market: reconnecting with digital media users in Africa
Franck Buge (TV5 Monde)
Internet penetration in Africa is still low by Western standards at 35% but growing steadily. Yet behind the headline figure lie multiple user profiles, from basic messaging to video-rich and high frequency mobile consumption. In early 2017 francophone broadcaster TV5MONDE launched qualitative research in multiple countries to gain insights into the Internet markets of French-speaking Africa. A total of 5 user populations were identified from the focus groups and each was subsequently plotted against the large-scale Africascope study for volumetrics. These research findings helped shape TV5MONDE’s latest digital offerings for African audiences.
Firewall climbers. Who are they, what do they do and what do they think about China today?
Betsy Henderson (Radio Free Asia)
Radio Erena’s audience – Quantitative research with recently fled Eritrean refugees in Ethiopia and Sudan
Christoph Spurk & Jan Lublinski (DW Akademie)
Radio Erena, an Eritrean exile radio supported by Reporters Without Borders, has in the past years broadcast independent information to Eritrea. DW Akademie was tasked to analyse the radio’s audience – the challenge being that research inside Eritrea is not possible. Instead, a quantitative survey amongst recently arrived refugees in Ethiopia and Sudan was conducted. The number of listeners was assessed, including age, education, urban-rural and gender characteristics, as well as program preferences and the impact of the radio on its listeners. The results were used to identify shortcomings in reaching specific audiences and gaps in programming content. The focus of the presentation will be on how a quantitative survey with refugees was used to estimate the size of the radio station’s audience and its characteristics.
Understanding News Consumption in the MENA region
Athanas Jamo (IPSOS)
news.Connect, a cross-national study about media usage in the MENA region, is done by Ipsos yearly to explore and identify viewers of international news channels at different levels. In 2017, news.Connect surveyed 17,100 respondents across 14 countries. The research explores people’s news consumption habits and behaviors in the MENA region, and highlights key sources and devices used for obtaining information as well as platforms people rely on for different news information. Even though more people are shifting to digital platforms for news consumption, TV remains to dominate in terms of usage and credibility.
Media and Information Literacy: A Comparative Survey
Dennis Reineck, Jan Lublinski & Roland Schürhoff (DW Akademie)
With fake news and hate speech on the rise, Media and Information Literacy (MIL) is the set of skills media users need to deal with media and information sources (including social media) confidently and competently. DW Akademie has developed a questionnaire for measuring Media and Information Literacy (MIL) cross-nationally in three different countries (Jordan, Moldova and Uganda). The challenge was to develop a test that appraises knowledge and skills rather than merely attitudes and reported behaviour, in order to attain a more valid picture of respondents’ levels of MIL, while at the same time being viable in three highly different cultural contexts. The rationale behind the test will be presented, focussing on the structure and design of the test. First results of the Media and Information Literacy Assessment (MILA) survey will be presented.
Who Needs Audience Researchers? Data Literacies in International Broadcasting Organisations
Marie Gillespie (Open University)
Journalists now have abundant access to social media monitoring tools alongside conventional audience data. Some adopt a DIY approach and/or consider that they don’t need research – their creative instincts and professionalism are enough to ensure quality content. Strategy and management teams also often consider themselves to be not only digitally literate but also data literate. Ratings still rule as do number crunching certainties but in an age of big data and data deluge how are we to navigate our way as researchers inside organisations? Some organisations even consider that it is high time to dispense with dedicated research and audience insights departments. Others, blur the boundaries between research for marketing, accountability, development, strategic planning. Budgets are small and shrinking and more for less is being demanded. Rarely, it seems, is research actually mobilised for creative content production. It is very hard to get people to sit up and listen to research findings. CIBAR itself reflects some of these changes.
The changing role of researchers within broadcasting organizations – the FMM experience
Claire Marous-Guivarch (RFI)
NAVIGATING THE DIGITAL SPACE
The grand chess board in the web world: What the web was and what it became. How user behaviour is changing and what we can learn from it?
Julian Reale (DW Market and Audience Insights)
The underlying dynamics of power on the Web have drastically changed in the last years, and the three dominating tech giants (respectively Google, Facebook and Amazon) are at the center of a fundamental transformation of the Web.
The presentation intends to analyze the challenges facing publishers and its media research in the context of changing user experiences in a web that is increasingly dominated by tech giants. What does the Web became, what will it become? How does this effect the dynamics of user behaviour? And how can publishers adapt to this and find new insights in order to continue understanding its audience?
The presentation wants to focus on the ongoing dynamics and the perspectives for publishers and will provide proper numbers that illustrate these developments.
Richard Alford (Senior Research Executive BBC World Service)
How do we drive people to owned sites from social media/ aggregators, following research proving that content goes unnoticed/ unattributed in third-party space, and that image is more important than headline for further engagement.
This research explores what makes images ‘thumb-stoppable’ and aids understanding on what attracts audiences to particular digital platforms and how to retain them.
International Media and the Digital Paradox
Hayk Gyuzalyan (Methods Director Kantar Public, Brussels)
Kantar’s annual Connected Life initiative surveys 70,000 respondents living in 56 countries across all continents on multi-platform media usage and behaviour, as well as on topics such as trust in digital data, platforms and services, and governments and institutions. Connected Life 2017 showed a continuing Digital Paradox: the more citizens are connected, the less confident they are in our connected world.
In our presentation I will discuss how levels of trust change depending on source of information and how they differ across different world regions. I will look at the key factors impacting on the level of public trust, and the implications for international media.
Something went viral on social media. Is that good news?
Daria Kutkovaya (RFE/RL)
Now, what if we admit that something, say a video of cute cats, has very little to do with either journalism or the broadcaster’s mission? In this presentation, we will take a look at few examples of very successful videos published by RFE/RL language services. We will study them in terms of numbers, reasons for their success and the results they brought for the company. We will also try to answer the question whether “cat videos” are things that a news organization should be sharing and look at some of the ethical questions around benefiting from obsessions on the internet.
Social Media: What works? Impact case studies from Bangladesh and Libya
Sally Gowland (BBC Media Action)
At BBC Media Action, the media landscapes of the countries that we work in – mainly fragile states – are changing fast. In Nigeria, over half of people now use the internet, and in Myanmar, mobile phone usage has increased from 12% in 2013 to around 80% now, with the majority of people using smartphones.
Our work makes the biggest difference when we succeed in getting people talking, whether face-to-face or across virtual networks. Social media enables such discussion, broadening it beyond geographically defined communities and existing editorial agendas. Social platforms also allow us to see how people respond to our content and debate the issues we raise in our programmes.
Yet despite the optimism about what social media can achieve, evidence appears limited. Research conducted as part of our social media projects at BBC Media Action has sought to better understand if, and how, these projects are having impact and why. This session will present key insights from research which has explored engagement and impact from a few of our social media projects – in Bangladesh, an experimental design which looked at the impact our youth focused Hello, Check! Facebook page on adolescents and their understanding of health issues and in Libya El Kul (For Everyone) which aims at informing young Libyans and building a less divided media landscape in Libya.
Rising stars of digital media: podcast and smart speakers
Nino Japaridze (DPhil, Vice President, Edison Research)
Edison Research has produced extensive research on the usage of digital media in the U.S. This session will give you the highlights from three recent Edison studies: The Infinite Dial, Share of Ear and The Smart Audio Report from NPR and Edison Research. The Smart Audio Report looks at trends in the ownership of Smart Speakers and behaviour of their Smart Speaker users. Our research reveals how voice-activation in one’s home environment is fuelling adoption of smart technology and affecting how often consumers engage with other digital platforms throughout the day. Edison has been tracking U.S. trends in digital audio, social media, mobile, and other media platforms for over 20 years with its Infinite Dial study. Edison’s Share of Ear research quantifies both reach and time spent with all forms of audio and is the gold standard for understanding how Americans consume audio.
Challenges and potential solutions of combining “reach” measures for digital media and traditional broadcasters
Diana Turecek (Director of Research, MBN)
Isn’t it all just research?
Paul Tibbitts (RFE/RL)
This presentation will discuss some of the challenges and opportunities presented by integrating traditional market research and digital analytics. At the end of the day, both present information that can help inform programming and strategy. Each type of input has value and merit and researchers need to have appreciation for the strengths and weaknesses of each as well as a basic understanding of the different inputs are collected. Today, researchers need a more diverse toolkit to better answer the questions asked by researchers and management. Currently, each side tends to have value judgements about which numbers are better and whose numbers are better. At the end of the day, numbers are just numbers and researchers need to be rigorous in terms of how all data are gathered regardless of the method or platform. And all good data have value in research if used appropriately. RFE/RL will share examples showing how traditional research and digital analytics are integrated in evaluation and strategic planning.
Simplicity in digital dashboards: Helping teams focus on what matters
Senior Research Executive BBC World Service
How do we keep the business across reach and growth throughout the year, ensuring there are no nasty surprises at the end of year review? What are the appropriate metrics for quantifying digital success and how do we measure success across platforms?
BBC World Service has created an internal dashboard which does just this:
- Provides an easy to view look across the platform with focus on the metrics that matter
- Captures and unifies digital metrics from different analytics sources to provide both a platform level and holistic view of audience behaviour
- Highlights content level depth which enables easy to identify stories that are working and not working
- Ensures performance is not viewed in isolation by providing contextual insights
Go big or go home? Using algorithms in large number qualitative research
Lennart Gerwers & Julian Reale (Deutsche Welle)
Remesh is a software platform that uses algorithms in online focus groups to analyze, understand and segment audiences’ responses. The browser-based software application is a new, convenient way to moderate virtual focus groups with comprehensive possibilities and analysis in real-time.
In cooperation with DW the US-based company Remesh will present use cases and its methodology behind the algorithm. It will demonstrate how it can help to understand the thoughts and opinions of any number of participants in real-time and invite to discuss the new possibilities in terms of enabling qualitative research to be conducted at a quantitative scale.