Ethnographic study on use of metrics in newsrooms

The Traffic Factories: Metrics at Chartbeat, Gawker Media, and The New York Times
A report by Caitlin Petre
Tow Center for Digital Journalism
Graduate School of Journalism at Columbia University

Audience metrics have become ubiquitous in news organizations, but there has been little empirical research on how the data is produced or how it affects newsroom culture and journalists daily work. The Tow Center sought to understand how the use of metrics changes reporters behavior and what this means for journalism. Thus, researcher Caitlin Petre conducted ethnographic analysis of the role of metrics in journalism, focusing on three case studies: Chartbeat, a dominant metrics vendor; Gawker Media, a newsroom intently focused on metrics; and The New York Times, a legacy news outlet where currently metrics were more peripheral. Petre offers the following key points based on her findings. The full report is online here, and available as a pdf and ebook downloads here.

• Metrics exert a powerful influence over journalists’ emotions and morale.
  Metrics inspire a range of strong feelings in journalists, such as excitement, anxiety, self-doubt, triumph, competition, and demoralization. When devising internal policies for the use of metrics, newsroom managers should consider the potential effects of traffic data not only on editorial content, but also on editorial workers.
• Traffic-based rankings can drown out other forms of evaluation.
  It is not uncommon for journalists to become fixated on metrics that rank them or their stories, even if these are not the sole criteria by which they are evaluated. Once rankings have a prominent place on a newsroom wall or website, it can be difficult to limit their influence.
• News organizations can benefit from big-picture, strategic thinking about analytics.
  Most journalists are too busy with their daily assignments to think extensively or abstractly about the role of metrics in their organization, or which metrics best complement their journalistic goals. As a result, they tend to consult, interpret, and use metrics in an ad hoc way. But this data is simply too powerful to implement on the fly. Newsrooms should create opportunities—whether internally or by partnering with outside researchers—for reflective, deliberate thinking removed from daily production pressures about how best to use analytics.
• When a news organization is choosing an analytics service, it should consider the business model and the values of the vendor.
  We have a tendency to see numbers — and, by extension, analytics dashboards — as authoritative and dispassionate reflections of the empirical world. When selecting an analytics service, how- ever, its important to remember that analytics companies have their own business imperatives. Newsroom managers should consider which analytics companies values, branding strategy, and strategic objectives best align with their own goals.
• Not everything can — or should — be counted.
  Efforts to improve audience analytics and to measure the impact of news are important and worthwhile. But newsroom, analytics companies, funders, and media researchers might consider how some of journalisms most compelling and indispensable traits, such as its social mission, are not easily measured. At a time when data analytics are increasingly valorized, we must take care not to equate what is quantifiable with what is valuable.