Why the ‘Most Thoughtful Metrics Can Still Screw Up a Newsroom’
April 15, 2014
Lisa Rossi


Today on AJR.org, Tony Haile, chief executive of Chartbeat, a media analytics company, recalls once meeting an editor of a U.S. metro daily who said she didn’t care if people read the stories “as long as they were clicking on enough pages.”

Haile says that the incentives plan the newspaper had adopted was making its journalists so focused on clicks that they had lost sight of how to build a core audience.

And that, he suggests, is one of the many problems created when newsrooms use metrics in the wrong ways.

Given the intensity of debates occurring in newsrooms today over best practices for web analytics, I think it’s worth reading Haile’s post. After all, he runs one of the key companies that sell real-time analytics to news organizations.

Two key mistakes he sees newsrooms making:

  • Paying based on metrics: Pay-for-performance systems have historically been found to “squash creativity,” leaving people to “stick with ‘what works’ and stop taking risks and exploring,” according to Haile.
  • Focusing on the wrong metrics, such as the pageview, to drive newsroom policy: Haile says the pageview, even though it is correlated to advertising revenue, measures the wrong thing – and that is the link. What it doesn’t measure is whether the visitor read the content, liked it or hated it.

Haile issues his cautionary words about linking newsroom pay to metrics as an increasing number of newsrooms are experimenting with performance-based pay.

For example, the Oregonian recently began measuring a large portion of reporter performance on metrics, including how often they post, according to Willamette Week.  On a smaller scale, the Savannah Morning News has begun experimenting with an incentive program based on pageviews — if reporters meet or exceed their monthly personal traffic goal, based on, in general, a 5 percent increase in pageviews, they get a $100 bonus.

“In the long run, incentive plans can lead journalists to stop relying on metrics as a trusted feedback loop and start seeing them as a cruel judge they need to satisfy,” Haile writes. “…Even the best, most thoughtful metrics can still screw up a newsroom if they are applied as a cudgel for compliance instead of a conduit of education. That often happens when metrics become the basis for incentive pay plans.”

I find it especially noteworthy that Haile is raising concerns about how companies are using certain metrics, given Chartbeat’s dominance in the market for real-time metrics. The company bills itself as one that works with 80 percent of the top publishers in the U.S.

When I worked as an editor at AOL’s Patch, my colleagues and I would commiserate over the addictive nature of Chartbeat, a tab that was always open on our computers, and for some of us, one that we checked first thing in the morning, even before breakfast.

It sounds like Haile is calling for, let’s say, a healthier way of consuming data on site performance.

Here is Haile’s post in its entirety. 

Comments