Measurement 101: What does good look like?

In this second of the series of blogs on developing a measurement framework, I want to cover what I think is the most important part of all – the metrics.

There are many ways to measure Communications and many metrics to choose from, and the selection of the appropriate metrics is key. The metrics must be relevant to the objectives of the team or they can be highly detrimental to those objectives.

Moving beyond AVE

A case in point – a while back now (15 years ago! Ouch!) I was working with a high street fashion retailer. Not unlike a lot of PR teams at the time, their key measurement metric was AVE. For the lucky few reading this, who have not been burdened with AVE for the last 20 years, AVE was an attempt by the PR industry to put a financial figure to earned media coverage by measuring the size of the coverage in sq cm and applying the advertising rate card to this. A big number and a happy Finance director.

The issue with the fashion retailer I was working with was with the behaviour this metric caused. Most of the team were trying to get coverage in small regional publications that were syndicated in order to multiply the AVE by the number of times the article was repeated. At first glance it appeared that the PR team were very successful but in reality they had gamed the system. The fashion retailer was aimed at women between the ages of 16-21 and all I had to do to change this metric was to ask the Chief Financial Officer ‘how many women aged 16-21 do you think are reading regional newspapers?’

This was a while ago now and it has become a useful story to tell clients when we are building the foundations of a measurement framework. My advice is to start with what good will look like and the metrics that are aligned with objectives of the team or indeed the business. In order to uncover this a little more I ask what data their board would want to see? What does their board care about and how does the PR team support this? This serves as a focus, a focus on what we want to achieve rather than what will look good.

Selecting the right metrics

Let’s take a look at some metrics and see how useful they are and where they might support this.

Volume

Pure volume in itself is not a particularly useful metric. ‘We got more coverage than last year, we got more coverage than competitor X.’ This might look good but it doesn’t pass the ‘so what’ test and I can’t imagine any board really caring about this. It’s a useful tactical metric but in terms of strategic goals the volume of coverage and the raw share of voice against competitors doesn’t provide us with any insight.

Below is a raw share of voice chart for Amazon in a set of global media titles related to volume only. It’s interesting to see but it is not telling me anything. At this point I have no idea what forms each competitor’s coverage in terms of sentiment or strength of mention.

Sentiment

Sentiment is an excellent indication of perception. How do Journalists perceive us and how is this represented in the media? Understanding if coverage is negative or positive gives your board a better indication of how the company is perceived. On its own it is useful, when added to other metrics it becomes even more interesting. More on this later.

Salience

The artist previously known as prominence. I always felt that prominence was a bit blunt, the aim being to get a headline mention or a mention in the first paragraph of the article. Salience allows a much more granular understanding of the dominance of coverage.

At Signal AI we will score each mention of a company determined by where it is an article and the strength of the mention. In itself this is an incredibly useful metric. If you add it to volume you get immediate context. 1,000 articles with high salience supports your objectives much more than 1,000 passing references. Add this to the share of voice against competitors, correlate it with sentiment, and you can demonstrate dominance in a market place. This is starting to sound like supporting a business objective.

Below is a reconfigured share of voice chart that includes high salience articles only in the same set of media as the previous chart. Interestingly we can see that Apple are winning when we focus on strength of mention.

Themes/Topics/Messages

This will tell us if we are associated with the topics that we want to be. We might have a strategy to be thought leaders in sustainability issues or to have a voice on issues such as diversity and inclusion. These metrics have a tangible link to business objectives. Investors will want to be sure about the ESG credentials of a business, companies will want to hire the best talent.

Managing the topics that a company is associated with should be a key part of any measurement programme. In the same sample of content as the chart above (high salient coverage in selected global media) I can see that Google’s voice on workplace topics is strong and Apple clearly have work to do in this area.

Volume is not enough

With a carefully targeted measurement framework I can demonstrate that the coverage I have gained or managed in the media is portraying my brand as I wish it to, that the sentiment is on track, that I am dominating the coverage, that I am supporting my business in all kinds of different ways. It is no wonder that people do not value a volume based approach, and until the advent of AI it was all that was available at a reasonable budget point. It caused our industry to measure with the most accessible metric, AVE.

This wasn’t only to the detriment of perception of PR, but to the detriment of doing good work. Bad metrics can cause bad behaviour, and using the right metrics can help the PR team demonstrate that it is aligned with and supporting the overall business objectives of their company or brand.

Access to more reliable quantifiable reputation data could improve business performance by an average of 63%, according to a survey of 1,000 business leaders by Signal AI. It is easy to see why, with the information available on the Signal AI platform I am able to accurately measure performance, set benchmarks for improvement and have access to the data that will inform my decision making and communications strategy.