How we calculate agent rankings for chat & ticket satisfaction reports
As you might have noticed we don’t use an average rating formula (so avg=positive ratings / total ratings) when calculating chat & ticket satisfaction for each agent.
Instead, we use Wilson score (it’s okay if you don’t know what it means), which lets us calculate the probability of getting a positive rating, taking into account both the number of positive and negative ratings in relation to all ratings.
In this article:
Why we use Wilson score
Still not sure why we chose some complicated formula instead of the average?
Let’s look at an example:
Jake has 2 positive ratings and 0 negative ratings, while Anna has 10 positive ratings and 2 negative ratings. Who do you think should be ranked first? I bet you said Anna.
If we used the average to calculate rankings, it would be Jake that is ranked higher than Anna. Because 100% of his ratings are positive in comparison to Anna’s 83%.
That’s why we use Wilson score (you can see the exact formula for calculating it below). Because it says that Anna has a 52% chance of getting a positive rating, while Jake has only 33% chance. Which means that Anna is ranked higher than Jake. Which makes things fair.
Want to calculate Wilson score to see if it works? Check out the next section.
Calculate Wilson score for your agents
Have some spare time on your hands? Check if Wilson score actually works. Here is the formula we use to calculate it:
Z=2.69 (confidence level)
n=number of positive ratings+number of negative ratings (total number of ratings)
p=number of positive ratings/number of negative ratings (probability of a positive rating)
zznf=Z*Z/(4*n) (constant for keeping the formula short)
Wilson score = (p + 2 * zzfn – Z * √ ((zzfn / n + p * (1 – p)) / n)) / (1 + 4 * zzfn)
And an example:
Jake: 2 positive, 0 negative
Anna: 10 positive, 2 negative
Wilson score for each agent using the formula provided above:
Anna: 0.5194200469 (52%)
Jake: 0.3312764381 (33%)
The higher the score, the higher the probability of getting a positive rating. Which translates to a higher ranking of the agent.
Care to calculate Wilson score for your agents? Cool! We prepared an excel (xlsx) file, that you can download and fill in with your agents’ ratings. The score will calculate automatically. Download the Wilson score spreadsheet.