The Simple Metrics - An Explainer

Wednesday 25th March 2020
Author: Sam Watson

The Simple Metrics

Followers of EquiRatings know our ‘why’ pretty well at this stage. In fact, almost everyone in every walk of life knows the ‘why’….in order to manage, you must first measure. We wanted to make some of our calculations easy to do and transparent to riders and fans at all levels to make data-informed decisions.

How We Measure

There are two approaches to metrics in sport.

  • The first is quite simple, it’s about summarising past performance.
  • The second is more complex, it’s about predicting future performance.

Summary metrics are factual. They do exactly what they say and there is nothing to argue with – an average is what it is. The only question is whether or not a summary metric is useful or adds value to the conversation.

In high-performance sport, there is one result that really matters and that is the next one. So, what management finds useful is a predictive metric. Most summary metrics have a degree of predictive power as past performance tends to be a decent indicator of future performance, but predictive ability isn’t the only use for a summary metric. When it comes to fan engagement and media reporting it is very useful to be able to quantify performance and achievement. ‘Who beat the most opponents’, ‘who had the lowest average’ and ‘who had the highest clear rate’ are all interesting to answer with quantifiable and factual metrics.

Summary metrics are accurate (e.g. an average in a season), whereas a predictive metric will never be 100% accurate, otherwise we could literally predict the future. Predictive ability becomes important around the high-performance table. It is the fundamental objective of the Prediction Centre where higher complexity gives higher accuracy but, for each of your personal journeys and things like Eventing Manager, we prefer our Simple Metrics to give clear accounts of recent performance.

Base Metrics and Context Metrics

Copy of Webpage banners (1)

For show jumping and cross-country, the variance in course difficulty from one event to another is very real, often caused by uncontrollable factors such as weather. For this reason, we have two metrics for these phases. A base metric, which summarises the recent performance of an individual and a context metric, which indicates how well the individual has performed relative to other competitors on the courses they have attempted.

Context metrics can be positive or negative. It is important know which is good and this depends on the phase. For cross country, we are focusing on clear rates so we want to see positive values. A higher clear rate is better. The base metric will be the horse’s clear rate (e.g. 70%) and the context metric, in this case called the XCJ adjustment, will be a positive or negative percentage value. If, for example we had an adjustment of +3%, this means that the base metric (clear rate) was 3% higher than the average clear rate across the competitions in that calculation. Therefore, the horse or rider has performed above average. If the number had been -9%, this indicates that the horse’s clear rate (70%) was 9% lower than the average for the competitions in that calculation, which means it performed below average.

In the show jumping phase, we operate in penalties rather than clear rates. Therefore, we want to see lower values because a lower number of penalties is better. Let’s say that the base metric, in this case the average jumping faults, is 3.2 and the adjustment (context metric) is -1.1. This means that the horse is averaging 3.2 jumping faults per competition, but this is 1.1 penalties lower than the average jumping faults for the courses in the calculation. Therefore, the horse performed above average because it incurred less penalties than the average for the courses it attempted.

Below, I describe and paint the picture for each metric, including the context metrics.  For a DIY guide for each of the base metrics (including examples!) head over to our how-to article.

Phase-By-Phase & Overall Metrics

Copy of Webpage banners (2)

Overall Performance
Name:
OBP6 (Opponents Beaten Percentage)
What it is:

The percentage of starters that you beat. Can be applied to a competition, season, level etc. 

How it is calculated:

The number of opponents that you finish ahead of (excludes yourself) in a competition, divided by the number of starters in that competition. If you win a competition with 10 starters the OBP is 90% (9 opponents beaten / 10 starters) but if you win a competition with 50 starters the OBP is 98% (49 opponents beaten / 50 starters). Therefore, OBP reflects the percentage and number of opponents beaten.

When calculated across a number of results, the OBP is the total number of opponents beaten divided by the total number of starters. So, if one result for a horse is 5th place in a competition of 25 starters and another result for the horse is 1st place in a competition of 80 starters, then the overall OBP is 99 (total opponents beaten) divided by 105 (total starters) which is 94.3%. Only competitions where a competitor started the cross-country phase are included in multi-competition OBP calculations for horses and athletes.

For Eventing Manager, we use the OBP6 which is the OBP for a horse in the last six runs where it started cross-country. A horse must have a minimum of three cross-country starts in order to obtain an OBP6.

Why it is useful:

The OBP method is diverse and could be used in a number of ways. For instance, you could look at horse’s career OBP at five-star level. For Ballaghmor Class, by the end of 2019 he had 343 opponents from a total of 356 starters. An OBP of 96.3% from 5 runs. You could also look at an athlete over an entire season. For example, Ingrid Klimke at all four and five-star competitions in 2019, beat 438 opponents from 475 starters giving an OBP of 92.2%.

Dressage
Name:

6RA (Six-run average)

What it is:

The average of your six most recent dressage scores

How it is calculated:

The average of up to a horse’s six most recent completed dressage scores, but a minimum of three dressage completions must be obtained in order to get a 6RA value. If a horse retires or is eliminated, there is no score to contribute to the average, so this result is not counted and the next most recent result available will be used.

Why it is useful:
  • Good summary of recent performance
  • Good way to compare horses and athletes
  • Good indicator of future performance
Show Jumping
Base Metric:

SJ6

Context Value:

SJ Adjustment

What they are:

The SJ6 is the average jumping faults of your six most recent show jumping completions. The SJ Adjustment is a measurement of how the SJ6 value compares to the average jumping faults incurred on the show jumping courses that the SJ6 is calculated from.

How they are calculated:

The SJ6 is the average jumping faults (time penalties not included) for a horse over its last six completed show jumping rounds. Retirements and eliminations are not included as it is an average penalty calculation, so these results will be skipped and the next most recent result included instead. A horse must have a minimum of three show jumping completions in order to obtain a SJ6.

The SJ Adjustment can apply to single results or to a number of results as is the case for the SJ6. For a single result, the SJ Adjustment is the jumping faults incurred by a horse minus the average jumping faults incurred by all starters on the course (calculated across multiple competitions that run over the same course where necessary). For the SJ6, the SJ Adjustment is the average of up to the last six SJ Adjustments for that horse.

Why they are useful:
  • Good summary of recent performance
  • Gain context around difficulty of courses
  • Good way to compare horses and athletes
  • Good indicator of future performance
Cross Country Jumping (XCJ)
Base Metric:

XCJ10

Context Value:

XCJ Adjustment

What they are:

The XCJ10 is the clear cross-country jumping rate from a horse’s last ten cross-country starts. The XCJ Adjustment is a measurement of how the XCJ10 compares to the average clear rates of the ten courses that the XCJ10 was calculated from.

How they are calculated:

The XCJ10 is the number of clear cross-country jumping rounds divided by the number of cross-country starts, from up to ten of a horse’s most recent cross-country attempts. Retirements, eliminations, run-outs, missed flag penalties and frangible device activation penalties are all non-clear attempts. A horse must have a minimum of three cross-country starts in order to obtain a XCJ10. For a horse with ten or more previous cross-country starts, the XCJ10 will either be 100%, 90%, 80%, etc.

The XCJ Adjustment can apply to a single result or to a number of results as in the case of the XCJ10. A single XCJ Adjustment for a competition is the cross-country jumping outcome (1 for clear, 0 for not clear) incurred by a horse minus the average cross-country clear jumping rate of all starters on the course (calculated across multiple competitions that run over the same course where necessary). The XCJ Adjustment for the XCJ10 value is the average of up to the last ten XCJ Adjustments for that horse.

Why they are useful:
  • Good summary of recent performance
  • Gain context around difficulty of courses
  • Good way to compare horses and athletes
  • Good indicator of future performance
Cross Country Time (XCT)
Base Metric:

True Speed Rating (TSR)

Context Value:

Top Speed Percentage (TSP)

What they are:

The True Speed Rating (TSR) is the average of your six lowest cross-country time penalty values from your last ten cross-country completions. Top Speed Percentage (TSP) is the average of your three highest speed OBPs, where a speed OBP is the percentage of cross-country starters that you had fewer time penalties than.

How they are calculated:

The TSR is the average of up to your six lowest cross-country time penalty values, from up to your ten most recent cross-country completions. Retirements and eliminations in the cross-country phase do not count in the ten most recent completions, but rounds with jumping faults do. This is mainly due to missed flag penalties and frangible device activation penalties being present when good times have been achieved. The requirement of the six best from ten results is to allow for intentional slow runs (either by design or because of jump faults) to be discarded.

A minimum of three cross-country completions are required in order for a horse to be assigned a TSR. In cases where a horse has six or fewer previous cross-country completions, the average time penalties will be calculated from all the available results. Once a horse has more than six previous cross-country completions, the TSR will be taken from the six lowest time penalty values.

The TSP is the average of the three highest speed OBP values from up to the ten most recent cross-country completions from a horse. A single speed OBP (opponents beaten percentage) is the number of horses that a horse had fewer time penalties than (including horses that started but failed to complete the cross-country course) divided by the number of horses that started the cross-country course (calculated across multiple competitions where necessary). A horse cannot be faster than itself, or any other horse with the same time penalty value.

If a cross-country course has 100 starters and 7 horses make the optimum time (all scoring zero penalties), then each of those 7 horses have a speed OBP value of 93%. They were faster than 93 other horses from 100 starters. A horse must have a minimum of three previous cross-country completions in order to be given a TSP.

Hopefully you are now getting an idea of how to calculate your own simple metrics. There is lots more for you to read here, including a how-to breakdown with examples

Don’t forget to share…