A Five-Star Look at the XCJ10 Simple Metric

Friday 10th April 2020
Author: Sam Watson

Note (28 Jan 2021): The XCJ10 Adjustment is now named the XCJ10 Difference to Par to more clearly capture what it tells us. Same exact metric, just a new name.


 

We like to call cross country jumping the ‘reliability’ attribute. If you don’t avoid jumping penalties, you don’t win. It’s that simple. At the five-star level, the clear rate over the last five seasons (2015-2019) is trending at 54%. This makes it by far the toughest level for keeping cross country marks off your score sheet. Here, I’ll take a look at clear rates across all international levels but then I’ll zero-in on the five-star level by examining the XCJ10 metric in detail.

Side note: Don’t necessarily confuse the cost of cross country jumping penalties with the ‘influence’ of the phase. Phase influence combines the cost of penalties plus the likelihood of penalties, so cost and influence are not necessarily equivalent.

Understanding XCJ Clear Rates

Before understanding the cross country jumping (XCJ) metrics, it is important to get a good grasp of what XCJ clear rates across the sport look like. The table below looks at the clear XCJ rates from 2015 to 2019, a five-year period, across all international class types.

Clear rates across international levels of eventing - EquiRatings Data for Horse Sports

The key takeaway is that generally, at international levels, 7 out of 10 (70%) competitors jump clear in the cross country phase. The new introductory level (1*) and 2*S format are edging closer to a clear rate of 75%, with the 2*L and 3*S class types hovering just above the 70% mark. There is a slight drop for 3*L and 4*S which trend around 68% and then we see the first significant drop to 4*L which is has a 60% clear rate. The second significant drop is just one level away at 5*L, where the clear rate drops by another 6% to 54%.

These figures should help to develop an understanding of what good, average and poor performance looks like for a horse competing at these levels.

The Metrics

To start measuring performance, we have two Simple Metrics for the ‘reliability’ attribute:

  • Base Metric: The XCJ10
  • Context Metric: The XCJ10 Adjustment

The XCJ10 - the Base Metric

The XCJ10 is a horse’s clear cross country jumping rate from its last ten cross country attempts. If the horse doesn’t yet have ten previous attempts, then it is the sum of the number of clear cross country jumping rounds divided by the sum of the number of cross country attempts, but a horse must have at least three previous cross country attempts in order to be given an XCJ10. (See the full how-to for calculating this metric here.)

What Does the XCJ10 Look Like?

The five-star level is helpful for demonstrating what the XCJ10 looks like. Over the past five seasons (2015 to 2019), 1,661 of the 1,681 competitors that started cross country at five-star had a full complement of ten counting runs in their XCJ10 value prior to their five-star attempt. The graph below shows the distribution of the XCJ10 for the 1,661 starters that had the full ten runs in their XCJ10:

Clear rates at the five-star level of eventing based on the EquiRatings Simple Metric of XCJ10 - EquiRatings Data Analysis for Horse Sports

There are lots of key points to take away from the above table:

  • The most common pre-competition XCJ10 values for a 5*L horse are 80% (27% of XC starters) and 90% (25% of XC starters). Together, these two XCJ10 values account for over 50% of XC starters at 5*L.
  • Only 3% of XC starters at 5*L enter the competition with an XCJ10 value of 40% or lower. This 3% of starters has a significantly low clear XCJ rate (below 17%).
  • The XCJ10 value at 5*L is most effective at identifying those with a low likelihood of going clear (for example, a clear rate of only 16.7%). There is no meaningful difference in the clear rates of horses entering a 5*L competition on XCJ10 values of 80%, 90% and 100% (clear rates of 58.3% - 60.5%). However, horses entering a 5*L with an XCJ10 value of 60% or lower are much less likely to jump clear, performing far under the baseline overall rate of 54.0%.

The XCJ10 Adjustment – The Context Metric

A single XCJ Adjustment is the outcome of a horse’s cross country jumping attempt (1 for clear, 0 for not clear) minus the overall clear jumping rate from all the horses that attempted the course. The course is usually specific to one competition for international events, but if multiple competitions run over the same course (e.g. two 4*S classes at the same event), then the single XCJ Adjustment is calculated across the two competitions combined (see the example at the end of this article).

Say the overall clear jumping rate on a course was 70% (that is, 0.70).  In this case, the single XCJ Adjustment for a horse that jumped the course clear would be +30% (that is, 1 – 0.70). The single XCJ Adjustment for a horse that started the course but didn’t jump clear (or didn’t complete) would be -70% (that is, 0 – 0.70).

The XCJ10 Adjustment is the average of all the single XCJ Adjustments from the results used to calculate the base XCJ10 metric. With the full complement of ten counting runs, there would be ten single XCJ Adjustments to average. The XCJ10 Adjustment will likely be a positive or negative percentage. A positive percentage indicates a horse has been performing better than the field and a negative percentage indicates a horse hasn’t been performing as well as the field. If the XCJ Adjustment is 0% (neither positive or negative), then a horse has been performing par for the course, so to speak.  The XCJ10 Adjustment is a very good indication of how a horse has been performing, regarding cross country jumping, relative to the courses he has been competing on.

What Does the XCJ10 Adjustment Look Like?

We can use the past five seasons (2015 to 2019) at 5*L to look at the XCJ10 Adjustment. The key things we are looking for are the range of XCJ10 Adjustment values (what does good or bad look like?), the distribution of these values, and how the horses with these values tend to perform in competition.  

 Clear rates at the five-star level of eventing based on the EquiRatings Simple Metric of XCJ10 Adjustment (Difference to Par) - EquiRatings Data Science for Horse Sports

There is a lot of value in this XCJ10 Adjustment table:
  • Horses that enter a 5*L with an XCJ10 Adjustment of 25% or higher have significantly higher clear jumping rates (circa 63%) than the overall rate of 54.0%. Nearly 1 in 5 starters (19%) at 5*L will come into the competition with an XCJ10 Adjustment of 25% or higher.
  • Horses that enter a 5*L with an XCJ10 Adjustment between 5% and 25% make up the majority of 5*L starters (approx. 52%) and they tend to perform just above the overall rate of 54.0% (circa 58%).
  • There is a big drop in performance for horses that enter a competition with an XCJ10 Adjustment below 5%. As the XCJ Adjustment drops from this point, the clear XCJ rate continues to drop significantly and steeply. It shows that the XCJ10 Adjustment is very effective at identifying horses with low expected performance with regards to reliability.  

Real-World Example - The Record Setter

I conclude with a record-setter. The highest value for the XCJ10 Adjustment, for a horse with the full complement of ten previous cross country attempts, came in 2018 for a horse that successfully jumped clear on the cross country at the World Equestrian Games – it was Italian rider Pietro Roman’s reliable championship mount, Barraduff, with a phenomenally high XCJ10 Adjustment of +49.1%.

Let’s look at how that record value of +49.1% was achieved.

The following shows you each counting run, the outcome for the horse, the course clear rate, and the Single XCJ Adjustment for each run.

Pietro Roman's and Barraduff's strong XCJ10 Adjustment (Difference to Par) value 2018 - EquiRatings Simple Metrics - Data Science for Horse Sports
 
The key numbers here:

  • XCJ10 value of 100% (base metric)
  • XCJ10 Adjustment (context metric) = Average of Single XCJ Adjustments = +49.1%

Note:  Jardy 4*S 2017 (6th run in XCJ10) ran two 4*S competitions over the same course. The course clear rate was 68.9% and is the value used for the XCJ Adjustment. The clear rate in the competition contested by Barraduff was actually 80.0%, but the other 4*S competition had a much lower clear rate of 59.0%. It is a good example of using the course clear rate rather than the competition clear rate.  


For more on the XCJ10 and other Simple Metrics, see our Simple Metrics Knowledge Hub.

Don’t forget to share…