The Eventing Prediction Centre is a new tool from EquiRatings which allows fans to tracks the chances of any combination finishing as the winner, on the podium or in the top 10. We bring together all of our algorithms with the new Predictive Analysis tools available from SAP. You can read about the methods here. You can listen to a podcast about it here.
We publicly launched the Eventing Prediction Centre (PC) at Chatsworth ERM last weekend, (after a private trial with the media at Badminton a week before). It has been our main project in the 2019 off-season and we believe it has huge potential. Let’s take a look back over these first two competitions through the eyes of the Eventing Prediction Centre and see how we got on.
The question everyone starts with, did we call the winners? A full week before a single combination went down the centre line at Chatsworth, Laura Collett and London 52 were flagged as favourites, with a PC win chance of 17%, nearly double that of their next competitors Alexander Bragg and Zagreb who sat at 10%. They only went on to solidify that position throughout the weekend, improving to 23% win chance post dressage, and heading into the final phase, they were sitting pretty with a 40% chance of taking home the first ERM leg, nearly 3 times as likely as her nearest competitors.
The winner story played out as expected, but there was still plenty of drama on the podium. Of the final top three – Laura Collett, Tom McEwen and Lucy Jackson - only Laura was anywhere near our favourites to make that top 3 pre-competition. Tom McEwen and Figaro Van Broekxhof were our 10th favourites to make the podium at 13% and Lucy Jackson and Superstition were way down our list in 25th at 4%. However as any good model should do, PC updated their chances throughout the weekend, and they made steady climbs in each phase, moving to 8th and 13th after the first phase, and before the final phase Tom was a 28% chance to make the podium (only 3% behind our second favourite Sam Watson and Tullabeg Flamenco) and Lucy Jackson’s chances had risen all the way to 20%. Still far from “locked in”, but that is the reality of sport.
Prediction Centre takes a probabilistic view on outcomes, no one is ever guaranteed to perform to a certain level and, for example, make it into the podium, but some combinations are considered more likely to achieve this than others. One interesting place to look at this is our top 10 predictions. These likelihoods of placing in the top 10 are pretty well calibrated. When running the PC model over all 4* and 5* international events in the past 2 years, a combination that was given a 20% chance of finishing in the top 10, really did do close to 20% of the time.
If we order the table by top 10 chance pre-competition, we can expect to have about 4 or 5 of our 10 favourites to actually finish inside the top 10 come the end of the event. This is harder than you think. Take a look at who was in our top 10 pre-competition and see who you would have swapped out? At Chatsworth we had Laura, Tom and Julia all inside our 10 favourite combinations to achieve a top 10 result.
After updating the Prediction Centre with the actual results of the first two phases we become much more certain about their chances. We might now expect 6 of our favourites to actually finish inside the top 10, and at Chatsworth, 5 of them did, with Lucy Jackson and Mollie Summerland joining our pre-competition list. Numbers like these might feel a little low (and they are an area we are consistently trying to improve in our model, especially around increasing accuracy in cross country time) but they are also indicative of how much performance variance there is in the sport. Before the off, would most punters really have expected horses like Zagreb, Quicklook V and Shannondale Titan to not finish the competition inside the top 10? I’d wager not. But if we can remove some of our emotional bias when thinking about performance, we would have to admit that they are all possible outcomes (and are likely more probable than we might intuitively feel).
We feel that Chatsworth was a pretty successful start for the Eventing Prediction Centre. It identified the heavy favourite, (who went on to win), was able to track the increasing odds of Tom McEwen and Lucy Jackson as the event progressed, and performed close to our expectations when calling top 10 finishes. On the other hand, there were definitely areas to improve. While we were close in simulating the average time faults for a clear jumping round to be 9.9 (10.1 faults on the day), we didn’t have horses like Fallulah and Figaro up there as the combinations to get closest to the time.
While the Eventing Prediction Centre has been designed with ERM and Aachen in mind (the relatively higher number of CCI4*S competitions have provided much more data to train and test on), we will also be running it all of the remaining CCI5* competitions this year. With that in mind we have a look at how it performed in its first time out at this level.
As much of EquiRatings pre-competition commentary pointed out, the late withdrawal of SAP Hale Bob removed the clear favourite from the field, and really opened up the competition. The Eventing Prediction Centre went from showing one outstanding favourite to now showing three at the top with little to separate them, Cillnabradden Evo, Mr Bass and Vanir Kamira were all highlighted as the three most probable winners with a 9-10% chance of finishing on top. Inside EquiRatings HQ, Mr Bass was probably considered ahead of the others with questions over Cillnabradden Evo at the level. While he did go on to set the best ever dressage test in Badminton history, he had to prove his stamina over a top-level cross-country course. The Eventing Prediction Centre model will, at the moment, have difficulty identifying horses whose short format speed doesn’t transfer so well to the long format, and this is probably a case where intuition is needed to see if we have overstated the winning likelihood of a combination. However, it works the other way as well. How many felt Vanir Kamira was second joint favourite pre-competition. None of those boys and girls on the EquiRatings Eventing Podcast Badminton Preview even put Piggy into their top three.
PC was also a little low on Oliver Townend and Ballaghmor Class, placing them only joint 5th favourites before the competition, at a 5% win chance. It wasn’t until their near perfect cross country round that they went to the top of our list, entering Show Jumping at a commanding 53%. Here is another area that the PC model cautioned restraint. Even though they had a pole in hand coming into the final phase they were still only slightly better than even odds to take the title. We are not saying that PC model predicted Vanir Kamira was going pip Ballaghmor Class on the final day - Oliver Townend was still far and away the most likely winner - but the model did see that she still had a chance, which it placed at a not insignificant 25%.
Perhaps one of the best learnings we can take from PC is to practice caution, or perhaps to limit despair. When following eventing, we can get caught in a bias towards our favourite horses, or an overreaction to a particularly good phase. There were a couple of times throughout Badminton that PC helped some members of the EquiRatings team (mainly the excitable Diarm, let’s be honest) to temper their initial response to some big scores. Like when Tom McEwen did that fantastic 24.7 in the first phase, although he finished the phase sitting third, he wasn’t locked in to a top 10 finish. On the flip side, look at Imogen Murray and Ivar Gooden (who seem to have a bit of a cult following in this office). Even though they started off with a disappointing 39.6, the model knew their Badminton wasn’t over, they still had a 10% chance of making it into the top 10 – much higher than many with better scores. The same was also true for Gemma Tattersall and Arctic soul, who endured a tough day at the dressage, but still at that point had a 14% chance of climbing back into the top 10.
Finally if we look at our top 10 predictions from before the competition, we would have expected around 3 from our 10 favourites to make the mark, and on this occasion 5 of them did (Vanir Kamira, Billy the Red, Swallow Springs, Cillnabradden Evo, and Ringwood Skyboy).
All in all, we think that the Eventing Prediction Centre has gotten off to a good start. While the favourites won’t always go on to win, it is still quite gratifying that in our first two events the winner has come from our top 3. The top 10 predictions have performed along the lines we would expect, and the model has done a pretty good job of tracking changes as the competitions have evolved (and on occasion cautioned against too much excitement around some big phase performances). For me as a newbie to the sport has definitely helped me to follow along, understand who were the big winners (and losers) of the day, and finally, finally, answer the questions about what someone’s chances of winning are.
Share this story: