A statistical approach to evaluating drivers.
16th July 2011, 20:03 at 8:03 pm #129757
For the past few weeks, and even more since Keith posted his article on the driver rankings, I have been thinking about what goes into saying “driver X is better than driver Y”. This post is somewhat of an open-ended reflection, so I hope it’s not too heavy :)
In general, when one says “driver X is better than driver Y” or produces a ranking such as Keith’s, there are some aspects that one always takes into account. Some are pretty straightforward: points and wins earned, qualifying and race form (particularly compared to the teammate) and, broadly speaking, a comparison of where you “expect them to be” and where they really are. Some of these aspects are quantitative, some of them are not, but all things considerered, they produce judgements such as ranking Di Resta above Sutil despite the latter having more points, and Petrov being down in 19th place, and all that groovy stuff we love to argue about.
Good. That is all perfectly alright. But in the end, it’s a subjective judgement, because there is no cold, hard number which allows one to say “over the past N races, driver X has shown more skill than driver Y”.
But what if it would be possible to produce such a number?
In the comments section of the rankings article someone (magon4, that is) showed a very interesting rating system based on a weighed average of the grades assigned to each driver after each race weekend. This is similar to what “kicker”, a German magazine, does with football, in that they assign a rating to each player after each match. This is a fine idea, but in the end, each individual rating is also subjective.
Of course, this “issue” is not unique to Formula 1. You could probably argue that all sports are like this, and that the search for “cold, hard numbers” is ultimately futile.
Enter baseball. One interesting aspect about it is that it is a sport which is very heavy on numbers. Statistics are an integral part of the game and of any discussion about it (which many people find boring, though). Anyways, what I want to highlight here is the emergence of a new approach to baseball statistics, which is called “sabermetrics”.
Say you want to determine which one of two players is more productive to a team’s offense. Broadly speaking, what sabermetrics will try to do is to take the data, remove the influence of any outside factors (an important difference compared to “mainstream” baseball statistics) and give you an adjusted index on which these two players can be more fairly compared.
What’s interesting about sabermetrics is that it allows you to compare players who are in wholly different situations (bad teams vs. good teams) and even different eras. Baseball teams now use them even for negotiating contracts and stuff like that.
Now… I consider Formula 1 to be a pretty numbers-heavy sport, too, which then makes me wonder if an approach such as sabermetrics’ could be developed (or if there’s already one in existence). It could be, well, interesting, particularly if one could even objectively compare drivers from different eras of the sport (which is a recurring discussion for all of us Formula 1 fans).
I hope you have not found this too boring! ;)16th July 2011, 20:41 at 8:41 pm #174552
No system is without its inherent biases, even the official FIA points table of course, but I prefer what f1-facts.com employs.
A driver is compared to his teammate each round,
+1 point for outqualifying teammate
-1 point for being outqualifyed
+2 points for finishing ahead of teammate
-2 points for finishing behind teammate
+1 point for better fast lap than teammate
-1 point for worse fast lap than teammate.16th July 2011, 21:24 at 9:24 pm #174553
you could have put an example so that we know what are we talking about. For instance, what are the external factors you propose to exclude and how to do that?16th July 2011, 21:29 at 9:29 pm #174554
I think that any statistics you take into account, it’ll still be impossible to compare Vettel with Alonso or Kovalainen with Liuzzi. Can’t be done because they have different cars and team-mates.17th July 2011, 0:34 at 12:34 am #174555
I’m curious though, to see Gaston’s response. He must have something in mind, right?17th July 2011, 5:08 at 5:08 am #174556
LL, I wish I had something clearer in mind, but I’ll give it a shot! ;)
Precisely, the great difficulty with Formula 1 compared to other sports is that players compete using unequal equipment. Even rules are changed often, which is something that doesn’t happen in other sports.
But precisely, the “holy grail” of any such system would be to remove the external factor of unequal equipment, thus making it possible to compare drivers who drive different cars.
I don’t know exactly how this could be done, but my intuition tells me that there should be a way of “normalizing” the data as to remove the “car variable”. One idea that comes to mind is to compare all the data to a common benchmark. For example, you could take all lap times of a given race and compare them to the circuit’s fastest ever lap, and then somehow analyze the deviations in order to determine how much of it is down to the car and how much down to the driver.17th July 2011, 6:20 at 6:20 am #174557
Sorry, don’t quite see how to do that, determine how much it’s down to the car and to the driver, at least in a mathematical precise way like you intend. I guess, if it were possible, it would have been done already.
But I don’t complain about the “tools” we have right now, or even to the pure subjective analysis. For example, I remember Schumacher impressing on his ever first GP on a Jordan, people realizing his outstanding talent and people predicting a great career for him after one year or two. And the same with Alonso, Pat Symmonds saying, when Fernando was 21, “on natural talent, he’s as good as Michael; if he acquires the same discipline that Michael had [has], he’ll achieve also comparable remarkable achievements”. And they were proven right. So, good eye, impartiality (that may be the holy grail) and a couple of seasons are usually enough.17th July 2011, 6:31 at 6:31 am #174558
That made me think about a common saying from nowadays (I myself have thought it and said it): that we have an outstanding, maybe unprecedent grid today in terms of talent. Yes, we have the provens Alonso, Vettel and Hamilton. And the unflowered Kubica. And Massa, Button, Rosberg, Webber and 42 year-old Schumi waiting in the wind.
But remember, at some point last decade we had Alonso, Schumacher on its prime and Raikonnen winning. And Montoya and Ralf.
In the 90′s, Schumacher, Hakinnen. Then Villenueve, Hill and Coulthard, Alesi and Berger also doing good things (ok, not so strong)
In the 80′s we had Prost, Senna, Piquet and Mansell, the most sucessful ones. So, maybe not unprecedent, but rather common to have 1 genious, 2 very good talents and a few more lurking. Just a thought17th July 2011, 16:03 at 4:03 pm #174559
I also think that subjective analysis is okay… I mean, after all, that’s one fun thing about sports: discussing with your friends or in a forum whether you agree or not with a given judgement. If it all were down to a number, perhaps it wouldn’t be that fun. But my geek side always makes me wonder about this kind of stuff, heh.
So doing a bit more of googling I stumbled upon the Elo rating system, which is a system often used to rank chess players. You can read more about it here: http://en.wikipedia.org/wiki/Elo_rating_system
The Elo system, as used in chess, tries to determine the relative skill of players based on the results of each match-up. The basic idea is the following: say player A (rating: 2500) plays against player B (rating: 2200). Since A has a better ranking than B, he is “expected” to win, so he would gain fewer points by beating B than player B would by beating A (same thing goes with losing points, if player B is beaten by player A, he loses fewer points than would player A if he is beaten by player B).
The original system was not really designed for multi-player games (such as Formula 1) but there are variations that allow for this (you could treat each 2-driver combination as an individual match, for example). Of course that this doesn’t solve the issue of “separating car from driver” (even though you could make a rating system for teams, too) but it could probably yield some interesting results.17th July 2011, 18:26 at 6:26 pm #174560
Here is a link to Peter Windsor talking about a method for comparison of drivers (taking the car factor out as well) based on statistical methods that was developed by Patrick O’Brien, the author of the 1994 book of analysis, Grand Prix: A Century of Racing, published by AA Racing (Kyalami)17th July 2011, 20:15 at 8:15 pm #174561
that’s got to be exactly what Gaston was looking for. It’s a damn good article, and I’m just saying that. Although it was lacking an exemplification of how the calculation is actually made, but I guess that’s reserved for the book. The results, though, didn’t surprise the common sense, as expected, or the subjective analysis (maybe Kubica, I was expecting higher from him, always thought he was outperforming the hell of that car).
Thanks for sharing, BasCB!17th July 2011, 20:44 at 8:44 pm #174562
I agree Jehto it is a brilliant article, it would be interesting to see some of the results for other seasons.
Thanks for sharing BasCB17th July 2011, 21:14 at 9:14 pm #174563
I’m looking for them, they mention they would keep posting results for this season, and it’s a blog. But I got lost listening to Nico Hulkenberg interview (he really seems a sensable guy, intelligent and honest; I think we’ll do very well when he finally returns) and reading Ross Brawn’s interview. Good and original content (not just quoting the F1 players declarations in their press releases/interviews) that they have, I’m getting quite fond of it. if I found other results, I’ll post here.17th July 2011, 21:24 at 9:24 pm #174564
this does the job:
an interesting comparation: Vettel vs Senna although it’s only their relative performances against their peers, not against each other:18th July 2011, 6:32 at 6:32 am #174565
Ha, got LL Jehto hooked on Peter Windsors blog now (try the episodes of the flying lap sometimes, Keith was there as well as a guest expert). I think he compared Fangio and Moss as well as doing some other interesting persons in following blog posts, really interesting.
You must be logged in to reply to this topic.