2014 model-based driver rankings

2014 was a year completely dominated by the Mercedes team, with the barest sniff of the title for Ricciardo. But who would have won the title if all cars had been equal? Were the Mercedes drivers the year’s best performers, or would other drivers have looked even better behind the wheel of a Mercedes? That’s a question you could debate for hours. Alternatively, we can pose the question to a mathematical model, such as the one I previously used to estimate all-time driver rankings.

In a nutshell, my model ignores races for each driver where they had non-driver failures (e.g., mechanical DNFs), then uses points per race in each of the remaining races as a performance metric for each season. While this is not a perfect metric (e.g., it doesn’t assign blame for crashes), it does capture one of the most important aspects of driver performance, and it can be easily applied across the board to all historical races without any subjective interpretation. Note that the model uses a scoring system based on the 10-6-4-3-2-1 system and includes fractional points for all lower race positions, allowing it to differentiate between drivers who finish 13th and 19th, for example.

Performance is assumed to be a function of both the driver’s performance and their team’s performance. Using the race results from 1950-2014, the model estimates the performance of each driver and each team so as to statistically best fit the race result data. This is possible because drivers are connected to one another by many different teammates, allowing their relative performances to be estimated.

2014 driver rankings

Using my model, I ranked the driver performances of 2014. Rankings are quantified by the adjusted points per race (ppr) up to a maximum of 10 ppr, as described previously. Each driver is discussed below. Note that the 2014 Marussia team and their drivers Jules Bianchi and Max Chilton were unranked, because to date Chilton and Bianchi have only driven alongside each other, so they cannot be compared to others. All other drivers are connected via common teammates. For the sake of completeness, I note that if we make the crude assumption that Marussia and Caterham were equal in performance in 2014, then the model ranks Chilton 17th and Bianchi 9th.

20. Esteban Gutierrez, 3.97 ppr

This was another disappointing season for Gutierrez. Last year he was outscored 54-6 by Hulkenberg. In 2014, he was closer to level with his teammate, but Adrian Sutil is himself lucky to still be on the grid after being convincingly beaten by Paul di Resta in 2013. With Caterham and Marussia both in peril, there may be nowhere for the Sauber drivers to go next year.

19. Adrian Sutil, 4.46 ppr

Sutil shows very occasional flashes of brilliance — usually at Monaco — but at this stage in his career it’s no longer a matter of waiting to realize potential promise. His lack of outright speed is not helped by the fact that he crashes out of races more often than any other current driver, with 1 crash every 5 starts. Sauber had a car that was capable of scoring points in 2014. They were potentially on offer in Australia, Monaco, Hungary, Singapore, and USA. Through a combination of extremely poor driver performances and unfortunately timed mechanical failures, they finished the year ahead of only Caterham.

18. Kevin Magnussen, 5.26 ppr

Rookies today are often held to the impossibly high standard of Lewis Hamilton in 2007. Formula 1 is a different world now, with preseason testing cut down to just 12 days — that’s 6 days per driver to fully acquaint themselves with a new Formula 1 car, while developing the car and typically losing time to technical problems. Given that, Magnussen’s 2nd place on debut was a stunning achievement. Unfortunately, it proved to be an aberration. That one race was almost a third of Magnussen’s total season points haul. Across the season he was outraced and outscored heavily by Button: 126-55. Magnussen will undoubtedly be stronger if he returns next year, but he now faces significant competition for his seat, both from Button and from the hugely impressive Stoffel Vandoorne.

17. Sergio Perez, 5.29 ppr

In 2014, Force India developed one of their most competitive cars, even leading McLaren for the first half of the season. Perez capitalized on that with a podium ahead of Hulkenberg in Bahrain, but he also threw away easy points in Monaco, Canada, Hungary, and the USA. Ultimately, he was outscored 96-59 by his teammate in a year where he could and should have been much closer.

16. Pastor Maldonado, 5.38 ppr

As always, Maldonado showed impressive bursts of speed, sometimes surprising Grosjean, but his frequent carelessness ruined multiple race weekends. As a positive, Maldonado didn’t crash out of any races in 2014, although he still caused the most egregious collision of the year in Bahrain when he ignored every possible guideline for sensible overtaking.

15. Marcus Ericsson, 5.66 ppr

With only Kobayashi to compare him to, and only 7 races where both Caterhams finished, Ericsson’s rating is still highly uncertain. Kobayashi beat Ericsson 5-2 in their races together, but Ericsson delivered an impressive 11th place at Monaco and seemed to relatively gain on Kobayashi towards the end of the season. Neither Kobayashi nor Ericsson had strong junior careers by Formula 1 standards, so little is expected of them in the long run, but we will have another chance to assess Ericsson next year.

14. Kamui Kobayashi, 5.90 ppr

Ditching his Ferrari affiliation for a year at Caterham now seems like a poor decision for Kobayashi, although you have to respect his desire to return to Formula 1 above all else. He spent much of the year as Caterham’s lead man against Marussia’s Bianchi. With a shortage of seats on the grid next year and a shortage of money on Kobayashi’s part, this is likely the last we’ll see of him.

13. Felipe Massa, 5.93 ppr

On paper, Massa fell short of his targets in 2014, scoring just 134 points to Bottas’s 186 points. In reality, it was much closer than that, as discussed under Bottas’s entry below. Massa was affected by numerous incidents outside of his control, which set him back in the points standings. Taking those into account, he deserves a higher ranking. Massa started the year stronger, Bottas launched into another gear in Spain and led the team through most of mid-season, then Massa came back very convincingly in the last few races. A second year of Massa vs. Bottas will give us a better idea of where these two relatively stand.

12. Sebastian Vettel, 6.16 ppr

Vettel had to come crashing back to Earth some time, just as Fangio did in 1958 following his own four consecutive championships. In Fangio’s case, a confluence of events hurt his competitiveness. Alcohol-based fuels were banned, which hurt Maserati. The rise of better handling mid-engined cars was suddenly making obsolete the front-engined cars in which Fangio had dominated. To make matters worse, Fangio was kidnapped in Cuba and his decision to attempt Indianapolis instead of Monaco completely backfired. In the end, Fangio made only two race starts, finishing 4th each time, which was tremendously disappointing for a driver who won 24 of his 51 race starts and only 6 times finished off the podium.

Vettel faced his own series of challenges in 2014, including a new breed of car that he struggled to come to terms with, a devastatingly quick teammate, a lack of early season mileage, and problems drawing performance from the Pirelli tyres without destroying them. Beating his teammate in only 3 races was a bitter disappointment. While Fangio retired in 1958 at the age of 47, Vettel is still just 27 and will move to Ferrari to answer his critics. But there he risks another hiding to nothing. If he beats Raikkonen, people will likely point to 2014 and say Raikkonen’s past his best. If Raikkonen beats Vettel, people will likely infer that Alonso >> Raikkonen > Vettel.

11. Daniil Kvyat, 6.18 ppr

Kvyat’s 2014 performance is considered just as strong as Vettel’s by my mathematical model, which is rather remarkable. This makes Kvyat the model’s rookie of the year. Kvyat failed to deliver the big points opportunities his teammate did and he had some weekends where he was mystifyingly slow. However, there’s no doubting the potential of a driver who beat Vergne 12-7 in qualifying and equaled him 6-6 in races. If Kvyat can achieve the kind of second-year progression many seem to be expecting of Magnussen, then Ricciardo had better watch out. It might seem cruel to Vergne, but Kvyat’s promise is definitely worth pursuing, even if it ultimately amounts to nothing.

10. Nico Hulkenberg, 6.23 ppr

Hulkenberg remains a difficult driver to assess. His junior results were sublime — he received the highest score of any driver using a junior career metric I previously devised. He took a similar path through junior series as Hamilton, but reached Formula 3 Euro one year earlier. He dominated GP2 as a rookie in 2009 ahead of 9 other future F1 drivers: Grosjean, Perez, Maldonado, Kobayashi, van der Garde, Petrov, d’Ambrosio, Chandhok, and di Grassi.

Given this pedigree, Hulkenberg’s first year performance alongside Barrichello at Williams was somewhat underwhelming. Replaced by the well-financed Maldonado for 2011 (who performed just as well as Hulkenberg relative to Barrichello), Hulkenberg spent a year as reserve driver for Force India before returning to a race seat in 2012. A sabbatical year spent testing worked wonders for a young Fernando Alonso at Renault in 2002, although test drivers in those days could do plenty of… testing. With most mileage now restricted to race weekends, gone are the junior driver testing apprenticeships and the specialist test drivers like Luca Badoer and Pedro de la Rosa.

On his return in 2012, Hulkenberg was beaten by di Resta 6-1 in the first 7 races before he began to turn his season around, dominating di Resta in the later stages. Overall, Hulkenberg narrowly beat di Resta 10-9 in qualifying, 10-9 in races, and 63-46 in points.

Hulkenberg jumped ship to Sauber for 2013, where he thrashed Gutierrez 18-1 in qualifying, 13-2 in races, and 51-6 in points. The comparison to Gutierrez was largely unhelpful in terms of assessing Hulkenberg, given Gutierrez was a rookie with a fairly unimpressive junior record. However, Hulkenberg also showcased his excellent racecraft in 2013 with several jaw-dropping defensive drives.

Perez served as a much better reference point for measuring Hulkenberg’s capabilities: here is a driver who had a slight edge on Kobayashi across 2011-2012 and who got outclassed by Button in 2013. Overall, Hulkenberg beat Perez 12-7 in qualifying, 9-7 in races, and 96-59 in points. Next year will provide additional data on this match-up. The data so far suggest a slight advantage to Hulkenberg, which would perhaps place Hulkenberg just slightly below the level of Button.

With the constant influx of junior talents, Hulkenberg’s chances of getting to a top team are seemingly diminishing. Much like Nick Heidfeld before him, Hulkenberg has proven he is very good, but his case has never been quite compelling enough to select him ahead of proven champions or team-affiliated junior drivers. The next thing Hulkenberg needs to demonstrate is that he can not only perform consistently, but also occasionally achieve something spectacular. After 76 starts, it is surprising that he is yet to achieve a podium. Currently, the model consider him one of the two best drivers in history to never score a podium. The other is Jean-Eric Vergne.

9. Romain Grosjean, 6.30 ppr

Grosjean endured a frustrating year, with many reliability problems and only a few positive results. He finished last season on a high, competing with Raikkonen on level terms and taking regular podiums as Lotus became best-of-the-rest behind Red Bull. However, it was always clear that Lotus were set for a dramatic fall in 2014, given their budget problems, dodgy investors, failure to pay staff, loss of key personnel like Boullier and Allison, and their partnership with the faltering Renault.

Despite the difficult situation, it was a strong performance from Grosjean in 2014. Two eighth places put him ahead of Kvyat in the drivers’ championship, despite a clearly worse car. Overall, Grosjean beat Maldonado 11-4 in qualifying, 5-5 in races, and 8-2 in points. Notably, Maldonado was out of contention both times that Grosjean scored. In Spain, after setting top 10 times in all practice sessions, Maldonado had an astonishingly careless crash in Q1. Then in Monaco, Maldonado had a fuel pump failure just before the start of the race, which either cost him a very good shot at points or saved him a later trip into the barriers, depending on your outlook.

Some might have expected Grosjean to beat his crash-prone teammate by an even greater margin, but Maldonado is considerably quicker than most give him credit for. His junior results demonstrate he is at least capable of mixing it with drivers like Perez. His F1 results emphasize that he is a capable midfield performer. In his rookie year, he was beaten by Rubens Barrichello 10-8 in qualifying, 9-5 in races, and 4-1 in points. For comparison, Barrichello beat the highly-touted rookie Hulkenberg 13-6 in qualifying, 11-7 in races, and 47-22 in points the following year. Maldonado subsequently beat Bruno Senna 15-5 in qualifying, 8-8 in races, and 45-31 in points, then was closely matched with talented rookie Bottas 7-12 in qualifying, 10-7 in races, and 1-4 in points.

Both Lotus drivers will be hoping for a progression into at least midfield next year. While fiscal problems persist and podiums are almost certainly out of the question, the team can at least look forward to Mercedes power in 2015.

8. Kimi Raikkonen, 6.92 ppr

Last year, Raikkonen was widely considered a member of the sport’s elite four, alongside Alonso, Hamilton, and Vettel. It was an appealing narrative that helped fans mentally sort the grid into nice clean boxes. A humbling year for both Raikkonen and Vettel, as well as a close teammate battle at Mercedes, has undermined that simple hierarchy and shown that things are closer and more confusing than ever at the top.

Many fans and pundits were excited to finally see Alonso and Raikkonen in the same car. This was a prospect that had mouths watering as far back as 2003, when Alonso and Raikkonen graced the cover of the September 2003 issue of F1 Racing magazine.

Sadly, it proved to be a very one-sided fight.

Before the season began, my mathematical model predicted a 94% chance of Alonso coming out ahead across the season. The model predicted Raikkonen would score around two-thirds of Alonso’s points. In reality, the end-of-year points tally was 161-55 in Alonso’s favor, suggesting Raikkonen underperformed and/or Alonso overperformed.

Some would say this result was no surprise, given Massa had a slight edge on Raikkonen across 2007-2009 (beating him 25-19 in qualifying, 18-16 in races, and 213-195 in points) and Alonso made mincemeat of Massa. Others would say that Massa was performing at a higher level before 2010. Both arguments have some merit. Interestingly, the model thinks most drivers would have looked just as bad alongside Alonso in 2014.

So which factors have hampered Raikkonen? He had bad luck at Malaysia (hit by Magnussen), Spain (team favoritism), Monaco (collision with Chilton), and Germany (hit by Hamilton), but these races explain very little of the performance gulf to Alonso. Raikkonen also faced the challenge of moving to “Alonso’s team”. However, Ricciardo clearly had no difficulties moving to “Vettel’s team” and Senna had no difficulties moving to “Prost’s team” in 1988, so this also seems an inadequate explanation, especially given the crumbling relationship between Alonso and Ferrari.

An obvious culprit is the F14T. While the car’s terrible handling characteristics are clearly to nobody’s tastes, they play to Alonso’s ability to radically adapt his driving style as needed. Alonso is not infallible in this regard, as his troubles adapting to a new braking system and tyres in 2007 showed, but he seems to have a wider operating window than his teammate, who has previously shown great reliance on a responsive front-end. Raikkonen struggled badly with a new front suspension system at Ferrari in 2008 and had McLaren build him a different front suspension system from Montoya’s in 2005.

According to Raikkonen, he has also struggled to adapt to the more conservative tyres produced by Pirelli after the spate of punctures last year. Pirelli used a new kevlar construction for Germany 2013 then changed the compounds (favoring harder tyres) from Hungary 2013, continuing that trend into 2014 to avoid any potential tyre problems with the new engines. Looking at Raikkonen’s race performances relative to his teammates across that time period lends credence to his claim.

Prior to Hungary 2013, Raikkonen beat Grosjean 21-3 in races (excluding non-driver failures). After the change, Grosjean beat Raikkonen 4-2 in races and Alonso beat Raikkonen 16-1.

Sadly, we won’t see a Raikkonen vs. Alonso rematch, but getting to see Raikkonen vs. Vettel in 2015 is more than adequate compensation. To date, Vettel has faced only one known benchmark across a season (Mark Webber), making it difficult to place him and Ricciardo relative to other current drivers. This new teammate comparison will help shed light on the true hierarchy. Based on overall career performances, the model predicts a small edge to Vettel. Based on 2014 performances, the model predicts a small edge to Raikkonen. It should be a fascinating battle to watch.

7. Nico Rosberg, 6.95 ppr

Prior to 2013, the Rosberg précis was essentially: A big brain behind a reliable pair of hands. He hadn’t distinguished himself with any especially bold moves or outstanding characteristics, nor had he disgraced himself any. His junior results were outstanding and he had some stand-out performances across 2006-2009, but nobody could be sure where he ranked compared to the sport’s best. He gained little more than a pat on the back for crushing the legendary Michael Schumacher, with most putting this down to age-related decline. What went largely unnoticed (but now can hardly be missed) was Rosberg’s blinding speed over a single lap.

Seeing Rosberg up against Hamilton for a couple of years has helped to identify their relative strengths and weaknesses. Hamilton is quite rightly considered one of the sport’s best ever qualifiers, having previously outqualified Alonso 9-8, Kovalainen 26-9, and Button 44-14. Against expectations, Hamilton did not assert dominance over Rosberg in qualifying in either 2013 or 2014. Excluding Germany 2014 and Hungary 2014, the overall head-to-head tally stands at 18-18.

Looking back, Rosberg’s tremendous qualifying pace should have been obvious. As a rookie he was beaten 12-6 by the exceptionally quick Webber, who outqualified every teammate he ever had except Vettel, including a 31-4 drubbing of Coulthard. Rosberg then beat Wurz 15-1, Nakajima 27-9, and Schumacher (arguably the best qualifier of all time in his heyday) 41-17.

My subjective perception in 2014 was that Rosberg excelled in picking patient, exact lines through slow corner complexes, while Hamilton was unbeatable under heavy braking. To test this observation, I recorded each driver’s best sector times for each track (set in any session, and excluding qualifying times for both drivers at Germany 2014 and Hungary 2014), and I recorded the number of heavy braking zones (reductions in speed of at least 150km/h) and the number of slow corners (minimum apex speed less than 100km/h) in each sector.

As you can see below, Hamilton gained net time on average the more heavy braking zones there were in a sector, while Rosberg gained net time on average the more slow corners there were in a sector.

Blue points are individual sectors. Black dots with error bars are mean +/- standard error of the mean. Red lines are linear regressions.

The coefficient of determination is obviously very low for these linear fits, as this is a very complex system with lots of noise that these two variables are not accounting for. The trends are nevertheless suggestive.

Fitting a multivariate linear model shows that Hamilton gained on average 0.030 seconds per heavy braking zone, while Rosberg gained on average 0.011 seconds per slow corner. For all other corners (minimum apex speed of at least 100km/h), Rosberg gained on average 0.003 seconds per corner.

On ultimate pace, there was clearly nothing between the Mercedes drivers. In the 57 track sectors, Hamilton’s best time was quicker in 29 and Rosberg’s was quicker in 28, with a mean advantage of 0.011 seconds to Hamilton. As the cars changed dynamically across a race distance, however, Rosberg looked more fragile and his advantages more transient. Rosberg also made a string of errors in 2014 that looked like poor pressure management, perhaps because this was his first ever championship battle while Hamilton was a seasoned championship veteran. It’s worth remembering that before 2014, Hamilton had 22 wins while Rosberg had only 3 wins.

Despite their contrasting qualities, Hamilton and Rosberg have proven to be two of the closest matched teammates in history. Almost every race between them has been a game of inches. Across 2013-2014, Hamilton was ahead 17-14 in races where neither driver had a mechanical DNF. That tally can be adjusted for cases where one driver experienced misfortune that changed their relative finishing positions — Malaysia 2013 (brake-balance adjuster failure in qualifying and team orders against Rosberg in the race), Britain 2013 (puncture for Hamilton), Korea 2013 (front wing failure for Rosberg), and Belgium 2014 (Hamilton hit by Rosberg) are all clear-cut cases, while Canada 2013 (KERS and radio failure for Rosberg in qualifying), Germany 2013 (Rosberg prematurely eliminated due to team holding him in Q2), Abu Dhabi 2013 (suspension failure for Hamilton in Q3 on his final hot lap), Monaco 2014 (Hamilton’s last lap hampered by yellow flag), and Germany 2014 (brake failure for Hamilton in Q1) are all ambiguous.

It will be fascinating to see how the Mercedes drivers perform next year, especially if they find themselves battling at least one other team for the championship. Will Rosberg go the way of Webber at Red Bull after narrowly missing out in 2010, or will he learn from 2014 and come back even stronger next year?

6. Jenson Button, 6.99 ppr

Jenson Button was a baby-faced 20 year old when he debuted in 2000. He had spent just two years in single-seaters, dominating British Formula Ford in his first year, then claiming 3rd in British Formula 3 and 2nd at Macau in his second year. He was clearly an exceptional talent and was soon being compared to Senna. The Williams team needed a driver to fill a seat while they waited for Montoya to complete his IndyCar commitments, so they signed Button.

No driver had arrived in Formula 1 with so little experience before. The closest was probably Mike Thackwell, who debuted in 1980 aged 19 in the midst of his third year of car racing. Button’s super license application went to committee, requiring 17 votes in favor out of 26 to pass. He received 18.

Many had concerns about Button’s ability to step up from Formula 3 cars with less then 200 horsepower to an 800-horsepower Formula 1 car, but he performed respectably in his first season. This paved the way for Alonso and Raikkonen to enter Formula 1 in 2001 with only two years of experience apiece, and for Max Verstappen to enter Formula 1 in 2015 with only one year of experience. This 2001 article excerpt captures the climate at the time.

Last year, Button — whose level of experience was only slightly higher than Raikkonen’s — provoked similar alarm. Dual world champion Mika Hakkinen predicted that the newcomer was in for a “nightmare four or five years” while he adapted to F1; ex-driver turned commentator Martin Brundle said Button had joined F1 at least two years too early. By the end of the season, however, Button was regularly qualifying ahead of established stars, and is now tipped to one day become a title winner. Many observers of Raikkonen’s brief career deliver equally glowing predictions.

Today, Button is the second oldest driver on the grid. McLaren are considering pulling the plug to make way for new talent, presumably in the form of Kevin Magnussen, or Stoffel Vandoorne if they want to put their support behind the star of their junior team. Certainly those drivers could be signed more cheaply.

Yet there are no signs of decline for Button, nor should we really expect any at age 34. Alonso appears to be at the height of his powers at 33, as was Senna in 1993. Schumacher drove one of his greatest seasons at 35. Prost easily dispatched teammates Mansell at 35, Alesi at 36, and Damon Hill at 38. Given Button’s incredible fitness and his recent results against Hamilton, Perez, and Magnussen, he shows every sign of remaining an elite driver for several years to come.

Button’s stand-out performance in 2014 was surely the Japanese Grand Prix, where he beat both Williams cars by almost a minute and was unlucky to miss the podium. As the statistics show, Button is peerless in the wet. Due to having spent his career in cars of mixed competitiveness, Button’s scoring rate in the dry (6.4 points per dry race, retroactively applying the 25-18-15-12-10-8-6-4-2-1 scoring system) is about half that of Vettel (13.5), Hamilton (12.1), and Alonso (11.2). In the wet, Button scores 50% more points per race (9.5), approaching Vettel (10.7) and Alonso (10.8), who both perform relatively worse in the wet than the dry, and not far behind Hamilton (13.6), who is another phenomenal wet-weather driver. While Button and Hamilton were teammates, they started 12 wet races together. Button won 5, Hamilton won 3.

Magnussen’s results versus Button in 2014 bear many resemblances to Perez’s in 2013. In qualifying, both battles were close: Button was ahead of Magnussen 10-9 (Perez was ahead 10-9). Come the races, Button was faster and more consistent. He beat Magnussen 14-3 in races (he beat Perez 11-5) and he beat Magnussen 126-55 in points (he beat Perez 73-49).

Magnussen undoubtedly had some bad luck, which affected his results. We can account for this by awarding back lost results for both drivers:

  • Bahrain: No prospect of points for Magnussen, but Button was running 5th when he hit trouble.
  • Monaco: Magnussen would have been 6th (demoting Button to 7th) without engine trouble.
  • Germany: Magnussen had the pace to beat Button by about 10 seconds in the race if not for colliding with Massa. Optimistically, that’s 7th place (demoting Button to 9th).
  • Hungary: Magnussen started from the pits, but ended up right behind Button after the safety car reshuffle, so nothing was ultimately lost there.
  • Italy: Unlike Malaysia and Belgium, this incident fell within a rules of racing gray area. Without the penalty, Magnussen would have finished 7th (demoting Button to 9th).
  • Singapore: Button had the tyres to likely resist Vergne and Perez, so would have finished 6th once Bottas hit trouble (demoting Magnussen to 11th).
  • Japan: Both drivers required steering wheel changes. Without this, Button would have been running 3rd, but the timing of the red-flag just after his last pit-stop would have left him 5th. Magnussen would have been in good position to beat the Force Indias for 8th.
  • Russia: Magnussen had a 5-place grid drop but could not have finished higher than 5th.
  • USA: Button had a 5-place grid drop. Without this, he would have likely finished 9th.

Restoring these results puts Button ahead 13-6 in races and 140-74 in points. As a rookie, Magnussen certainly has potential for improvement, but improving to the level of Button remains a huge task.

5. Jean-Eric Vergne, 7.00 ppr

2014 was a bittersweet year for Vergne. On the one hand, he scored over 70% of Toro Rosso’s points and finally caught some mainstream attention for his excellent racecraft and wet-weather skills. On average, Vergne has scored 3.6 times as many points in the wet (13 points in 5 wet starts) than in the dry (38 points in 53 dry starts), which is the biggest wet/dry discrepancy of all current drivers (Button is next with 1.5 times as many points per wet race). On the other hand, he was overlooked again for a seat at Red Bull and is now losing his Toro Rosso seat.

In the Red Bull junior program, potential greatness is worth more than proven competency. Helmut Marko is therefore willing to gamble until he finds the next Red Bull superstar, even if it means putting talented drivers’ careers on the line (both Vergne’s and Verstappen’s in this case). This is why Frijns turned down the Red Bull program, fearing he might become another statistic. But you miss 100% of the shots you don’t take (which is easy to say when you’re Wayne Gretzky) and Vergne’s fate is certainly still preferable to sitting on the sidelines at Caterham.

Versus Kvyat, Vergne continued to show that qualifying is not his strongest suit, losing the qualifying battle 12-7 to his inexperienced teammate, with the balance of power shifting progressively in Kvyat’s direction as the season went on. However, Vergne tended to have more consistent pace in race trim and showed some extremely feisty racing. He was also one of the only drivers to hold a Mercedes at bay, keeping Rosberg stuck behind him for 19 laps at Hungary, before his tyres finally went over the degradation cliff and Hamilton brilliantly scorched past. Kvyat and Vergne were tied 6-6 in races, but Vergne showed the ability to step up to a higher level when it counted, three times making the top 8. Between that and Kvyat’s unfortunate failure in Abu Dhabi, Vergne was ahead 22-8 in points.

Ricciardo’s astounding season at Red Bull raised two obvious questions: (1) Is Vettel as good as we think, and (2) is Vergne not getting anything like the credit he deserves? After all, Vergne and Vettel both trailed Ricciardo by a similar margin as teammates, as shown below.

In Vergne’s case, he was beaten 29-10 in qualifying, 17-12 in races, and 30-29 in points across 2012-2013, despite Ricciardo having a slight head-start in Formula 1. Ricciardo and Vergne were certainly two of the strongest drivers to ever come through the Red Bull Junior Team. Right now, one looks like a potential Formula 1 champion while the other will be very fortunate to have a seat next year. Such is the way of Formula 1.

4. Valtteri Bottas, 7.13 ppr

Before his 2013 debut, Frank Williams described Bottas as “quite simply one of the most talented young racing drivers I have come across.” On the strength of his junior results, Bottas seemed destined for great things in Formula 1. His first season for Williams was difficult, as most rookie seasons tend to be in this era. Bottas had some stand-out performances, such as qualifying 3rd in Canada, but more often than not he finished races behind Maldonado.

In 2014, Williams achieved an extraordinary recovery, just in time for Bottas’s coming of age. Bottas wasn’t quite able to achieve a race win against the all-conquering Mercedes team, but his six podiums tell us a win can’t be far away, just as Raikkonen’s four podiums in his second season marked his intent. Whether Williams will maintain the same level of competitiveness in 2015 is questionable, since the massive spending power of Ferrari, Red Bull, and McLaren should help them overcome any smaller budget ingenuity given a relatively static rule-set and time to learn from their mistakes. The Mercedes engine advantage, which played perfectly into Williams’ hands, will also likely diminish over time. Nevertheless, a clear 3rd position in the constructors’ championship was extremely impressive.

Since the model makes estimates of team performance, we can directly ask how the points standings would have looked if all teams were running identically skilled drivers — imagine an individual driver cloned 20 times. This allows us to assess which teams overperformed and which teams underperformed.

These estimates suggest that Williams performed right to expectations, while Ferrari massively overperformed and Sauber massively underperformed.

On paper, Bottas was Williams’s leading driver, scoring 58% of the team’s points. In qualifying, it was 11-6 to Bottas (excluding Monaco and Russia where Massa was prevented from setting a representative time). In races where neither driver had a mechanical DNF, it was 10-8 to Bottas.

However, Massa was a magnet for so many problems in 2014 that this match-up requires careful dissection. Here are the issues outside of each driver’s control.

  • Australia: Massa was taken out at the first corner while ahead of Bottas. He had the pace to be 5th, demoting Bottas to 6th.
  • China: Massa had a 1-minute pit-stop. Otherwise, he was set for 6th, demoting Bottas to 8th.
  • Monaco: If not for his engine failure, Bottas would have been 7th, demoting Massa to 8th.
  • Canada: Massa was taken out by Perez. Otherwise he would likely have been 4th, demoting Bottas to 8th.
  • Britain: Massa had an unavoidable collision with Raikkonen’s car. He made a terrible start and would likely have lost ~20 seconds moving back through the field. That would have been 5th.
  • Hungary: The safety car was unfortunate for Bottas, but Massa had slightly better pace anyway.
  • Belgium: Massa lost an estimated 40 seconds due to tyre debris. He should have been 5th.
  • Russia: Massa adopted a disastrous strategy to make up for issues in qualifying. Had he started further up, he would have at least jumped Perez and Raikkonen for 9th.
  • Brazil: Bottas had multiple issues. He should have finished at least 4th.

If we award back these lost results, the new tally is 11-8 to Massa in races and 197-184 to Bottas in points. The Williams drivers were actually very closely matched in 2014. This suggests that Bottas is still some way behind Alonso in absolute terms, given Alonso dominated Massa every year from 2010-2013. But it also means that Bottas’s performance is rapidly on the rise.

3. Daniel Ricciardo, 7.97 ppr

Ricciardo was undoubtedly the breakout star of 2014. In the pre-season build-up, not even Ricciardo fans were seriously entertaining the prospect of their man beating quadruple-champion Vettel. After all, Vettel was fresh off dominating Webber 17-2 in qualifying, 15-0 in races, and 397-199 in points in 2013. Most expected the younger Australian to put up a tougher fight, but the Sky F1 team still predicted a qualifying tally of between 18-1 and 12-7 in Vettel’s favor. What we saw instead was a 12-7 tally in Ricciardo’s favor.

At Toro Rosso, Ricciardo looked phenomenal in qualifying, but there were questions raised by his tendency to lose positions in races, especially given Vergne outscored him 16-10 in their first year together. Ricciardo obliterated any concerns by beating Vettel 11-3 in races and 238-167 in points. The correct interpretation now seems to be that both Ricciardo and Vergne are exceptional race drivers, we just had nobody to compare them against. This interpretation is bolstered by Ricciardo’s strong record as a rookie against Liuzzi in 2011: 5-5 in qualifying and 4-1 in races.

Ricciardo’s race record in 2014 was particularly impressive given his Webber-esque starts. Statistically, he was the worst starter in 2014, losing an average of 1.3 positions on the first lap. He more than compensated for this with some of the year’s keenest racecraft and a superb ability to extract life from the tyres while living with an unstable rear end. Across all uninterrupted stints  in 2014 (i.e., no stops triggered by problems or safety cars), Ricciardo averaged 25.1 laps on primes and 17.2 laps on options, while Vettel averaged 22.8 laps on primes and 15.9 laps on options. Ricciardo’s ability to gradually eke performance out of the tyres often made the difference of an extra pit-stop across a race distance.

So what are we left to conclude? Is Ricciardo better than Vettel and is Vergne therefore about as good as Vettel? All three were similarly brilliant juniors, so this is within the realm of possibility. Or was this an anomalous season for Vettel? Top drivers are not immune to downturns in form. Senna suffered one in 1992 when McLaren lost their advantage and he was left watching Williams cars disappearing into the distance. In Senna’s case, he raced more brilliantly than ever the following year. No driver should be definitively judged on a single year.

Examining Vettel’s topsy-turvy relationship with Webber provides additional insights. From 2009-2010, Vettel beat Webber 18-13 in races. From 2011-2013, Vettel beat Webber 43-7 in races. What caused this sudden change in their relative performances? There are two good candidates: Pirelli tyres and changes to the regulations surrounding exhaust configuration that affected diffuser blowing. Webber temporarily regained the initiative in early 2012 when new restrictions on exhaust design reduced Red Bull’s rear stability. In 2014, a single upward-facing tailpipe exhaust was mandated, which completely eliminated the possibility of diffuser blowing. Vettel looks less at home in the new cars with their reduced rear stability, but he himself has stated that no single factor is to blame:

“It’s a combination of things. It’s not saying ‘okay, we lost downforce, we lost blown exhaust’ it’s not that, it’s a combination of things.”

One potentially important factor that has not been carefully analyzed elsewhere is the loss of mileage Vettel suffered early in the season due to an enormous number of failures in practice and races. As Vettel noted,

“What is really puzzling is that I have done so little mileage in a race season. That you could call really extraordinary.”

In today’s Formula 1, drivers are lucky to complete a total of 6 race distances in pre-season testing, meaning they are still learning the car on race weekends. Graphing season mileage for Ricciardo and Vettel highlights the problems Vettel faced.The graphs show the three pre-season tests (PRE), the three in-season tests (T), and two data points for each race weekend (Rn): these are combined free-practice mileage and race mileage, respectively. Due to technical glitches, both drivers completed less than 3 race distances in pre-season testing. Further problems for Vettel dropped him almost 5 race distances behind Ricciardo by the Canadian Grand Prix. Vettel partially closed the difference by the end of the season.

How much did these factors impact Vettel’s performance? Would Vettel have performed better against Ricciardo in a different car? These are questions that can’t yet be confidently answered. We will need to see both Vettel and Ricciardo matched up against different teammates before we can fully understand this year’s performances.

 2. Lewis Hamilton, 8.23 ppr

2014 was one of Hamilton’s strongest ever years. In qualifying, he looked unusually vulnerable, but in races he was supremely quick and more composed than ever. The new regulations proved a great boon to Hamilton due to the increased role of energy recovery. Under braking, the rotational kinetic energy of the wheels and the linear kinetic energy of the vehicle is converted into heat and sound by the brakes. At the front wheels, none of this energy is recovered. At the rear axle, some of the energy is harvested by MGU-K (motor generator unit – kinetic) to improve fuel efficiency.

One of Hamilton’s greatest gifts is his brake control — this is the area where he really troubled Alonso in 2007. Hamilton has a fantastic ability to pick the last possible braking point and he’s phenomenally good at feeding out the brake pedal at the correct rate. This allows him to run slightly more rear brake bias than most drivers, as he’s able to judge exactly how much pressure the unloaded rears can handle without locking up. Thus, he can brake incredibly late, setting up overtaking moves that most drivers would find impossible.

The upside of running extra rear brake bias, besides gaining time, is less energy wasted at the front wheels, more energy harvested at the rear axle, and therefore greater fuel efficiency. Time and again, Hamilton’s preternatural feel for the brakes allowed him to go faster than Rosberg while paradoxically using less fuel. Over one lap in qualifying, this wasn’t worth much, but over a race distance it paid dividends. All year, Hamilton seemed to be pushing this advantage to its absolute limits. We saw that in his surprising rear-brake lock-ups at Austria and Brazil, as well as his rear brake failure at Montreal (renowned as one of the most brake unfriendly tracks on the calendar) after the MGU-K system failed.

At most races, Hamilton had a small edge on Rosberg, allowing him to win 11 races to Rosberg’s 5. However, Hamilton’s points advantage over Rosberg was limited by the incredible performance of the Mercedes, because even a relatively poor performance was still usually good enough for 2nd place. With a slightly less competitive car, poor performances would have been more heavily punished and the Mercedes drivers would have more frequently had to engage in combat with other cars, which might have been to the better driver’s advantage.

I decided to use a numerical simulation to test this hypothesis. I modeled a field of 20 drivers, whose race performances are represented by numerical values (higher value = better performance) drawn from a Gaussian distribution (mean = 0, standard deviation = 1), and with each driver having a small (1 in 7) chance of failing to finish each race. To this field, I added 2 drivers, each of whom has their own Gaussian distribution for their race performances with their own average performance value. Both drivers were assigned a standard deviation of 0.2, since this achieved a quantitatively realistic spread in race finishing positions across the season (it would be interesting to consider drivers with unequal standard deviations, but I didn’t do that here). I then added a team performance factor to both drivers’ race performances. These two drivers also had a small (1 in 7) chance of failing to finish each race.

In each simulated race, the finishing order was determined by the numerical order of race performances. I used this method to simulate 10,000 seasons, each of 20 races, for a variety of different settings. I was then able to ask: What is the probability of the better driver scoring more points across the season, and how does this depend on: (a) the performance difference between the two teammates, and (b) the performance of their team?

The results are graphed below. The horizontal axis represents the performance difference between the two teammates, quantified by the probability of the better driver beating their teammate in a race where they both finish. The vertical axis represents the team performance, quantified by the average number of points the team scores across a season with two equal drivers.

The graph shows that when a driver has a 60% chance of beating their teammate in a race, their chance of finishing ahead in the championship is about 60% in a very dominant car (team points ~700), about 75% in a midfield car (team points ~200), and about 60% in a backmarker car (team points ~20). In other words, the stronger driver has the best chance of outscoring their teammate when they are in a midfield or lower midfield team (team points ~100-300). When the car is near the front, championship positions are more affected by random DNFs. When the car is very uncompetitive, championship positions are largely determined by the single best race result in the season, which can involve considerable luck (e.g., Marques vs. Alonso).

1. Fernando Alonso, 9.13 ppr

It has been a remarkable year for Fernando Alonso, reminiscent of Senna’s 1993 or Schumacher’s 1996. The 2014 Ferrari was hopelessly underpowered and difficult to handle. It belonged in midfield, where his world champion teammate spent his season, finishing between 6th and 12th in all but one of his races. Alonso finished in the top 5 eight times and astoundingly almost won in Hungary despite being caught by the safety car just after he had passed the pit-lane. Alonso dragged his car into places it simply shouldn’t have been, routinely battling wheel-to-wheel with Red Bulls and McLarens. In Austria, he even came close to catching Williams in what he described as his best race of the season. Without Alonso’s contribution, the model thinks Ferrari would have been a distant 6th in the constructors’ championship (see Bottas’s entry above).

Before 2014, Alonso’s brilliant career already put him among the all-time greats. The model rates him the 3rd greatest driver since 1950, behind only Jim Clark and Jackie Stewart, and marginally ahead of Michael Schumacher. This year further cemented his legacy. In the same car, he made a world champion look completely ordinary. Overall, Alonso beat Raikkonen 16-1 in races and 16-3 in qualifying. In qualifying, Alonso was on average 0.53 seconds ahead. In races that both finished, Alonso was on average 33 seconds ahead. One has to look back to Prost’s 1986 domination of Keke Rosberg — another Finnish champion struggling to adapt to turbo engines and fuel-saving — to find a double champion duo that delivered such lopsided results.

One of Alonso’s greatest strengths is his ability to manipulate and provoke the car on corner entry, which he exploited to help tame the unruly F14T. Different cars respond to different types of manipulation; Alonso is adaptable enough to employ whichever methods the vehicle demands. In the mid-2000s on Michelin tyres, he mastered an extremely violent turn-in that looked foreign to Formula 1. Rapidly increasing the front slip angle caused a dynamic response, giving the car some initial attitude followed by sudden induced understeer, with the front then sliding on the approach to the apex. If the car bit more than expected, his famously fast hands were ready to catch any oversteer. There’s a superb explanation of this peculiar but massively quick driving style in this article.

With the Pirellis tolerating less slip angle than the Michelins and having a tendency to fall to pieces if provoked too much, Alonso has been gentler in his initial steering input in recent years. However, he has continued to use his feet to manipulate the car on entry. For most of his career, Alonso has had a tendency to briefly tap the throttle (up to about 5-10% throttle) at the end of the braking phase, just before arriving at the apex. This helps to rotate the car into the corner, setting it up for a straighter and therefore faster exit, which is particularly important with the monstrous torque engines of 2014. This is a stylistic choice, which doesn’t suit every car or driver. Ricciardo is another fan of the gentle throttle tap, while Hamilton uses a style more closely aligned with Schumacher’s, progressively releasing brakes, dialing in steering, and easing in throttle to achieve a neutral premeditated slide at the apex all in one fluid movement, rather than jabbing the car into sliding with the throttle. Bottas doesn’t tap the throttle, but gets on it very early, introduces it progressively, and then reacts to any excess oversteer generated on exit. Raikkonen has typically been closer to Hamilton/Schumacher in his throttle application and much prefers a car to respond sensitively to his inputs on corner entry, rather than needing blunt manipulation.

One of the most impressive aspects of Alonso’s season was the lack of any serious errors in a car that frequently brought Raikkonen unstuck. Although Alonso spent the year savagely battling other cars, he made fewer errors than the Mercedes pair, who were usually on their own out front. As shown in the graph below, the model ranks Alonso’s performances over the last several years at Renault and Ferrari a significant margin ahead of his main championship competitors. The difference is comparable to Schumacher’s performance advantage over his competitors in the 1990s.

Interestingly, there are no signs of Alonso slowing down, with stable performances since 2008. The model rates 2014 Alonso’s 2nd best year, narrowly ahead of 2005, 2006, and 2012. The only higher-rated performance was 2009, a year in which Alonso scored all of Renault’s 26 points.

One way of visualizing Alonso’s impressively stable form is to look at how his race performances have varied over time. For each of the six drivers who have been active since 2007, I took all race results relative to their teammates, excluding races where either driver had a mechanical DNF. Using a sliding window 9 races long, I calculated the fraction of races in which they beat their teammate. Alonso’s record is shown below.

Following his difficult start to 2007, where he seemed to have difficulty adapting from Michelin tyres and Hitco brakes to the rounder-profile Bridgestone tyres and the stiffer Carbon Industrie brakes, Alonso has consistently beaten his teammates around 60-100% of the time.

Compare that to Hamilton, whose form seems less stable by comparison. He has gone through three difficult phases over the same time period: late 2007, 2011, and late 2013. Hamilton has always been his own worst enemy, so it will be interesting to observe whether his seemingly heightened maturity will carry forward into 2015.

Vettel’s fortunes have also risen and fallen. While he dominated Mark Webber overall, there were relative dips in 2009, 2010, and 2012.

Rosberg had a slight dip in his performance relative to Nakajima in 2008. He then ran into progressively more difficulties against Schumacher across 2010-2012.

Button had a temporary up-turn in his performances relative to Barrichello in early 2009, which helped carry him to the championship by cashing in on points when Brawn were at their best. Against Hamilton, he had a temporary gain in 2011 as Hamilton slumped. He then dominated Perez in early 2013, but had less of an advantage later in the year.

Finally, Massa showed an enormous relative improvement to Raikkonen in 2008. He then fell progressively backwards relative to Alonso, before relatively gaining on Bottas in late 2014.

Alonso’s very stable form across multiple teams, multiple teammates, and multiple waves of technical changes is phenomenally impressive. If McLaren-Honda can deliver a championship contending car, expect him to be there ready to snatch the title.

Advertisements

81 comments

  1. just brilliant brilliant stuff.

  2. Wow!!! An incredible article, spectacular!

  3. Great article. I’d like to play myself someday with those statistical models you are using. Keep up the good work.

  4. Excellent read, I enjoyed this a lot!

  5. THIS IS MAGNIFICENT. It is not only your mathematical grasps but the analysis you provide where you take fact from history and make it easy for us to read. Your work can not get any better. Truly outstanding. Thank you very much!

  6. Great article.but between Hamilton and Alonso i thing this year Ham deserves 1 st place.ur right they are far better drivers than anybody in the current grid.there is aways neck and neck between them.great article man.

    1. Actually hamilton was closer to ricciardo in the tally then to alonso. I think.ricciardo was the best driver this year. you say hamilton and alonso are far better drivers then anyone on the grid, yet hamilton only just beat rosberg, and vettel beat both drivers to 4 championships in a row.

      1. only because of the car, not his ability

      2. So all those statistics spoke nothing to you I see.

  7. Thanks a lot. Glorious read!

  8. The numbers being exceedingly well researched, it is the qualitative aspect of the article that fully seals the deal – outstanding! Gives perspective and makes a lot of sense.

    Thank you!

  9. A. Clemons · · Reply

    Fantastic article, and well analyzed.

  10. Wow. Just wow

    Amazing

  11. One area this misses which I believe contributes to Alonso beating Hamilton, is that whilst Lewis had pressure on every race once it became clear very early in the season that either him or Nico would be champion, every race for Alonso was ‘nothing to lose’.

    Completely opposite pressures on them.

  12. absolutely impressive piece of work…or art I would say!

  13. Terrific analysis. Thank you for putting in the effort this must have taken

  14. This is just excellent, great read! You sir/madam are a legend 🙂

  15. Statistics are incredible, fantastic job. However, the interpretation is really unrealistic. A piece of advise: It doesn’t exist any mathematical model that involves all the variables (for example, it’s obvious that drivers change every year), therefore the most important thing is the interpretation. You have a SUPERB model, but probably have to improve that second part of the equation.

    Here are some examples of evident mistakes caused by the misinterpretation:

    1.- RedBull 22 points ahead of Williams with identical drivers: That’s not realistic. Massa has outperformed Bottas consistently when both of them have been far from problems. Bottas has NEVER shown better race pace than Massa (you can analyse every race), and Massa was extremely far behind Alonso in this aspect. The RedBull has only been clearly quicker than the Williams in Malaysia, Spain and Monaco, while we can discuss about China, Bahrein and Singapore. (An example of worg estimation: if Massa hadn’t have any problems in Sochi (Q1), he would have finished at least 4th. Never 9th…).

    2.- Perez P17: ¿Behind Ericsson? ¿Really?

    3.- Bottas P4: “This suggests that Bottas is still SOME WAY behind Alonso in absolute terms, given Alonso dominated Massa every year from 2010-2013. But it also means that Bottas’s performance is rapidly on the rise.” ¿”SOME WAY”? Massa had a half a second slower race pace than Alonso (on average). Bottas has had a slightly slower race pace than Massa all year, therefore Bottas is, at least, 0.5 s slower than Alonso in race pace. And that’s an entire world… Finally, remember that Bottas was frequently outperformed by Maldonado last year.

    4.- Räikkönen P8: Well, Alonso is an incredible driver, but Kimi’s pace has been even slower than Massa’s one, this doesn’t make any sense… Räikkönen can’t be ahead of Grosjean, I think.

    5.- Hülkenberg P10: How that is possible? Don’t you remember that Perez beat Kobayashi, and Kobayashi destroyed Trulli in his first GP? Hülkenberg has easily outperformed Pérez (despite the 2 tenths / lap weight penalization) every GP, although strategy has always been favorable to Pérez (Austria, Canadá…). I simply can’t understand it.

    Anyway: Congratulations for the article. Really hard work. I hope you understand my comment, I respect you, you’ve done an incredible effort.

    1. Thank you for the kind words and the time you devoted to writing this. I’m afraid I disagree with each of your points though.

      Different observers this year have espoused different opinions regarding which was the better car of the Williams and the Red Bull, and we can add your argument to the pile. You’ve pointed to Williams being the quicker car (in your estimation) more often. Others will point to Red Bull easily outscoring Williams despite a seemingly sub-par performance from Vettel. The chief difficulty in comparing cars is naturally the fact that they have different drivers. That’s where a model can be extremely useful.

      Many of your points revolve around the premise that Massa outperformed Bottas in races in 2014. The evidence doesn’t really support this claim. As you can see under Bottas’s section, I analyzed what would have happened if neither driver had experienced any serious misfortune and the answer (give or take some points here or there that we can always argue about) is that it would have been very close between the Williams drivers. I don’t see the salience of Bottas being frequently outperformed by Maldonado in 2013, which I discussed incidentally, given that he was a rookie at the time. You also note that Bottas/Massa are a long way behind Alonso, but so is everyone by the model’s reckoning.

      I acknowledged that the performance metric used (points per counting race) is a little rough on Massa given his improbably bad luck in 2014. I said that he therefore deserved a higher rating, probably just ahead of Raikkonen and much closer to Bottas.

      Regarding Ericsson, I already discussed the unavoidable uncertainty in his ranking. Nonetheless, I don’t see anything to suggest that Perez is obviously a better driver than the likes of Kobayashi and Ericsson. Perez’s junior career was highly unremarkable.

      Regarding Hulkenberg’s ranking by the model, I think that is adequately addressed in the article.

    2. @davidizki Agreed.

  16. Daniel Saxlid · · Reply

    KEEEEEEEEEP UUUUUP THE GREAT WORK! The amount of effort and time that goes into this material does not pass unnoticed! Thanks!

  17. 1.- “As you can see under Bottas’s section, I analyzed what would have happened if neither driver had experienced any serious misfortune and the answer (give or take some points here or there that we can always ARGUE ABOUT)”

    Here it’s exactly where I don’t agree with you. I think that some estimations (Massa 9th in Sochi, for example) are too conservative.

    2.- “I don’t see the salience of Bottas being frequently outperformed by Maldonado in 2013, which I discussed incidentally, given that he was a rookie at the time. ”

    Maldonado appears P15 in the Ranking, eleven positions behind Bottas. Do you really think that Bottas has improved so much in a 10 months period? I can’t agree. He was a rookie, ok. But this improvement is a little bit unrealistic for me.

    3.- “Regarding Ericsson, I already discussed the unavoidable uncertainty in his ranking. Nonetheless, I don’t see anything to suggest that Perez is obviously a better driver than the likes of Kobayashi and Ericsson. Perez’s junior career was highly unremarkable.”

    Perez finished 2nd in his 2nd season in GP2 (5 wins). Ericsson finished 6th in his 4th season in GP2 (1 win), with the best team (DAMS). Honestly, I appreciate a huge difference.

    4.- “Regarding Hulkenberg’s ranking by the model, I think that is adequately addressed in the article.”

    Hülkenberg has made three mistakes this year: Hungary (race), Japan (Q), Bahrein (Q). The rest of the year has been PERFECT, although he has had a tremendous lack of luck (Bahrein, Spain, Canada, Austria, Belgium, Italy, Singapore, Japan, Russia and EEUU). If you take into account penalizations, lack of upgrades (not Perez’ case), damaged floor (Sapin and Italy)… It’s very hard to imagine that, all the drivers with the same car, Hülkenberg would finish 10th.

    I’m looking forward to your answer, this discussion is a real pleasure for me.

    1. There is no answer for this?

    2. 1. I agree, there are some arguable cases, but it’s not changing the overall picture of a very close match up between the Williams drivers.

      2. The model takes Bottas’s 2013 results against Maldonado into account, and provides no provisions for a driver being a rookie.

      3. Regarding Perez’s junior career vs. Ericsson’s, you’re comparing the one most remarkable thing Perez achieved to one of the least remarkable things Ericsson achieved. Prior to GP2, Ericsson actually looked quite strong, then he was completely mediocre in GP2. Perez kind of just coasted along never looking bad but never looking like a top-level talent either.

      4. Regarding Hulkenberg, I described his teammate comparisons in the article, which have not been outstanding to date. He outperformed Perez in 2014 by a small margin, but Perez is not a particularly high target. There are many different fan opinions of Hulkenberg. I personally don’t have much difficulty imagining him finishing 10th, but my own opinion could of course be wrong, as could anyone’s. I hope 2015 will give us a more decisive verdict on the performance difference between Perez and Hulkenberg.

      1. Well, I’m not 100 % agree with you, but I think we have brought our positions a little bit closer. I still can’t understand how is possible that Bottas was at the same level as Maldonado last year, and 11 positions ahead now. Anyway, thanks for your explanation. I still think this is an incredible article!

  18. I really like this article; I agree about Masa 13 place, He is no consistent like others drives, I also think Alonso is One of the best drivers of the last years, Perhaps Vettel is better than 12 position, but is performance shows that, maybe next year will be much better.

  19. I rarely post comments, but this article is simply brilliant. Thank you for the effort in producing such thorough an insightful article. Keep up the good work.

  20. I wonder why you settled on 10-6-4-3-2-1 as the best points system to use? Instead of say, the one used the most since the start of the F1 Championship (1950-), which is the 9-6-4-3-2-1 system.

    Only a few teammate linkages would remain from the pre-2010 points system anyways (HAM-ALO in 2007, MAS-RAI 2009-09), so it might’ve been better to bring those results up to the current points system, rather than the other way around.

  21. Congratulations! I seem a hard work with very real results.
    Many times I wonder what races are seeing people…
    Thank you so much.

  22. Bro….amazing read ! thank you for taking out the time to make this…..the best article f1 in 2014 !

  23. Excellent as always!
    Can you explain to me why Raikkonen was rated so highly when he was so convincingly beaten by Alonso? They only thing I can think of is that Alonso is so highly rated, but you yourself said that you expected Raikkonen to get 2/3 of Alonso’s points, which he didn’t come close to?

    1. You’re correct, it’s because Alonso is ranked so highly. Had Raikkonen performed to model expectations (about two thirds of Alonso’s points), he would have probably been ranked 4th this year.

      1. Thanks for the reply, it makes sense. I suppose the model would also boost Massa’s performances relative to Alonso in a similar fashion.

        One more question: How are team-mates compared when one is a rookie? For example, I understand how Magnussen could be ranked relative to Button, because Button is a known quantity, but how is Button then ranked? If it’s relative to Magnussen, then Button is only really ranked relative to himself, which makes no sense.

      2. That is an excellent observation. As you say, a rookie has no prior performance baseline, so (no matter what method you choose) it makes teammate performance estimate difficult. In these cases, the model estimates the non-rookie’s performance and their car’s performance based on how they performed in prior years. It can then estimate the rookie teammate’s performance relative to that. In other words, Button’s 2014 performance estimate by the model is based on his typical performance.

        The same applies to privateer teams with only one driver, which is uncommon today, but used to be semi-common.

  24. The best F1 article I have ever read.
    I am a statistics major and have been an F1 fan since 2001, and finally I can have a good framework to compare driver against other drivers and against himself over time.
    One question tho, Webber has beaten a lot of teammates (and beaten Alonso in junior formula). If the ranking considered data from 1950, I am surprised to see Vettel that low.

    1. Thank you. In terms of his all-time ranking, Vettel is still quite high (top 15), the model just thinks this was a particularly poor year for him.

  25. I’ve purposely held back my opinion about this year’s revelations but your analysis confirms it. If I were Ron Dennis, I’d hire VERGNE for 2015.

    Reasons:

    1. He’s outperformed KMag and has virtually tied Button

    2. Without a 2015 seat, he’s got to be bloody cheap.

    3. He’s great in the wet

    1. He would also match up well with Ricciardo, as at Toro Rosso – dry race, Ricciardo wins. Wet race – Vergne wins!

  26. Reblogged this on El Abuelo F1 and commented:
    El mejor piloto (matematicamente) de 2014
    – obligada lectura –

  27. Hey,
    I would like to thank you for your effort to develop a model that helps clarify the greatness of Formula 1 drivers of this era. The value of any scientific model is its ability to predict future outcomes. with respect to a discussion of meaningful outcomes for Formula 1 drivers, I am interested in season-long performance. It’s not clear to me that your model is very useful in that regard.
    I wI’ll only bore you with one critique. Your model does not appear to adequately account for how the team-driver admixture affects driver performance.
    Two principal interactions take place between the driver and team. The first is car development. The second is performance during a race weekend. The former, I would argue, is more rational than the latter and therefore more predictable than it as well.
    If I understand you commentary correctly, your model discounts similar performance of team-mates when during a car that is superior to the rest of the field. If that is true, then that is exactly wrong.

    It is no badge of honor for Alonzo to out-drive his team-mates so comprehensively year after year. It only demonstrates that he has been unable to help the technical side develop a better car.

    Unlike, Vettel, who, until this season enjoyed the benefits of the aerodynamic genius of Adrian Newey, Hamilton’s cars have been competitive because of Hamilton. Hence both he and his team-mates both past and present have profited from his technical feedback in the off- and preseasons. (Did your model predict the improved performance of Rosberg resulting from HAM’s arrival or the Button’s decline as a result of his departure?)

    Separately, the on-track, driver-modeling must account for team performance relative to team-mates. This a bit more of an art, to be sure, but looking at intra-team, pit stop battles, for example, can provide a foundation for weighting team bias for or against a driver.

    Equally difficult is the challenge of a model to account for strategic bias. Unfortunately, team strategy regularly affects driver performance. We saw it affect both Alonzo and Vettel the latter part of the season. We also saw it affect Hamilton at the end of 2012.
    Thank you again for stimulating this discussion.

    1. Thanks for the message!

      “If I understand you commentary correctly, your model discounts similar performance of team-mates when during a car that is superior to the rest of the field”

      That is certainly not true. I’m not sure how that was miscommunicated between us. The model simply fits driver performances and team performances so as to best describe the empirical race result data. It doesn’t treat drivers in front running teams any differently from others, nor does it apply any sort of arbitrary penalties to drivers in more competitive cars.

      There are certainly many underlying factors that contribute to team performance and driver performance, including the ones you describe, as well as many others.

      There is a strong argument for not even attempting to quantify these factors separately when it comes to using a model to rank drivers. A driver might beat their teammate due to being better at qualifying, being less crash-prone, being a later braker, being better at set-up, being better able to adapt to the car, being more consistent, etc. All these factors are integrated by the one sporting metric that matters: race results. While it would be extremely insightful to quantify the numerical contributions of various factors, it doesn’t yield us a better metric.

      I did mention explicitly in the article that team/driver interactions are not modeled, and that’s largely because they cannot be reliably quantified. Saying that a particular driver has or hasn’t helped car development or is more suited to a particular car than their teammate is at best qualitative speculation. There is simply no hope of quantifying these factors in any given modern year, let alone for the assortment of drivers and teams in earlier eras. Metrics such as pit-stop timings fail to generalize even between decades, let alone back to 1950,

      Regarding the model’s predictive value, it was shown to do a reasonable job of predicting lap-time data from 2010-2013 in the published paper. I intend to also apply it to predicting teammate results in the near future,

      1. Thanks for the reply. First, I have to admit that I made a logical leap that I did not make explicit. Fir me, measuring driver performance is really only meaningful at the Driver-Championship level. I regard season-long performance as the appropriate measure of a driver’s ability.

        I further restrict my analysis to the great and nearly great based on championship performance and head-to-head competition.
        I was referring to your comment in the fourth paragraph of the Hamilton analysis where you make the point that Hamilton’s superiority to Rosberg is less obvious because of the superiority of the car. The model penalizes Hamilton by not providing enough granularity of measurement in the areas that I mentioned earlier.
        As for aggregating useful metrics of team performance that affect driver performance, one doesn’t have to use absolute times of pit stops. One could count the number of times a driver had faster pitstops than his team-mate(s) in a race or a season. In the absence of bias, one could reasonably expect the number to be equal over a season within a delta which could be determined by avg the team-mate delta of the top three constructors.
        The Same could be done with failures. First seek an intra-team bias, then compare actual number to the avg number of failures for drivers of the top-three constructors.

        This sort-of analysis may also clear-up the misconception that some drivers are “harder on cars” than others. Again I point to Alonzo’s end-of-season failures after having gone 30(?) races without one. I posit that these are bias-driven. Yet, by excluding DNFs, you inadvertently include the bias in your model by penalizing drivers who finish races out of the points because partial failures, eg., loss of a tire, failed wiring loom, etc.

      2. I think you might have been confused by me using two completely different models. In the Hamilton section, I used a season race simulator model to determine the likelihood of one driver outscoring another across a season depending on the car.

        I use a different model for ranking driver performances based on points per counting race (not all races), the mathematics of which is described in the previous all-time rankings article. Due to the sigmoidal scoring function in that model, it does in fact account for the simulator-based observation that teammates’ points scoring rates tend to be somewhat compressed in very dominant or very bad cars. In other words, the model should be able to detect differences in driver performances equally well regardless of the car. There’s certainly no bias towards compressing rankings of drivers in top cars.

        Like I said, pit-stop data don’t generalize to other eras. For one thing, those data were never collected going back a few decades. For another, pit-stop timings vary based on race strategy with refueling. For another, in the early days of F1, pit-stops were a different beast altogether, with drivers often having no pit-stops at all.

        Car failures are also not useful for quantifying team bias, because the numbers are much too small to detect systematic bias from random chance.

  28. I like your “scientific” approach to F1, but to me these rankings are too often wrong. They do not show how a driver actually performed during a season because “ranking” a driver is impossibile with just maths and statistics. Of course your method as to simplify some things and not count others, but still only eyes can evaluate a performance since you take count of everything happening during a race weekend, which is impossibile to do with any method. And still rankings are so hard to make as drivers do not have the same team and machinery.

    About the specific rankings, some look pretty wrong. Raikkonen had a terribile year but he’s still 8th, massa was extremely good in the second part of the season, but is back in the field. Magnussen is a rookie and did not perform bad against the solid and expert Button, Grosjean so high despite having so many troubles and retirements that is hard to make any comparison with other drivers near him, and Alonso first with 1+ points than Ricciardo, who was absolutely amazing?

    I think that, despite your nice effort, it is impossibile to have maths evaluate drivers.

    1. I don’t mean to single you out, but opinions like these suggest a fundamental misunderstanding of the purpose and function of models. Perhaps this is due to a widespread lack of formal scientific training in this domain.

      The primary goal of building models is NOT to achieve a perfect, unassailable representation of the underlying system. It is instead to gain new insights into the system through abstraction and to understand and quantify the roles of different assumptions by making them mathematically explicit.

      When you tell me where you think certain drivers should rank, you are also using a model, whether you are aware of it or not. Using the set of data you have available, you are making certain inferences and assumptions. These processes occur inside your brain rather than inside a mathematical model in a computer, but it is a model nonetheless.

      When you tell me that my model is “right” or “wrong” on certain points, the method you are using is to compare the results of your internal model to the results of my mathematical model. I don’t consider this a satisfactory method of validating a model, for several reasons, many of them hopefully obvious. Perhaps most obviously: you shouldn’t be the arbiter of which model is “right” when they disagree. Cases of disagreement might well be highlighting biases and errors in your internal model.

      If we compare your internal model to my mathematical model, there are several clear differences.

      1. My model is objective and guaranteed to give the same results given the same inputs every time. Yours is subjective and liable to bias, uneven weighting of data, misperception, and lack of replication in results. My model has been validated by scientifically testing its predictions against data; yours has not.

      2. The assumptions and limitations of my model are laid out plainly for everyone to see. It is easy to interrogate the model and understand the effects of each of its assumptions. We can understand why one driver is ranked above another and debate whether the assumptions that lead to that result are fair or reasonable. The assumptions and limitations of your model are not possible to see; they are implicit and they are difficult for anyone to fully interrogate (even including you, since opinions can be swayed by factors we are not even consciously aware of).

      3. Mathematical models can be objectively compared by testing their predictive value on various data sets. By contrast, every F1 fan has an internal model they use to rank drivers — you and me included — and there is no suitable method for comparing those models. In other words, why is your opinion regarding Alonso and Grosjean any more valuable than that of the fan who tells me their rankings are just right?

      1. Hi, thank you for your reply.
        First of all, I would like to underline that I completely understand what you are trying to do and how you use your model. I am myself a physicist, I am into scientific methods and mathematics everyday so I agree with you that of course what you want to do is to use facts and quantify things.
        And I agree that, by saying that your model is not ok, I am using my own model. BUT, as I said in my previous post, I think that evaluation of a weekend performance cannot come from “quantifiable data” only. Too many aspects of a driver’s year come from more than qualifying and race position and so on. If you look at a race weekend, you know that driver X had a problem during practice, had a bad set of tyres during a stint in race, had a terrible pit stop and so on. Or driver Y made a fantastic recovery, a great series of overtaking manouvres, and so on. These things are not, obviously, in your model but are in “mine” because I saw-read-knew them. Obviously the question with my model is, is it really impartial? Of course not, because two overtakings have the same result, but one can be basic, another can be amazing and only “opinion” counts for this kind of things. But they are necessary, to me, to a good evaluation of a driver’s season. Data for a good model would be too much to add, and still be not enough, this is my point.

  29. Fantástico artículo

  30. Dear f1metrics

    I thank you for your response to Beyond. I do so because with it, you help explain just how radical your approach is. Radical because it is the first time anybody has rated drivers SCIENTIFICALLY! Your work is akin to Alec Issigonis’ when he conceptujalized and produced the MINI. Until then, nobody had thought about putting the engine transversally with the gearbox underneath it. That invention, which was improved over the years was marvelous to begin with as it brought mass produced motoring into a new era.

  31. WOW!! 9.13 for Alonso this year. This must surely be a top ten year. And apparently even better in 2009. The championship years, 2005 and 2006, are surely not far behind.

    I just found this blog in a link from F1Fanatic and absolutely loved it. The July post about the best F1 driver ever was jaw-dropping. Everybody has their own list of favorites but they are mostly based in anecdotes and sentimentalism. This was by far the best I ever read on the subject, and believe me, I’ve read plenty.

    But together with the new post on 2014 a lot of questions are left open. Surely with this tenth “virtual championship” Alonso has matched Schumacher’s record. But what about Clark’s record of three years in the top ten? Has now Alonso matched it?

    Reading the July post I gather that the top year ever was Jochen Rindt’s 1970, then Clark’s 1962 and 1965. Then.. little more. Again Clark’s 1965 was 7th and Ascari’s 1952 was 8th. Schumacher never made it, his best was 1994’s 13th. I’d guess Stewart and Alonso have at least one each, may be two (but not three). Still there is at least one year in the top ten missing (Fangio??). Did I miss something? And how has Alonso’s 2014 changed the ranks?

    This “top ten years” is only a token sample of all the wealth unearthed by your model. I really would like to see a table with all the (significant) drivers’ scores year by year according to your model, and so much more. I hope future posts will take care of this, I can’t get enough. Or if you have this material published somewhere please let me know.

    And please keep up the astonishingly good work!!

    1. Thank you! You would probably find the published paper very interesting, as it addresses some of your questions, such as the list of top single-year performances. Feel free to send me an email (listed on the About page) and I’ll get you a copy.

  32. … And another big question, the virtual championships. They seem to follow Matthew’s law, the rich get richer. Schumacher goes from 7 (real) to 10 (virtual), Fangio from 5 to 6, Prost from 4 to 6, Alonso from 2 to another unbelievable 10… but Vettel from 4 to 0.

    The last 22 years have been owned by Schumacher (1993-1998, 2000-2002, and 2004) and Alonso (2003, 2005-2006, 2008-2014) with Frentzen on top in 1999 and Hamilton in 2007. BTW the 7-year stint by Alonso was surely the longest ever… and it came very close to a 10-year one, being minimally outperformed in 2007 by the second-best rookie of all time (after James Hunt).

    With the data already posted I can make oiut large chunks of the virtual championship table…
    But I want it all!! Please!!

  33. I’m with Hyoko. Is it too much to ask if you’d be kind enough to share your list of top single-year performances as a post? This should allow for an even better understanding of your fantastic work. Thanks

  34. Matt Karshis · · Reply

    Once again a great article. I figured that Alonso would compare well in this model as his comparison point of Kimi this year underwhelmed.

    Hamilton did well with 3 DNFs this year which is right at the target and well below McLaren levels – can you tell me how Rosberg and Hamilton’s numbers would have changed if he had pulled the car in early to get a DNF instead of a 14th in Abu Dhabi?

    Alonso’s late season effort talk of him “phoning it in” late this season may bear merit in the future particularly if the Honda-McLaren package is not competitive in 2015. His race pace in the last 5 races after Singapore was 1.3% off the lead and managed 3 6th places and a 9th – whereas till Singapore he was only 0.6% off pace and as mentioned nearly won in Hungary and scored a podium in China averaging 5th place. In the same period Raikkonen went from 1.22% off pace thru Singapore to 1.7% off pace for the final 5 races. So he dropped more than Raikkonen did – and more than Vettel in the same period even though Vettel was leaving RBR.

    1. Good question! Had Rosberg parked it in Abu Dhabi, it would have very slightly increased his adjusted ppr to 7.14. Since positions 4-7 were all so close in ppr, this would taken him to 4th in the list.

  35. Matt Karshis · · Reply

    Thanks, I thought it would be close. By the way, did you count Abu Dhabi with single points – or double – and if double did anyone’s ranking change in the last race?

    1. Every race in history is treated the same by the model, so no double points!

  36. How much of a difference would Perez & Massa (I can’t remember if you made that an exception in this model) between having a classified finish just outside of the top 10 and driver DNF’s?

    1. Very little, as they would be scoring almost no points either way. The model awards fractional points for lower positions, going down by a factor of 10 for every 5 places.

  37. Fantastic piece of work you’ve got here! It’s brilliant to have an entirely objective analysis done with what appears to be an outstanding model.

    I do have a question though, if I may. From what I gather of your model, it harms a driver’s scoring rate if he ends his race by his own hands (e.g. crashes), but DNFs from mechanical failures are excluded and don’t harm scoring rate. My question is what happens if one driver ends another driver’s race, e.g. driver A crashes into driver B. Does driver A suffer scoring rate loss, while driver B has the race excluded from scoring rate calculations? Or do both suffer a scoring rate loss? How is it decided whose fault it was?

    My reason for asking the question is because I always felt that Hamilton performed very well in the 2012 season (I know it’s subjective, but hear me out :P), but your model ranks it as one of his lower performing years. As I’m sure you are aware, his points total was harmed by a lot of bad luck that year. He had 6 retirements, none of which were his fault. Three of his races were ended by other drivers crashing into him: Maldonado in Valencia on the penultimate lap while Hamilton was running 3rd, Grosjean at the start in Spa when Hamilton started 8th and Hulkenberg in Brazil while Hamilton was leading the race. All three times the other driver was given a penalty and Hamilton was judged to have done nothing wrong – do you follow steward’s decisions in deciding whose scoring rate is harmed? If you do, how are things decided for 1950s, 1960s etc. accidents? If you don’t follow steward’s decisions, then do you use your own judgement, or do both just get the blame?

    Also, in regards to mechanical retirements, Hamilton had three reliability-related retirements in 2012 – two of them were from the lead of the race in Singapore and Abu Dhabi (the other, Germany, came after he had got a puncture on the first lap so he was down the order anyway). Is any sort of boost given to driver rating for qualifying on pole/leading much of the race, or is the entire race just excluded? What about races where the driver suffers mechanical problems but still finishes (e.g. Korea 2012 where anti-roll bar failure greatly increased Hamilton’s tyre wear and dropped him from fourth to 10th) – are these treated as normal races?

    There was also Spain, where Hamilton ran out of fuel on the way back to the pits after a dominant pole in qualifying (0.5 seconds clear of the field), due to an error by the team (the fuel rig guy had the handle set to drain fuel for a few seconds, and didn’t put enough fuel back in to fully compensate for this). Even if you adjust for the lighter fuel load Hamilton would have still been on pole, but instead he was excluded from qualifying and started at the back of the grid. Hamilton’s Japanese GP weekend was also hampered by rear-suspension problems. There were also pit stop problems earlier in the 2012 season, including Monaco (put Hamilton down from 3rd to 5th), Valencia (pit stop problem dropped him behind ultimate race-winner Alonso and eventually meant that Hamilton was fighting with Maldonado for 3rd, leading to his retirement), and Bahrain (two slow pit stops cost him 22 seconds in total, dropped him from podium positions to 8th) – do races like these get treated like normal races, or excluded due to factors outside the driver’s control?

    James Allen calculated that Hamilton lost around 110 points due to factors outside his control in 2012. The article didn’t even take into account Spa or Germany (due to the difficulty of estimating where he would finish) or Brazil (the article was published before the Brazilian GP), so he probably lost around 145 points overall from bad luck. Note he was 89 points behind Vettel at the end of the championship – it’s part of F1 that you always suffer some bad luck of course, but if McLaren had a similar operational record as the other front-running teams then Hamilton would have been one of the main championship contenders.

    Obviously the model can’t calculate everything, and I can appreciate that it would probably be too difficult to individually look at every incident/reliability issue (etc.). As I said before, I think your model is brilliant, don’t think I am arguing with it! 🙂
    I just felt that Hamilton’s 2012 performance was better than most people tend to rate it, he made very few mistakes all season, but suffered a from massive amount of bad luck – being dogged by reliability issues (especially the timing of when they occurred), numerous pit and operational errors by McLaren and just plain bad luck – and I was wondering how much of this is taken into account in the model.
    To be honest, the multitude of reliability and tactical blunders by McLaren were probably a good thing in the end, as I imagine they contributed greatly to his decision to move to Mercedes – which worked out rather well!
    Sorry for rambling (I think I went a bit off-topic there), but thanks very much if you managed to read this wall of text!

    1. No fault is assigned by the model in the case of accidents. This is for a couple of reasons: (1) It becomes a subjective exercise, and I don’t want ANY subjective variables in the model. (2) There is no footage or even written reportage of many accidents in older races. The same goes for other “bad luck” situations, such as pit-stop errors and mechanical difficulties that hamper the driver but don’t cause an outright DNF.

      In the long-term, good and bad luck should balance out for most drivers, so I don’t see this as a major flaw in the overall ranking scheme. One might also get into a philosophical argument over whether drivers who consistently put themselves in situations where bad things might happen ought to be punished for that (Valencia 2012 and Interlagos 2012 both spring to mind as accidents that were clearly not Hamilton’s fault but that a more prudent driver might have avoided altogether knowing the conditions and who he was racing against).

      Nevertheless, I agree that the points per race metric is not necessarily entirely fair in any given season, because a handful of unlucky events can have a significant effect on points scored. Massa’s 2014 season is a good example of that, which I discussed under Bottas’s entry. For the model, I settled on removing mechanical DNFs as a totally objective method of accounting for the most damaging events. It’s difficult to go further than that without introducing subjectivity.

      1. That’s very true, eliminating subjectivity is one of the best aspects of this model, so I can see why assigning blame for crashes wouldn’t work (and would be impossible for older races, as you said).

        Also, your comments about drivers putting themselves in risky situations is a very good point, and something that hadn’t occurred to me – Hamilton does tend to put himself at risk a lot (he demonstrated that multiple times this season actually! 🙂 ), but it does make him exciting to watch. Some of his moves such as the one that annoyed Rosberg in Bahrain and his overtake at Austin seemed particularly risky. His defending at Spa is difficult to say – he could have taken a wider line to leave Rosberg more room, but I’m not really sure where Rosberg was planning to go with that move. It just seemed a bit clumsy to be honest (I can’t believe some people actually think that he intended to puncture Hamilton’s tyre though, that would require some surgeon-like precision).

        On an unrelated note, this model just goes to show how much some people have underrated Alonso’s season – some people accused him of losing motivation (which he denied), and I have seen some media outlets rate his season with scores like 7/10, despite his huge performance advantage he had over Raikkonen in the same car.

        Everyone seems to agree that he’s the best driver in F1, but I think most people don’t really realise just how good he actually is. Alonso’s performances alongside Massa have often been put down to Massa declining after his accident, and this season many just assumed that Raikkonen’s season is unrepresentative because of his struggles with an unresponsive front end and the tyres. Massa’s form this season (once he got past his atrocious luck) suggests that the driver who came close to the championship in 2008 is still in there somewhere, and whilst I am sure that Raikkonen largely underperformed this season, it must be said that the points gap was huge.

        The playing down of his achievements reminds me of Nico Rosberg’s situation before this season actually – he didn’t really get many plaudits for flattening Schumacher as people just assumed Schumi had declined (he most likely had, but probably not to the degree most assumed), and when he performed very well against Hamilton in 2013 many just assumed it was because of Hamilton’s struggles with the Merc’s brakes. This season I think he finally proved that he is one of the top drivers, though I think he will have to work on his race pace and craft over the winter if he wants to beat Hamilton next year – excluding Monaco/Spa (due to the circumstances) and races where one had reliabilty problems in qualifying or the race, Nico only beat Lewis in Austria and Brazil (and that was largely due to spins by Lewis at those weekends). Lewis meanwhile will probably want to be more consistent in qualifying next year, as he made lots of mistakes in qualifying this year, which made some races tougher for him than they should’ve been (though having the driver with the better race pace frequently starting behind made for some good races). Although they were often just small mistakes (e.g. lock-ups in Canada and Brazil) they were costly because of the extremely fine margins between the two in one-lap pace.
        I got the impression that Rosberg’s ability to consistently put in supreme lap times often put Lewis under great pressure to give his absolute maximum, leading to frequent qualifying mistakes such as lock-ups (and sometimes spins after rear lock-ups) while trying to exploit his braking advantage to the maximum. Lewis certainly didn’t make this many qualifying mistakes against Kovalainen or Button.

        As a final note, I’m interested to see how Alonso performs against Button next year. Whilst I’m fully expecting Alonso to win the points battle (based on what I’ve seen of your model), Alonso is moving into a new team, while Button will be in his sixth season with Mclaren, so hopefully he can put up more of a fight than Raikkonen did (especially as Button seemed to struggle less with the new cars). I wouldn’t want Button to get the kind of criticism for underperforming that Massa and Raikkonen got while up against Alonso, as he’s been through a lot to get the drive for next year. It might have actually been a good thing for Magnussen to miss out on this year’s drive, as if he got hugely outperformed by Alonso next year he might have been booted from McLaren altogether instead of getting his third driver role (rather like what happened to Grosjean in 2009), especially as they have the highly promising Stoffel Vandoorne waiting in the wings.

      2. Hi Marcus

        I enjoyed your post. I particularly liked your admission that some drivers are more crash prone because they get into situations that make it difficult to avoid them. Hamilton used to be one such driver. If you’ll remember his 2013 battles with Massa at Ferrari, how often were the 2 of them putting each other at risk? Same with Maldonado vs. Hamilton, while it does make racing more exciting, what does one gain losing positions because of these mistakes?

        Button on the other hand is all about business. He’s not the fastest but I’d rate him along the cleverest (with Alonso). This is why I think as a tandem they’ll be very productive. My thinking is that if the 2015 McLaren is fast, Button will be pretty close to Alonso. If it is difficult to drive then Alonso will be significantly ahead.

  38. You may or may not be aware of this, but I thought I would mention a little-known fact that Hamilton actually suffered from a loss of engine power during the race in Hungary this season. I’m not sure if that race is included in his “ppr” calculations anyway (due to the qualifying fire and subsequent pit lane start), but it cost him significant time and likely the race win – apparently he suffered the loss of engine power from his second pit stop on lap 40, costing him around 0.5 seconds per lap for the rest of the race (lap 40 to lap 70, so ~15 second loss overall) – it’s mentioned in this interview: http://www.formula1.com/news/interviews/2014/8/16227.html

  39. Hello again, I’ve been fiddling around on Excel to try and get a more visual representation of the figures in this article. I produced this graph of all the 2014 driver performances based on the “ppr” figures in this article: https://imagizer.imageshack.us/v2/725x408q90/r/904/PLoA9w.png

    *IMPORTANT NOTE: I set Nico Rosberg’s “ppr” figure to 7.14 instead of 6.95, the higher value being the one you commented that Rosberg would have had if he had retired the car in Abu Dhabi instead of driving it to the end and finishing 14th. I know it’s sort of deviating from the conventions of the model, but I thought it would be a better representation of his performance because let’s be honest, it was pretty much a retirement, he only carried on because it was the final race and he wanted to drive to the end. I could easily edit the value back to the original one if anyone prefers it that way though.*

    I found the graph interesting, as it made me notice that there were some considerable jumps between certain groups of drivers. Most interestingly, there seems to be a sudden jump of almost 0.7 ppr separating the merely very good drivers from the drivers most consider world championship material. There is also another significant gap between these “champion material” drivers and the top drivers, then amazingly another large gap from the top drivers to Alonso. There was also a considerable gap between the Sauber drivers and all the other drivers, emphasising how you said that their car was capable of points but hindered by poor driver performance (although in fairness I’ve heard that the Sauber was very difficult to drive). I illustrated these gaps with another graph, where I split the drivers into several categories (based on points where there were noticable gaps between two drivers) and then took the average ppr of all the drivers in each category: https://imagizer.imageshack.us/v2/801x230q90/r/538/qPygoN.png

    Here are the categories I split the drivers into:
    Best: Alonso
    Top: Hamilton, Ricciardo
    Great: Rosberg, Bottas, Vergne, Button, Raikkonen
    Good: Grosjean, Hulkenberg, Kvyat, Vettel
    Decent: Massa, Kobayashi, Ericsson
    Underwhelming: Maldonado, Perez, Magnussen
    Very Poor: Sutil
    Worst: Gutierrez

    “Great” and up encompasses all but one of the world champion drivers, as well as many drivers considered to be future world champions. In my opinion, all drivers that are “underwhelming” and up are good enough to be in F1 (particularly Magnussen as he was a rookie), though I imagine some might feel that drivers should be “decent” or higher to deserve an F1 seat.

    NOTES: The rookies could be said to be one category higher in terms of how impressive they were (due to the difficulty of being a rookie in the Pirelli era), so Kvyat’s debut was great, Magnussen’s was decent (etc.). I wasn’t quite sure which category to put Ericsson in at first, as he was sort of halfway between the other “Decent” drivers and the “Underwhelming” ones. He was slightly closer to “Decent” though, so I put him there as I didn’t feel his position warranted an extra category.

    Notably, the only world champion below the “great” category is Vettel, emphasising how torrid this season was for him. As you’ve said in the article, Massa deserves to be higher up than he is because he was so unlucky this season, so it will be interesting to see if he can manage to get into or at least closer to the “great” category next year – since, as you said, he seemed quite well matched with Bottas when he wasn’t having problems or being crashed into.

    1. I also sorted the drivers performance into their teams, ordered by the team’s position in the constructor’s championship: https://imagizer.imageshack.us/v2/645x410q90/r/661/Kqz71X.png

      This gives a rough idea of which teams had the best driver line ups, but I decided to create a clearer graph to represent this by averaging the ppr of the team’s two drivers to produce an average driver ppr for each team: https://imagizer.imageshack.us/v2/802x290q90/r/633/0ZBYwC.png
      As you can see, the best driver line up was Ferrari, though Mercedes is close behind. Sauber have the worst driver line up by some margin. I was surprised to see that Force India is considered to have the second worst driver line up, especially because the driver line up is generally seen as being one of the stronger combinations by most in the paddock. This seems to largely be due to Perez’s low ppr – I feel that the model was quite harsh on him this year, as he actually performed very well relative to Hulkenberg in the second half of the season – unfortunately for him, this was when the car was less competitive so he didn’t get many points for his efforts, while Hulkenberg had strong form when the car was at its most competitive. The result was a fairly significant points gap between them, despite being close in the second half of the season (Hulkenberg was definitely the stronger driver overall though).

      Although Raikkonen is no slouch, Ferrari’s position as having the best driver line up is largely thanks to Alonso – Ferrari has the biggest ppr disparity between teammates of all the teams, mainly because of Alonso’s extremely high ppr (in addition to Raikkonen underperforming). The Caterham drivers were the most closely matched teammates.

      Interestingly, the Mercedes teammates weren’t particularly closely matched this season, despite the common perception – they actually had one of the larger gaps as teammates (and this is using the higher 7.14 ppr figure for Rosberg – if 6.95 ppr had been used, the gap would have been larger than the one between the Williams drivers). They were certainly very closely matched over one lap, but Hamilton almost always had the advantage in the races. I feel like the perception that they were very close came from a couple of things:
      1) Their extremely close lap times over one lap (Practice and Qualifying), coupled with Hamilton’s frequent qualifying issues.
      2) The Mercedes being so dominant that, excluding reliability issues, you would finish at least second even if you put in a poor performance and were beaten by your teammate – meaning that the better driver could usually only pull out an advantage of 7 points per race (resulting in a close points battle).
      3) Due to the small points swing per race, reliability playing a much larger factor since it would usually take 4 race victories (net 28 point gain) to claw back the deficit from one retirement (25 point loss). Hamilton suffered much more unreliability in the first half of the season (2 retirements (potential 50 point loss) and two back-row starts (potential 20 point loss) to Nico’s 1 retirement (potential 25 loss) and ERS failure (7 point loss)), though this largely evened out over the course of the season.
      4) Hamilton’s lacklustre 2013 season (by his standards anyway: your model rates it as his second worst season, only above his meltdown in 2011) – it was his first season in a new team and he struggled with his car’s brakes, while Rosberg was in his fourth season at the team. The result was a close points battle (189-171 in Hamilton’s favour) despite Rosberg suffering more mechanical retirements, giving the impression that the two were very closely matched.

      This season Hamilton seemed much further ahead of Rosberg – 11 wins to 5 is enough of a statement, but noticeably, if you exclude Monaco/Spa (for obvious reasons) and races where one driver suffered a mechanical issue in the race or qualifying, then Rosberg only beat Hamilton on 2 occasions – Austria and Brazil (and both times he was helped by Hamilton spinning, and Hamilton still managed to finish within 2 seconds of him both times). Additionally, he never overtook Hamilton and on track and made it stick all season (unless Hamilton was suffering reliability issues – e.g. Australia start, Canada after brake failure etc.).
      To get an idea of how likely Rosberg is to beat Hamilton next year, I took the ppr values of Hamilton’s career performances each season from 2007-2013 from the graph your have in Alonso’s section (well, I estimated them to the best of my ability), and added them to a graph that also included Hamilton and Rosberg’s 2014 ppr values. This was the result: https://imagizer.imageshack.us/v2/579x250q90/r/537/yY31gk.png

      Interestingly, all of Hamilton’s season’s have been better than Rosberg’s 2014, apart from his 2011. This suggests that, unless Hamilton suffers another 2011-esque meltdown or gets particularly unlucky with reliability, Rosberg will need to up his game fairly significantly to beat Hamilton next year – even Hamilton’s second worst season (2013) is around 0.35 ppr higher than Rosberg’s 2014. If Hamilton continues performing at his current level (8.23 in 2014, 1.09 ppr higher than Rosberg’s 2014) or even improves next year, it will be very difficult for Rosberg to catch up to him – especially as Rosberg will be in his 10th season of F1, so huge improvements between seasons aren’t expected (although Hamilton did have a jump of around 0.7 ppr from 2013-2014 – that may be somewhat attributed to settling into the team and being comfortable with the car however). Although, it’s difficult to make a reliable judgement without seeing the model’s data of Rosberg’s previous career performances (that would be very interesting to see!) – I presume he has performed very well in the past, given his 7th ranking on your greatest of all time post based on peak years of 2010-2012, though I am not sure how reliable the comparison to Schumi is given his age + years away from the sport (evidenced by Rosberg falling to 15th if 2010 comparison with Schumi is excluded).

    2. Really nice work, thank you for putting that together!

  40. […] the model lays Sauber’s predicament this year squarely at its (pay) drivers’ feet. If you think that Magnussen, Perez, Maldonado and Ericsson have […]

  41. If all your predicaments are correct. How Vettel became a 4 times world champion? ….
    From 2007 to 2014 Hamilton won 2 championships was not Alonso.
    And if Alonso is the highest score, how’s that for 8 years has not won a single championship? ….
    These numbers are useless; only justify the mediocre result of a single driver!

    1. In Formula 1, the car is the most important performance factor. Typically, there are at most 2 teams with any realistic chance of challenging for the WDC. The best performing driver is therefore obviously not guaranteed to win the championship in any given year. The model is rating driver performances, as described in the article.

      1. Once again, your articles are far to “pro-Alonso”…there’s absolutely no way he was the best driver in 2014. And why no mention of the fact that Alonso would have benefited from being de-facto number 1 in his team? Sorry but your articles always seem to give Alonso far more credit than he is actually due.

  42. Gonzalo · · Reply

    This is a very interesting top. However, some interpretations are not realistic. For example:
    1.Magnussen P18: Magnussen has been overperformed by Button, but in some races, he has been much closer to his teammate Button.
    2.Pérez P17: Pérez isn’t so Bad to be behind Maldonado and Ericsson. He must be at the lowest part of the top 10 (P9)
    3.Raikonnen P8:Maybe Raikonnen has done amazing seasons, but 2014 was a very dissapointing year for him. He must be in P19.
    4.Bottas P4: The Williams Car is the second cand ahead by a diference of 0,4s/0,5s of Red Bull.
    He must be in P13. He must get more podiums last year.
    5.Hulkenberg: Only in P10? He must be much higher in the top. In a car, who would finish 11th (12th or 13th according to his Bad luck in lots of race and the weight penalization) he has finished 9th.
    Here are some examples of the bad luck situations of Hulkenberg:
    1.Bahrein: Massa forced out of him of the track. This dropped him to 3th to 4th.
    2.Bahrein: When the air cleaned, the SC apear out (caused by the collision Maldonado-Gutiérrez) so he had to defend of the Red Bulls, who had a faster car.
    3.Spain/Austria/Italy: He had some problems with the car.
    4.A startegy mistake in Canada (in the start) costed him the victory.

  43. Will you post the rankigs for the past seasons in the future? I think it would be very interesting if you make a post like this, but with the past rankings 🙂

    1. It’s in the works!

  44. Reblogged this on secreteyes4.

  45. “If Raikkonen beats Vettel, people will likely infer that Alonso >> Raikkonen > Vettel.”
    What if Vettel beats Raikkonen with the same or bigger margin compared to Alonso though?

    Also, what sort of ranking puts Vettel behind Raikkonen for 2014 performances?? lol

    1. If Vettel does beat Raikkonen by a similar or greater margin than Alonso did, I think it certainly would have a positive effect on his overall perception and ranking. I have no problem with the model’s relative rankings of Raikkonen and Alonso in 2014, however.

  46. […] be used to reexamine the results of each year in the sport’s history, much as I did for the 2014 season previously. Since the model estimates both driver and team performances, we can pose two hypothetical […]

  47. […] the Sauber drivers were rated the two worst performing drivers of 2014 by my model, Mercedes is still predicted to win both the 2014 drivers’ and constructors’ titles, […]

  48. […] year, I presented model-based rankings for all the 2014 season drivers. These rankings are derived from a mathematical model, described […]

  49. […] I did for 2014 and 2015, as well as other seasons historically, I applied my model of driver and team performance […]

  50. […] I did for 2014 and 2015, as well as other seasons historically, I applied my model of driver and team performance […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: