2015 model-based driver rankings

2015_driver_image

Last year, I presented model-based rankings for all the 2014 season drivers. These rankings are derived from a mathematical model, described here, that statistically estimates the strength of driver and team performances in each year, as well as overall career performances for drivers. It achieves this by finding the best fit to all race result data from 1950 to the present. As new races are completed, model rankings are continually updated.

Like all models, this one isn’t perfect, and I’m in the process of revising the model to include additional factors, such as driver age (more on age effects below). Nevertheless, it provides a unique and self-consistent way of ranking performances in a sport where it is notoriously difficult to compare drivers between different teams, and where we are usually left to bicker over the vagaries of subjective analysis.

2015 driver rankings

The metric the model uses to assess performance is the number of points scored per “counting race” across a season. Non-counting races include mechanical DNFs and technical disqualifications. While this is a fairly robust metric for assessing performance across a driver’s career, it can occasionally be unrepresentative in a single season if a driver is particularly unlucky (e.g., most of their best performances occurred during non-counting races). Such issues are discussed in the individual driver entries below.

Driver rankings are quantified by the adjusted points per race (ppr) up to a maximum of 10 ppr, which represents how the model thinks each driver would have performed in completely equal cars.

At the top of each driver entry, I have added a graphic showing the driver’s record relative to their teammate in points, races (excluding non-counting races such as mechanical DNFs, DNSs, and technical DSQs), and qualifying (excluding sessions where a driver was beaten because they set no time due to a mechanical failure).

The Toro Rosso Drivers

Before we get to the list, note that the two Toro Rosso drivers are unranked. This is because they have only driven alongside each other in Formula 1, so they cannot be compared to other drivers via teammate links. For the same reason, the 2015 Toro Rosso is unranked. If I assume the performance of the Toro Rosso car was on average halfway between Lotus and Sauber, the model inserts Sainz at 15th in the list of 21 drivers and Verstappen at 5th. This puts Verstappen in contention for the model’s rookie of the year title, alongside one other new driver, discussed below.

Sainz and Verstappen make for a fascinating comparison, given their different backgrounds. Sainz raced for five years in single-seaters, including one and a half seasons in Formula Renault 3.5. Verstappen spent only one year in single-seaters before jumping to Formula 1 — a new record for the shortest duration between first car race and Formula 1 debut. These junior career differences are summarized by my junior career metric. Sainz’s junior achievement score of 104 places him around the middle of the F1 field, but his excitement score (a measure of how quickly a driver progressed) of 12 is relatively low. Verstappen’s junior achievement score of 54 is low due to his brief career, but his excitement score of 27 is very high.

At the beginning of the season, Sainz and Verstappen seemed difficult to separate in qualifying and race sessions. Given Sainz’s prior experience in powerful cars, this seemed a bad omen for him. If Verstappen’s talent was sufficient to compensate for less experience, one would extrapolate an advantage once both drivers got to grips with Formula 1. So it proved, with Verstappen emerging as the more exciting of the two prospects in the latter half of the season, although Sainz’s progress was not helped by a string of mechanical failures. On a qualifying or race comparison across the season, the pair were tied. The big difference was in the points haul; when Verstappen was good he was utterly brilliant. Given this is still a developing battle, a second year together should give a more decisive picture of their relative abilities, although we will still remain limited in ranking them relative to the rest of the grid.

19. Pastor Maldonado, 4.35 ppr

2015 marked Maldonado’s fifth season in the sport, and his most underwhelming season so far. Last year, he often kept Grosjean honest, tying him 5-5 in races, but trailing 8-2 in points and 11-4 in qualifying. This year, the gap extended. Maldonado’s efforts were not helped by a string of hapless incidents, which some may consider karmic, but it was a disappointing performance however one views it.

In many ways, Maldonado’s career continues to resemble Vittorio Brambilla’s. Both could be quick on their day, but they share a violent, accident-prone style that undermines serious success. Both also share a single grand prix victory in their second Formula 1 season; Brambilla famously spun and crashed as he took the checkered flag.

Brambilla’s damaged car on his victory lap at Austria 1975.

Brambilla’s career dried up after his fifth season, with only sporadic drives thereafter. With the help of PDVSA money, Maldonado will start his sixth season in 2016. How long the funding can continue is unclear.

With Grosjean to be replaced by Palmer — a driver with a similar junior record to Maldonado — and a likely reversion to Renault engines, Lotus seem in very bad shape for 2016. If not for a desperate need to pay their bills in their short term, it is doubtful either Maldonado or Palmer would have acquired a seat. If FOM and Renault can resolve their differences, it will likely be a transitional year before a more committed works effort for 2017.

18. Will Stevens, 4.43 ppr

Early in his career, Stevens was managed by Brundle and Blundell, and even tipped as “the next Lewis Hamilton” by British press. His junior results since then have been underwhelming. He won only 5 races in his 7 years of single-seaters, never finished top three in any series, and was outqualified by teammate Kevin Magnussen by three-quarters of a second on average when they were rookies in Formula Renault 3.5. His junior achievement score is the lowest of any Formula 1 driver since Yuji Ide.

While Stevens’ results never merited consideration in Formula 1, he clearly played the other side of the game very well, maintaining connections within the sport and acquiring key sponsors for Manor to leverage a drive. For most of the season, Stevens was alongside Roberto Merhi, making by far the weakest driver line-up on the grid. While Merhi’s junior career was also weak by Formula 1 driver standards, Stevens’ early season advantage was surprising. Across the year he beat Merhi convincingly in qualifying, aided by a weight advantage of 10-15 kg. In races, the pair were closely matched. Rossi proved a more difficult opponent overall.

17. Nico Hulkenberg, 4.76 ppr

Testament to Hulkenberg’s enormous talent is the fact that he has lasted five (going on six) years in Formula 1’s midfield despite bringing virtually no sponsor money to his teams. Between his incredible junior career and his recent Le Mans victory, it seems Hulkenberg is unbeatable in anything other than a Formula 1 car. In Formula 1, the results continue to fall short of high but gradually lowering expectations, with 2015 the most disappointing year so far. For years now, Hulkenberg’s name has been raised in conversation whenever a top seat is available. His results against Perez may have finally ended that discussion. In 2015, Hulkenberg was consistently outmatched by his teammate, with the exception of qualifying. The overall tally for Hulkenberg-Perez is now almost equal: 154-137 in points, 16-16 in races, and 23-15 in qualifying.

16. Roberto Merhi, 4.91 ppr

With substantially less backing than Stevens, a spotty junior CV, and a poor start to the season, Merhi’s seat always seemed at risk. By the end of the year, he was able to almost level the tally with Stevens, and the model has ranked him slightly ahead of Stevens by virtue of his better finishing positions. I think it is doubtful that we will see Merhi in Formula 1 again.

15. Alexander Rossi, 5.17 ppr

Rossi’s debut at Manor pitted him against one of the weakest drivers in Formula 1, but with the added challenge of jumping into the seat mid-season. Failing to beat Stevens would have been career-ending, and Rossi performed very respectably in the circumstances. Rossi’s results now at least merit consideration for a full-season drive. Whether he will get one at Manor in 2016 remains to be seen, as driver selection at this end of the grid usually depends greatly on factors other than talent. For the seat he will be in competition with Manor’s development driver Jordan King, son of Manor’s 2015 chairman and former Sainsbury’s CEO Justin King.

14. Marcus Ericsson, 5.28 ppr

Ericsson was the lucky recipient of a race seat in the great Sauber driver lottery of 2015. Between them, Ericsson and Nasr are thought to bring around $40-50 million to the ailing Sauber team. Relative to most drivers with this level of financial backing, Nasr is very talented and Ericsson is about average. In his rookie year, Ericsson was beaten 5-2 in races and 10-4 in qualifying by the more experienced Kobayashi. This year, Ericsson was convincingly beaten by his rookie teammate. Both drivers will continue at Sauber into 2016, but Ericsson must do more to justify a longer term drive in Formula 1.

13. Felipe Massa, 5.75 ppr

Last year, I demonstrated that Massa and Bottas were much more closely matched than the points tally suggested. I also concluded that “a second year of Massa vs. Bottas will give us a better idea of where these two relatively stand.”  With an additional season of data, it now seems clear that Bottas’s advantage over Massa is persistent but small. The net advantage to Bottas is 322-255 in points, 17-16 in races, and 22-14 in qualifying. Overall, the model rated Massa’s season as solid, if unspectacular, and puts him very close to his teammate in the list. With Ferrari moving into touching distance of Mercedes and Red Bull receding, Williams were left in a relatively uncontested third. Between them, the Williams drivers finished in the range 4th to 6th a total of 19 times (10 times for Bottas and 9 for Massa).

12. Kimi Raikkonen, 5.79 ppr

Raikkonen’s 2014 struggles were often attributed to the F14T’s tendency to understeer on turn-in, contrary to Raikkonen’s preference for a pointy front end. After voicing great satisfaction with the F15T’s handling characteristics, Raikkonen delivered a similarly muted performance again. To put the result in perspective, Raikkonen’s points haul in the occasionally win-worthy and usually podium-worthy F15T was smaller than Alonso managed in last year’s disastrously uncompetitive F14T.

To be fair to Raikkonen, facing Alonso and Vettel is no easy task. The model thinks that most drivers in the midfield currently (including names suggested as Raikkonen replacements) would have been beaten by a similar margin across 2014-2015. However, one naturally expects more from a past champion who commands a champion’s salary. As the graph below of Raikkonen’s career performances illustrates, he has hit a nadir similar to his woeful 2008 season. The difference is that Raikkonen is now seemingly on a steady decline and he is 36 years old.

Since Ferrari have renewed Raikkonen’s contract for 2016, in 12 months we should be able to definitively conclude whether this is a terminal age-related decline in Raikkonen’s abilities, or just another temporary dip in performance.

11. Sergio Perez, 5.87 ppr

Perez’s career has followed an unusual path. After a semi-successful junior career, including 2nd in his second season of GP2, he joined the Ferrari Driver Academy, who found him a Sauber seat for 2011. In his first year he was outmatched by Kobayashi, but turned the tables in 2012. In May 2012 a bombshell dropped as Hamilton signed with Mercedes. McLaren were left needing a replacement driver, but were out of step with key contracts in the driver market. Their most advanced junior, Kevin Magnussen, was also not yet ready for Formula 1, following a quick but erratic rookie season in Formula Renault 3.5. McLaren was realistically left with a choice between Perez or the Force India drivers, di Resta and Hulkenberg. At that moment, Perez’s stock was high with three podiums to his name, while a clear number one at Force India was yet to emerge — di Resta dominated the early season before Hulkenberg turned things around.

Perez thus found himself with a golden opportunity to impress at a top team alongside a world champion teammate. Instead, we got a convincing demonstration that he was not quite at Button’s level, being beaten 11-5 in races. Unimpressed by Perez’s results and work ethic, and with Magnussen ready for promotion, McLaren dropped Perez like a hot potato. Having severed ties with Ferrari to join McLaren, this seemed to spell the end of Perez’s top-team ambitions. A year at Force India being beaten by Hulkenberg certainly did Perez no favors.

In 2015, Perez looked a new man, genuinely outperforming his teammate. Both Force India drivers’ careers are at a critical point now. If either driver can really impress in 2016, they have a small chance of getting back on Ferrari’s radar as they look for a driver to replace Raikkonen. If not, they could find it hard to sustain a seat into 2017. Driver potential is a valuable commodity in the Formula 1 driver market, and there’s usually little of it left after six years in the sport.

10. Romain Grosjean, 5.95 ppr

Last year, Grosjean asserted his superiority over Maldonado. This year, he straight up destroyed him. While Grosjean can be justifiably pleased by such a one-sided score-line in the teammate battle, successes for Lotus were limited. The season high-point was a thoroughly deserved podium at Belgium with the bailiffs at the door. Despite an obvious improvement in pace with the Mercedes powerunit, Lotus continually hemorrhaged points through a combination of poorly timed mechanical issues and crashes. For his part, Grosjean made errors at Japan and Russia, and was the victim of first lap collisions at Britain and Italy.

2016 will see Grosjean move to Haas. There he will have the opportunity to assert himself against Gutierrez, a teammate with a relatively weak record in Formula 1. It is an interesting move, with an eye to scoring Raikkonen’s Ferrari seat in 2017 — the only likely opening at a top team for an unaffiliated driver in the next two years. For that seat he will face competition from several drivers. If Ferrari want a new hot-shoe, Verstappen and Nasr are both appealing choices. If they want the best driver available, it is probably Ricciardo. However, Ferrari have traditionally run a clear number two driver, who is just quick enough to support the lead driver without taking many points from them. That, along with a presumed objection from Vettel, likely rules out Ricciardo. By the model’s estimation, Grosjean and Bottas are both excellent candidates for the number two role.

The Haas team’s performance in 2016 will test the viability of their unusual team structure. Their relatively small group of ~200 personnel is spread across three countries (USA, UK, Italy). Their projected team budget for next year is only $100 million, putting them just ahead of Manor, as shown in the graph below of estimated 2015 team budgets and the 2016 projected Haas budget.

Around 70 of the Haas team personnel were hired temporarily from Ferrari until November 1, 2015. Other teams viewed this is as Ferrari exploiting a loop-hole, which the FIA has now taken measures to close. Taking advantage of the fact that the Haas team is not subject to the usual restrictions on wind-tunnels and CFD simulations until they officially enter the sport next year, Ferrari’s loaned personnel were effectively free to sandbox aerodynamic concepts for 2016 and beyond, taking whatever knowledge they glean back to Ferrari. Interestingly, the Haas team chose not to convert their state-of-the-art “Windshear” rolling-road wind-tunnel from 100% scale to the FIA-mandated 60% scale. This suggests they may also be using their temporary freedom to run 100% Formula 1 chassis models, and perhaps even correlating the data with the 60% models at Maranello.

While Haas undoubtedly face a significant challenge next year, and are very unlikely to escape the lower midfield, their alliance with Ferrari should at least make them more competitive than the five last teams to join the sport: Caterham, Lotus (the other one), Virgin, HRT, and Marussia. Additionally, Lotus/Renault are facing a tough year with a step back to Renault engines, and continued financial woes. While Grosjean (or Vergne) would have been a sensational lead driver for a French works team, his gamble on Haas seems well motivated.

9. Valtteri Bottas, 6.14 ppr

Last year, Bottas appeared at 4th in the model’s season driver rankings. As I noted at the time, this was partly an artifact of Massa’s poor luck in 2014, as a more detailed analysis of the data suggested only a small performance difference. 2015 gave a fairer assessment of the pair. By the model’s reckoning the Williams was comfortably the third best car, and both Bottas and Massa performed about to par. The table below shows the predicted number of points each team would have scored in 2015 (excluding Toro Rosso) if they each had equally good drivers.

Based on this list, Ferrari (428 points) and McLaren (27 points) punched well above their weight, helped by Vettel and Button/Alonso, respectively. Force India (136 points), Lotus (78 points), and Sauber (36 points) are rated the greatest underperformers, due primarily to the points opportunities squandered by Hulkenberg, Maldonado, and Ericsson.

Alongside Grosjean, Bottas seems to have achieved a level slightly below the leading group of drivers, and slightly ahead of the Force India duo. Bottas is certainly good enough to win races if he lands in the right car, but the model doesn’t currently see him as a driver capable of dominating the next era. That will perhaps be left to Red Bull’s drivers and some of this year’s rookies.

8. Daniel Ricciardo, 6.72 ppr

Besides McLaren, Red Bull endured probably the most frustrating season of any team. All year they were plagued by reliability and power issues, with Renault’s late upgrade setting them even further behind. In the early season, they struggled to even outpace their low-budget sister team, suggesting their problems were not limited to the power unit.

In this difficult position, both Red Bull drivers were pushing hard to make up positions, and they often overstepped the mark. It was not the confident, inch-perfect racecraft we came to expect from Ricciardo in 2014. Overambitious moves contributed to or wholly caused collisions at Malaysia, Monaco, Hungary, Britain, and Japan. Kvyat meanwhile bumped into Perez in Austria, was penalized for exceeding track limits in Hungary, and had big shunts in Japan and the USA.

Across the season, Kvyat outscored Ricciardo by 3 points. In such a close match-up, the raw points tally can be misleading. Ricciardo’s mechanical failures at Silverstone (likely 5th), Belgium (likely 4th), and Russia (likely 4th) cost around 40 net points to Kvyat. Add a low points finish in Brazil (8th) to give 44 net points. Kvyat lost likely points places in Australia (7th) and China (10th), plus a likely 4th in Singapore due to a poorly timed Virtual Safety Car and bungled pit-stop, and a likely 7th in Abu Dhabi due to ERS problems. Paying back all these results to both drivers would result in an overall points tally of 130-105 to Ricciardo.

Alternatively, we can assess who should have finished ahead in each race without misfortunes in qualifying or races.

Overall, this analysis confirms that it was an extremely close match-up at Red Bull, but it definitely slightly favored Ricciardo. Although the model gives a slight edge to Kvyat, a more detailed assessment suggests their rankings for 2015 should really be swapped.

7. Nico Rosberg, 6.76 ppr

After close challenges to Hamilton in 2013 and 2014, Rosberg began the year as something of a spent force. After Singapore, he trailed Hamilton 12-1 in qualifying, having been a close match for Hamilton in qualifying previously, with an 18-18 record across 2013-2014.

In the final stages of the season, Rosberg seemed to drastically improve relative to Hamilton, beating him 5-0 in qualifying after Singapore. It is unclear whether this improvement was due to extrinsic factors, normal fluctuations in form, or the diminished pressure as the championship was effectively settled.

Overall, Hamilton now leads Rosberg 954-810 in points, 27-20 in races, and 30-25 in qualifying. If Mercedes remain the team to beat in 2016, Hamilton should remain the title favorite. But a continued good run of form for Rosberg, or a few unlucky DNFs for Hamilton, could easily tip the balance.

6. Daniil Kvyat, 6.81 ppr

Going into 2015, very few pundits were game to predict that Kvyat could outscore Ricciardo. While Kvyat’s rookie season against Vergne was extremely impressive, Ricciardo had just dismantled quadruple-champion Vettel, staking a strong claim to being among the Alonso-Hamilton-Vettel club. Looking back to my own comments last year, I was intentionally vague but gave Kvyat a fighting chance, “If Kvyat can achieve the kind of second-year progression many seem to be expecting of Magnussen, then Ricciardo had better watch out.”

The analysis in Ricciardo’s entry above confirms the subjective impression that Kvyat got a little lucky in outscoring his teammate across the season. Nevertheless, this was one of the closest teammate match-ups on the grid, with the advantage shifting between the drivers from one track to the next. On raw pace, Ricciardo’s advantage was clear, with an 11-7 result in qualifying and better pace in some of the races where he scuppered his own chances through mistakes.

Right now, Red Bull have probably the third strongest driver line-up on the grid, behind McLaren and Mercedes. They will not overlook a chance to strengthen that line-up and bring fresh junior talents into the queue behind. Ricciardo may not be losing sleep over Kvyat outscoring him, but 2016 will be a serious test of both drivers’ mettle, and I would not be surprised to see sparks flying between them, especially given the implied threat to their seats from Verstappen.

5. Felipe Nasr, 6.98 ppr

Nasr had a phenomenal rookie year that was largely overlooked in favor of Verstappen’s swashbuckling brilliance. As noted in the Toro Rosso entry, the model rates the season performances of Nasr and Verstappen very similarly, under the assumption that the Toro Rosso was slightly better than the Sauber. The cars were very different beasts. Toro Rosso were chronically down on power, but their James Key designed chassis was often credited as one of the year’s best, although its pointy front-end clearly took great feel to master. The Sauber had a strong power-unit but seemed to suffer from an underdeveloped chassis.

Nasr’s junior career was extremely impressive before GP2, including the British F3 championship ahead of Kevin Magnussen in only his third full year of single-seaters. From there he jumped straight to GP2, showing initial promise with 10th in the championship as a relatively inexperienced rookie. However, the next two years of GP2 showed slower progress than was anticipated, dampening enthusiasm for his Formula 1 chances. This year, Nasr has silenced the critics.

In assessing Nasr’s year it is essential to consider the fact that his teammate was not a rookie. Since the 2008-2009 testing restrictions, rookies have had consistently poor results against non-rookies (even relatively weak ones), as the table below shows.

Colors indicate superior (green) or inferior (red) results compared to the rookie’s teammate. *Kobayashi started two races in 2009.

While Ericsson is far from the strongest driver on the grid, Nasr had an impressive edge over him in races for most of the year. In stark contrast to the 2014 Sauber drivers, Nasr stepped up his game whenever points were on offer, scoring 75% of the team’s points. When compared to the results of rookies against non-rookies since the testing restrictions of 2008 and 2009, Nasr’s results are very strong.

Importantly, Nasr brings an estimated $24 million of support from Banco do Brasil. Between this and his strong driving performance, Nasr seems a very attractive option for teams in the upper midfield, and possibly Williams, where he was reserve driver in 2014. If he can continue his progress into 2016, a move up the grid in 2017 seems plausible.

4. Jenson Button, 7.39 ppr

Button emerges from 2015 with an impressive statistic to his name. He is the only driver to have outscored either Hamilton or Alonso across a season as teammates*. That fact probably provides only small relief for a season spent in a hopelessly uncompetitive car. Moreover, it’s not a statistic with great value, since there were very few weekends in 2015 where both McLaren drivers were free from technical issues. To really gain some insight into this driver match-up, it is imperative to do a detailed race-by-race analysis, as I did for the Red Bull drivers above.

Overall, Alonso was clearly the better performer, but the analysis also shows that Button gave Alonso a far stronger fight than any past teammates, besides Hamilton and perhaps Trulli. While the car may be a disaster, McLaren currently have what the model considers the strongest driver pairing on the grid. This creates a quandary for driver selection, because on the sidelines McLaren have Vandoorne, a driver with an incredible junior record who now holds the GP2 season records for most wins, most feature race wins, most points, and biggest percentage points lead.

With decreased earnings from prize money and sponsorship on the horizon, and a huge hill to climb in performance, McLaren have found it difficult to justify buying a seat for either Vandoorne or Magnussen in 2016. Consequently, they have lost Magnussen (another bright talent), and it would not be a huge surprise to see a frustrated Vandoorne leave if another big team asks for his services. Ferrari have potential options for placing juniors on the grid (e.g., via Haas or Sauber) and are currently rebuilding their junior academy in the wake of losing future-star Bianchi and failing to get the expected results from Marciello in GP2. The very talented Charles Leclerc is rumored to be one of their targets. Might they try to poach Vandoorne?

McLaren may be spoiled for choice with drivers at present, but their current champions do not have unlimited shelf-life. Button will be 36 in January. Alonso is 18 months younger, but commands a crucial role as the driver who could potentially drag the car a few extra places up the grid once it is remotely competitive. At what point will their skills begin to deteriorate? In other sports, it is typical for athletes to have a peak plateau around the ages of 25-35, but this varies with the demands of the sport. Reaction times decrease by age 30, whereas physical endurance performance stays near peak levels to about age 35. Playing some sports can take a physical or cognitive toll, such as contact sports with frequent injuries or concussions. In other sports, experience gained over time compensates for decreased physical performance. Below is an analysis, redrawn from “Bridging Different Eras in Sports” by Berry et al., and normalizing the data to place them on the same axes.

Ice hockey and baseball show a very similar age curve, with an increase in performance up to about 27, followed by a progressive slow decline. In golf, the peak occurs much later and has a more stable plateau. Formula 1 is an interesting case to compare, since it requires a combination of fast reflexes, decision-making, deep experience, and physical endurance.

I did an analysis of the effects of age on model-estimated driver performance using all the drivers from 1950-2015 in my database. I fit a linear model to average changes in performance with age for each individual driver. The average age effects are plotted below.

On the y-axis is the predicted difference in performance compared to the peak performance (at age 35). To help interpret the numerical values, a difference in performance of 0.3 is about the same as the average career difference between Hamilton and Bottas. The peak period appears to be relatively late on average, running from about 28 to 37, with rapid decline thereafter. These are average effects and individuals may progress very differently, but this at least gives some indication of when age-related decline begins for Formula 1 drivers.

*Alonso finished 2004 with more points than Trulli, but Trulli was ahead of Alonso 46-45 before he was sacked by Renault.

3. Lewis Hamilton, 7.91 ppr

2015 was a year of immense career success for Hamilton. He overtook Senna and Vettel to take 3rd in all-time grand prix wins, and he joined an exclusive group of ten drivers with at least three titles. Overall, the model considers 2015 Hamilton’s best year yet, narrowly ahead of 2014 and 2007, concurring with Hamilton’s own assessment that it was the best year of his career. Although the car was completely dominant — by end of season the W06 has climbed the list of most dominant cars in history from 7th to 3rd — Hamilton saw off the challenge from teammate Rosberg with impressive ease.

On his day, Hamilton looked untouchable and able to outpace Rosberg with consummate ease. But there is room for improvement too. Hamilton’s error-strewn race at Hungary, following a clumsy mistake at Silverstone, evoked memories of 2011. Spain and Monaco created a blot on the early season record; the former down to Rosberg’s better performance, the latter down to a combined bad call by team and driver that sabotaged Hamilton’s clearly superior pace. From Japan, the teammate battle also switched decisively in Rosberg’s favor.

The key question now is whether Mercedes can maintain their sizeable advantage into 2016. Two changes to the technical regulations next year have the potential to change the team hierarchy. The first is the mandated change to engine exhaust systems, which will effectively allow teams to make some changes without spending tokens. The second is allowing teams to choose between three elected tyre compounds at each race weekend, which could allow other teams to more easily find their ideal operating range. At a higher level, two more factors will influence the Mercedes vs. Ferrari battle. The first will be how cleverly Ferrari have used the Haas loop-hole to their advantage. The second will be how soon the teams shift their focus to development for the extensive changes in 2017.

 2. Fernando Alonso, 7.93 ppr

In 2015, McLaren-Honda set tragicomic records for grid penalties, while Alonso’s former team made significant inroads into Mercedes’ advantage. At this stage in his career, however, the only record Alonso claims to care about is another world title. Whether Ferrari or McLaren-Honda can overhaul Mercedes in Alonso’s few remaining years remains quite unclear, especially with a likely shuffle of the hierarchy in 2017 when cars will undergo their greatest dimension changes since 2009.

During 2015, Alonso put in many extraordinary drives, including the brilliant 5th place at Hungary. But by Alonso’s reckoning, it was not his strongest year.

“I did some good laps here and there. I remember Japan in Q2. The Austin race, I think it was good in difficult conditions, and I felt confident with the car and I could push. But apart from these two moments I don’t think that I had the best season, so definitely I need to improve for next year. When you are running at the front and you have more motivation that helps, of course. But I’m on standby, let’s say, in economy mode, to have full energy next year.”

The model agrees that 2015 saw a downturn in Alonso’s form, and this was especially the case in the latter stages of the season. Indeed, it is the first year since 2007 that the model has awarded the championship to a driver other than Alonso. The graph below shows the model’s year-by-year ratings of the current generation’s top three drivers.

For almost anyone else on the grid, Alonso’s 2015 would be a stunning performance, but his standards of performance are extremely lofty. A natural question to ask is whether this is a temporary dip in form or the first sign of age-related decline. A quadratic fit to Alonso’s yearly performance data suggests he is now beyond his absolute peak, but given a competitive car he is still more than capable of delivering the goods for a few more years. The question then is how quickly can McLaren-Honda deliver?

The rate of progress for different teams across 2014-2015 can be estimated by studying qualifying times at tracks that had dry sessions and the same tyre compounds in both years (Bahrain, Spain, Monaco, Canada, Austria, Hungary, Singapore, Japan, and Abu Dhabi). Below is the median percentage time difference from the 2015 Mercedes for each 2014 and 2015 team on their best timed lap in each session.

Marussia’s times are omitted from the graphic. They were slower at all comparison tracks in 2015 than in 2014, falling from 5.2% behind to 6.6% behind. This reflected their inability to develop a new aerodynamic package for the new nose regulations, as well as the loss of their strong lead driver, Jules Bianchi.

Ferrari achieved a gain of 1.10%, and a relative gain on Mercedes of 0.55%. Sauber also made a large step forwards, despite limited aerodynamic upgrades over the winter, confirming Ferrari’s huge power gains. On the other hand, Red Bull gained only 0.29% from 2014, with a relative loss to the Mercedes team of 0.36%. Red Bull were actually slower at 4 of the 9 comparison tracks in 2015 than in 2014. Lotus made a gain of 0.94% thanks in part to changing suppliers from Renault to Mercedes. Their relative gain compared to others would suggest the change of engine was worth around 0.5%.

Despite McLaren making major steps on their chassis and aerodynamics from the disastrous 2014 car, they lost a net 0.70% on their 2014 car’s pace. The obvious culprit is the Honda engine. More detailed analyses, based on GPS data, suggest around 70% of McLaren’s 2.7% time deficit to Mercedes stems from the power unit, which would account for 1.9% total. Even a gain of the same magnitude as Ferrari’s this year would optimistically put them around 5th in the constructors’ championship next year.

1. Sebastian Vettel, 8.30 ppr

2015 was in many ways a make or break year for Vettel’s reputation. While it is difficult to simply overlook Vettel’s four titles and the obvious quality of his driving in those years, the validity of his place among the all-time greats is a question that keeps coming up. It arises for three reasons.

  • Vettel’s junior career was fairly pedestrian by the standards of drivers who go on to win world drivers’ championships. This included losing the F3 Euro Series title to Paul di Resta in 2006, when both drivers were in their second full season in the series.
  • Vettel’s four titles all came in clearly the best car, and at least half of Formula 1 success is about being in the right place at the right time. By the model’s assessment, only 11 drivers in Formula 1 history have spent at least four seasons in the best car.
  • Webber was relatively old when he faced Vettel, and Vettel was soundly beaten by Ricciardo last year. To quote a recent statement by Hamilton, “I have a lot of respect for him, but it’s difficult to assess how good he really is. He’s never been in a team with someone like Fernando Alonso, but always with people like Mark Webber, who was not on his level, and Kimi Raikkonen, who is no longer at the peak of his performance.”

Solid counter-arguments exist for each of these points. Junior success is not always strongly predictive of Formula 1 success. Vettel has certainly had incredible cars at his disposal and relatively weak teammates, but the margin by which he beat Webber and Raikkonen is still informative. As for 2014, it’s still unclear just how good Ricciardo is, and large dips in driver form are unusual but not unheard of. The model currently rates Vettel’s 2014 dip the 13th largest mid-career dip in history, based on the change in a driver’s performance relative to the years immediately before and after.

Vettel’s 2015 performance should put most questions surrounding his quality to rest. With two notable exceptions (Bahrain and Mexico), Vettel looked mightily impressive as he occasionally took the fight to Mercedes and usually stayed comfortably out of reach of the Williams cars that hounded his embattled teammate. This season marks the first time that the model has awarded Vettel a drivers’ title, putting him in a group of just 22 drivers awarded titles by the model to date.

Since we will likely never see Alonso and Vettel as teammates, it is informative to see them against the same teammate in two consecutive years. Moreover, it’s an interesting hypothetical for many fans to wonder how 2015 would have looked, had Alonso continued his contract with Ferrari. On most statistics, Vettel and Alonso beat Raikkonen by staggering margins, but based on their expected points scoring rates the model still rates Alonso’s 2014 adjusted performance (8.79 ppr) significantly better than Vettel’s 2015 adjusted performance (8.30).

As another way of quantifying their relative performances, I studied the timing data. In qualifying, I calculated the percentage difference between the Ferrari drivers’ best times, only including times set in sessions where both drivers ran and had no mechanical issues. For each race, I calculated the percentage difference between the Ferrari drivers across all laps that both drivers completed, excluding lap 1 and safety car periods. I then calculated the median percentage differences between the Ferrari teammates across each season and the interquartile ranges. Note that it is important to use percentiles here instead of mean and standard deviation, because the sample is not normally distributed and has unrepresentative outliers that dominate the mean (e.g., the races at Monaco 2014 and USA 2015, where Raikkonen crashed and rejoined, losing lots of time).

The results of this analysis are graphed below.

Median (thick line) and interquartile range (box) of the time advantages held by Vettel (blue) and Alonso (red) over Raikkonen (dashed line at zero) during their years as teammates.

From this we can conclude that both Vettel and Alonso had enormous advantages over Raikkonen, being quicker than him well over 75% of the time. Their advantages tended to be slightly larger in qualifying than in races, which is consistent with Raikkonen’s reputation as a better racer than qualifier. The analysis also shows a large relative time advantage to Alonso over Vettel, if we take Raikkonen’s performance level to be relatively consistent across these two consecutive years.

On this basis we should conclude that the Alonso of 2014 could have gotten slightly closer to Mercedes, but there was never any beating such a dominant team. Moreover, the model’s assessment of 2015 driver performances implies nobody would have done a better job this season in Vettel’s position. He was the star of the season. If Ferrari can continue their rate of relative progress into 2016 (and it’s a big if), they should be capable of taking more regular race wins. At that point, you could not count Vettel out from a fifth championship.

The 2016 form guide

Driver performances can be hard to predict in advance, due to fluctuations from one year to the next. Vettel’s return to form from 2014 to 2015 is one obvious example. Nonetheless, we can look at the average performance of each driver over the past 3 years to get a general sense of the driver hierarchy at the present moment. I performed that calculation for all drivers who were active for at least 2 of the last 3 seasons, with the hierarchy graphed below.

As you would expect, the ordering is similar to the 2014 and 2015 driver rankings. A notable absentee from the 2016 grid is Vergne, who is currently ranked 5th in the hierarchy, just behind Ricciardo and just ahead of Rosberg, Button, and Kvyat. Raikkonen is beginning to plummet into the pack, after being ranked the 5th best performer of 2012 and the 4th best performer of 2013. With another year like 2015, he will likely sink behind Grosjean and Bottas, both hungry for his Ferrari seat. Perez has now just inched ahead of Hulkenberg. Meanwhile, Gutierrez has the lowest rank of the current group, not helped by one of his two assessment years being a rookie year. He will get another chance to redeem himself in 2016 against a good benchmark teammate in Grosjean.

While the focus here is on drivers, the sport’s entertainment level in 2016 will be largely dependent on the team hierarchy. Can McLaren-Honda deliver Alonso and Button a competitive car? Can Ferrari turn the championship into a two-horse race? Can Red Bull solve their engine woes and get on terms with Williams? And can Toro Rosso become a serious force once their excellent chassis is allied with Ferrari power? These are questions we will revisit next year.

Advertisements

68 comments

  1. Wow thanks for this! Been waiting since Abu Dhabi! It’s really interesting to see how all the drivers have done in comparison to each other.
    I was quite surprised though by some of the results. Firstly I didn’t expect Ericsson to be above Hulkenberg. I mean Ericsson did terribly and Hulkenburg did… well decent (even if he is becoming a bit of a Maldonado).
    Also when it got to the midfielders I was suddenly expecting them all to be ranked higher (not realising how many relatively good drivers there are.)
    I expected this to be Hamilton’s year as well, being that he dominated like last year and Alonso drove less good. But I have to say, I am glad my current Favourite driver has finally won a championship on your model. Well done Vettel!
    Couple of questions. What would the drivers standings actually look like at the end of all 20 races? I mean I know the order now but how big would the margins be?
    Also how did you work out the Manor drivers? They’re all rookies?
    Finally if Ricciardo had better performance overall than Kyvat how did Kyvat finish ahead? Was it just down to the points difference?

    1. Andrew M · · Reply

      Re the Manor drivers, Stevens had a few races against Kobayashi last year, so that’s how he’s correlated with the rest of the field (I assume)

    2. Here are the predicted points tallies in equal equipment, excluding Toro Rosso drivers.

      1. Vettel 184
      2. Alonso 165
      3. Hamilton 163
      4. Button 140
      5. Nasr 124
      6. Kvyat 117
      7. Rosberg 116
      8. Ricciardo 114
      9. Bottas 95
      10. Grosjean 90
      11. Perez 87
      12. Raikkonen 85
      13. Massa 84
      14. Ericsson 71
      15. Rossi 68
      16. Merhi 61
      17. Hulkenberg 57
      18. Stevens 50
      19. Maldonado 48

      The Manor drivers are very loosely connected to the rest of the grid via Stevens’ single race last year. Their ratings should be taken with low confidence.

      Yes, Kvyat had slightly more points per counting race, but very close to equal.

  2. Andrew M · · Reply

    Interesting analysis as always. Alonso’s ranking reinforces my suspicion that he’s overrated by the model a bit due to the favourable treatment he received at Renault/Ferrari, I don’t think it’s a coincidence that his performance level is a lot lower in his two years at McLaren where he had competitive teammates. Then of course if we’re playing that game, Vettel is probably overrated because Raikkonen is no longer near his peak, while Hamilton is overrated because he beat Rosberg, who himself is overrated through beating Schumacher, so who knows!

  3. What was the actual raw data for model points, between Alonso and Button? Alonso’s 5th in Hungary nets 2 pts in your model, while Button’s 6th nets him 1 pt. So a 100% difference in your model, versus a 25% difference in the current points scoring system. I am surprised that Alonso finished ahead of Button this season, and I can only put it down to the extra counting races, along with the big discrepancy in model points for their best results, which both required faster cars/drivers to have trouble or bad races. What does a 12-14 for ALO in CHN or 11-16 in JPN prove, in terms of relative driver ability?

    To prove the point, imagine if Lewis and Nico finished 1-2 in every race, and Alonso-Button finished 5-6 in every race. Your model will give a much bigger benefit to Alonso over Button (100% better) than it would to Hamilton (66.7% better). The main fact though would be that Alonso and Hamilton would’ve beaten their teammates every time, though Hamilton’s would’ve been for wins instead of for 5th’s.

    As for Hamilton, what would the rankings be if the season ended after Austin? Three 6’s to close out the season (when the title race was done) no doubt brought his PPCR down, while it brought Rosberg’s up. The only time Hamilton was responsible for finishing outside of the top 2 was Hungary, which one could also say was set in motion by a bad start, at the last race for assisted starts. Since the advent of driver-centric starts, Hamilton hasn’t lost a spot off the line.

    Last question: regarding your 3 yr peak, is Hamilton’s still 2007-09, or is it now 2013-15? Might it be better if you did best 3 seasons of any 4 year period? Perhaps Vettel’s 2012-15 (excluding 2014) would be his peak period?

    It’s an interesting model, and I appreciate your work on it.

    1. You’re correct that the model uses a system based on 10-6-4-… that extends to all positions (i.e., points beyond 10th place). Checking the raw data, these are the points:

      Alonso: 2.68

      Button: 2.21

      Alonso also had fewer counting races (11 vs. 14).

      You’ve actually just alerted me to the fact that the ppr values listed next to the McLaren drivers are from an earlier version and slightly off. This doesn’t change the list order, but I’ll correct the values now.

      Also understand that the performance variable in the model does NOT depend linearly on points scored, so it doesn’t work quite the way you’re assuming in terms of percentage differences in points. The model recognizes that there is potentially a ceiling effect in terms of scoring for drivers in very strong cars.

      Oh, and the question about Hamilton that I missed. His peak three-year is still 2007-2009, but 2013-2015 is very close, and 2014-2015 is now the two-year peak. He may have a new three-year peak interval next year.

      1. Ok, so even before any regression, the raw data gives Alonso a sizeable lead, even if you used all races as the denominator. 2.68/2.21 = 1.213 … a 21.3% advantage to Alonso. Of course if you used the current points system, it would be an advantage to Button, that would largely mirror the 16/11 that they scored.

        2.68/11=.2436364
        2.21/14=.1578571

        .2436364/.1578571 yields an even bigger advantage to Alonso of 54.3%.

        Could anyone honestly say that Alonso has been 54.3% better than Button this year?!?? No way. At best it was a saw-off.

        I think that you might need to reassess whether your basic units of measurement are providing the data validity that your whole model depends on to work.

        Here’s something to test. If ALO retired the car in ABU as he wanted to, would he take the top spot in 2015? I bet he would.

        Also, what would the rankings say without the last 3 races?

      2. Again, performance is NOT a linear function of points. You have a fundamental misconception here that you need to address. It doesn’t make sense to say Alonso has been 54.3% better. That’s not how a scoring rate relates to performance at all. The model concludes (rightly, I believe) that the performance advantage to Alonso this year over Button was fairly small. That Alonso should be ahead of his teammate seems undeniable given the analysis I undertook of their race results.

        Alonso would certainly not be top spot with one more mechanical DNF. I ran your suggestion and he remains 2nd with an increase in his adjusted scoring rate from 7.93 ppr to 7.97 ppr. Vettel stays at 8.30 ppr.

        I was running rankings all year, and nothing much changed in the driver list in the last 3 races. Vettel was leading the championship from at least midseason. Hamilton and Alonso flipped places a few times, as their very close ratings would suggest.

      3. I get that it’s not linear. However, at some point you are putting in .2436364 for Alonso this year, and .1578571 for Button. That’s the “premise” of their head-to-head in your equation(s). Given that this is their first year together, it would also have knock-on effects for other past driver pairings involving Alonso and Button. The primary building block of the model is flawed in my opinion, and needs reevaluation.

        Try this: plug in equal PPCR values for Alonso and Button for this year, and see what that does. Or take away 1 best and 1 worst CR result, to get a more accurate view of what position a driver usually brings a particular car home to. Perhaps do an all-results PPCR, then a Best13/19 (over two-thirds of a season’s races, as used for most F1 seasons), and then weight them both 50%.

        I can see now how and why the model inflates Alonso’s rating. Don’t get me wrong, he’s without a doubt one of the best on the grid. I just don’t think he’s had a particularly good year, and that’s accounting for the car and the myriad of reliability issues. 4-4 in two-car finishes, and the close qualifying battle between him and Button. That last stat is quite surprising, because Button’s never been strong in qualifying.

        GrandPrixRatings has Alonso/Button as 171/169. That makes sense.

      4. You are greatly overestimating the knock-on effects, as exemplified by the Abu Dhabi simulation I gave you. 2015 as a whole had almost no effect on Button’s or Alonso’s overall ratings in the model. There’s no “inflation” going on, and you’re inaccurately representing Button’s performance this year when he was clearly outperformed on any detailed analysis. Unfortunately, you still seem to be under some misconceptions about the model here too, and not really appreciating the effects of the nonlinearities correctly given all of your statements, but I think I’ve spent enough time on responding to this particular flawed line of reasoning now.

      5. As a non-applied mathematician 😃, of course I don’t fully get how all the moving parts work. I don’t need to. What I’m saying is that it’s obvious that the points system you’re using is not aiding your efforts at a representative ranking. It’s a case of GIGO. Like wearing a $1000 suit, then putting on white athletic socks with it. You’ve got the hard stuff right, but the simple stuff wrong.

      6. I completely and fundamentally disagree. To outline my points again:

        * The points system used in the model is clearly superior to a system that ignores what happens beyond 10th place. That’s easy to demonstrate, and the Alonso-Button case actually supports that claim very strongly.

        * As noted in my blog, I have run the model with other points systems too, including an extended version of the 25-18-15-… system. It doesn’t alter the main conclusions at all. I think you’re still stuck on the idea that performance scales approximately linearly with scoring, even if you’re saying otherwise. I just demonstrated that a 10% increase in Alonso’s raw ppr for 2015 changes his adjusted ppr by only 0.5%. The results simply aren’t highly sensitive to the points system used in most cases.

        * I’ve just linked you to a model that uses no points system at all, only a linear model of relative race positions. It comes to very similar conclusions regarding today’s top drivers, including ranking Alonso far ahead of anyone else.

      7. Ah – you mean Grand Prix Rankings, KRB; I was looking for an update on 2014 and 2015 on Grand Prix Ratings! I wouldn’t put much score by the Rankings – it has Stevens above Alonso and Button, based on the reliability of the McLaren-Honda. It doesn’t detach car from driver at all.

      8. “2015 as a whole had almost no effect on Button’s or Alonso’s overall ratings in the model.”

        Is this a roundabout way of saying that Alonso starts any given season as the top driver in the yearly rankings, simply owing to bossing Fisichella and Massa for many previous seasons? That he basically needs to blow it badly, in any given season, just because the dead hand of prior F1 history has him ensconced at or near the top, straight out of the gate?

        F1 Fanatic’s Keith Collantine ranked Button ahead of Alonso in 2015, on a race-by-race review. I don’t believe it’s beyond reason to give the nod to Button this year.

        Secondly, I don’t get how you arrive at your “full season” figures for junior categories. You have Rosberg as having 5 full junior seasons. Really he had 4 (2002 BMW ADAC, 2003 Euro F3, 2004 Euro F3, 2005 GP2). If you count 3 races in 2001 for the BMW Jr Cup Iberia, then that’s a problem.

        Same with Hamilton. He didn’t have 6 seasons, but 5 (2002 Formula Renault 2000 UK, 2003 Formula Renault 2.0 UK, 2004 & 2005 in Euro F3, and 2006 GP2).

        Rosberg took 2 titles in his 4 junior seasons, while Hamilton won 3 titles in his 5 junior seasons. Both won in the F1 feeder series (GP2, F3000 before) in their inaugural season).

      9. Obviously past performances have an influence on assessment of present performances, as they should do in any model, but individual year parameters are independent. So no, I do not agree with your statement.

        Collantine’s totally subjective driver rankings are baffling this year, since the race-by-race record and timing data (for anyone who studied those) clearly favored Alonso in 2015. Collantine’s notes certainly don’t alter that assessment.

        What you count as a “full season” for a junior career is in some cases debatable. You’re right that I should probably exclude 2001 for Rosberg, and possibly 2001 for Hamilton.

  4. It would be really helpful if you touched a little bit more on how this season has affects previous seasons. The only comment I’ve seen so far about it is in the reddit thread: “Regarding shifting rankings, it’s difficult for me to exactly give you causality, since everything is tied to everything else. One thing I have noticed is that Massa’s rating has gone down a bit, which may have thus impacted Alonso. That could be partly due to Raikkonen’s decline, but it all gets a bit circular to figure out!”
    Considering the only variables to the question are from this season alone, we can point to notable over and under performances across the season and vs expectations. Maldonaldo’s and Raikkonen’s poor form may have a larger impact on the Williams drivers because of the amount of weight those teammate comparisons have (Bottas has 1/3 of his career with MAL, Massa Vs RAI 3.5/12.5).

    When you cited Alonso’s 2014 scoring rate when comparing him to Vettel, you used the scoring rate from your 2014 driver rankings. The updated one as shown in the graph in Alonso’s section appears to have the updated value that is around 8.80. I know it is a circular question, but how has much of the grid changed in either overall performance or how last season’s performance is rated?

    1. Thank you for the correction regarding Alonso. That was indeed an oversight.

      Regarding the all-time rankings, I’d like to discuss that in more detail in a future post. I’m working on an update to the model.

  5. Another great article! To my surprise it seems that the driver values were generally lower in 2015 than in 2014. No driver had a really outstanding season, Vettel had a couple of bad races, Hamilton got bored towards the end of the season, while Alonso and Button had too many reliability issues to be accurately ranked (I think, however, that Button was a little better than Alonso; for example, he was ahead in Monaco and managed to make his supersoft tires last longer than Alonso’s soft tires. By the way, Alonso retired the Spanish GP with a brake failure.) Still I’m a bit surprised by the low rankings of Hülkenberg, Pérez, Ricciardo and Bottas. It seems that Bottas and Ricciardo were a bit unlucky not to outscore their teammates, while Pérez was quite impressive after the summer (do the results change if you consider the B-spec Force India as a different car?). I think that the rankings are fair in general, I really like the age-performance graph and I think the relative performance analysis is very well done.

    1. Regarding the apparent downscaling in ratings this year, it’s important to note that the ppr value has to be calculated in a particular reference car. I arbitrarily chose the 2013 Red Bull when I defined the model, so changes in its relative competitiveness will rescale the ppr values, which is mostly what you are seeing here. You may have noticed this if you compared the graphs of driver performances over multiple years from last year to this year. Most of them have shifted down by about the same amount. Also, thank you for the error you caught. I’ll correct that now.

      1. Would it be possible to change the reference car? Maybe update it to the 2014 or 2015 Mercedes? Or even the 1961 Ferrari (most dominant car ever)? I felt the Sauber/Mercedes driver swap analogy helped with perception of understanding PPR results.

        The other problem is mid-year changes that affected performances in all three years – 2013, tyre make up; 2014, too low a weight limit; 2015, tyre pressures. Hopefully 2016 will be stable before the new rule set and thus make a better benchmark going forwards.

      2. I could update the reference car each year, but either way the adjusted ppr values will all be rescaled, so I’m not sure it improves anything. The important thing is that everyone is measured on the same scale, which any reference car will do.

  6. Again, an over accentuation put on Alonso in particular. He has perpetually been overrated by these models and conversely, Hamilton perpetually underrated. I do once again, question the impartiality of the author, particularly when it comes to Alonso, and by extension, Hamilton.

    1. I don’t have a horse in that race, but I will report the model predictions. It’s worth noting another F1 driver ranking model that was recently devised, with quite different underlying assumptions, concludes very similar results for Alonso vs. Hamilton. Ultimately they are only “overrated” or “underrated” relative to your subjective perception.

      1. Which other model is this? It’s a hard area to model, because you only have two points of reference in any given year with which to evaluate a car, and then from there relative performance. 3-car teams would likely help a lot with this.

      2. There is a general feeling that the model does seem to overplay Alonso.

        Leaving the actual model aside for one moment, I just feel that when it comes to Hamilton in particular, there seems to be more emphasis on the negatives. Let’s just take a look at the driver descriptions. What i see, with Hamilton in particular, more tends to be written about the negatives. Small example. You mention Monaco being a bit of a blot, with a joint bad call from team and driver. Yet strangely, no mention of Monza with Vettel. That decision to try the one stopper was also with Vettel’s input? Clumsy mistake at Silverstone which evokes memories of 2011 for Hamilton? That comment was not reflective of what happened and references to 2011 was a little below the belt. Yet absolutely no mention of Vettel’s poor qualifying at Silverstone,coupled with a poor start in the race that saw Vettel initially drop down the order?. Australia, Vettel got a poor start and there are reports than he touched his team mate yet the author sees fit to only mention Bahrain & Mexico for Vettel? It’s little things such as this where i feel that there is a bit of a built in bias against Hamilton, from the author. The author will go through Hamilton’s races, even if they were generally good (such as Silverstone) and then search for and magnify any small error. Sorry to say, but for me, there is a feeling of “nit-picking” by the author when it comes to Hamilton.

      3. I think you’re reading into this what you want to read. The report is not intended to be exhaustive. I don’t share your sentiments, nor do I have any particular bias against Hamilton. I’ve personally rated him one of the great talents of his generation since his debut. On a personal level I previously disliked Alonso and now feel fairly neutral towards him. In 2007, I was cheering for Hamilton and Raikkonen. And even if I did harbor any of the biases you are imagining, it would be irrelevant to the model rankings, over which I have no control. I personally disagree with the model on many points, some of which I have noted.

        Moreover, I just linked you to another model, by completely independent authors, that is apparently also biased towards Alonso. You can even go further back to the model of Stadelmann (http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1017292), which generated very similar conclusions. Alonso was 6th in their all-time list. All three models, with very different premises, give a clear consensus. It’s the fans with particular agendas who seem to be most often at odds with the actual data.

      4. Just one further thing to add. I do agree that Vettel had a really good year and i personally, would agree with a top 2 rating for him. The model places great emphasis on performance relative to team mate. What the model fails to recognise is that at Ferrari, the team was unquestionably built around Vettel, with Raikkonen clearly there as a supporting role. Contrast the situation at Mercedes. Complete driver equality. In other words, the situation at Ferrari “facilitated” Vettel in being able to draw a large gap between him and Vettel. Hamilton, on the other hand, did not have such a luxury and the model fails to recognise this.

        Tying error
        Should have said:

        “In other words, the situation at Ferrari “facilitated” Vettel in being able to draw a large gap between himself and Raikkonon”

        This is perhaps why rankings by say the team bosses are a little more reflective. Not only are they “privy” to all kinds of relevant inside data, they also are able to take things in to account which a model, based purely on stats, can’t.

      5. This issue has been discussed to death elsewhere on the blog. Please take the time to read it if you want to add something novel to the conversation.

      6. Fascinating discussion, but as f1metrics said, how do you quantify “team preference” etc.

        The next step, if we are going down a path of non-quantifiables, is to look at the areas outside of F1 that could impact a driver’s performance. I recall earlier this year (or it may have been last year) that Massa said that being the number 2 driver at Ferrari cost him maybe a tenth or two a lap. He said it was difficult to put in a full 100% effort each and every time (closer to 98/99%).

        So if we look at something like Vettel’s performance in 2014, we’d need to look at many elements that could have kept him at less than 100% – going from a car that could race for wins and the WDC to one where 4th or 5th was the best they could hope for on the day, combined with a myriad of technical issues both in preseason and in the early races; the birth of his first daughter; MSC’s injury and coma; the natural decompression from winning 4 WDCs on the bounce (both Mika Hakkinnen and Jackie Stewart talk about the strain that comes with winning the WDC, and Lewis this year talked of wishing for a vacation after he won in Austin); and an experienced teammate who was incredibly hungry for any opportunity to shine (people keep saying Ricciardo was Vettel’s rookie teammate, but he wasn’t, he had 2 years as a test driver with STR, 11 races in 2011 with HRT and another 2 seasons with STR before joining RB).

        It’s not just Vettel, Hamilton had a poor 2011 due to off track distractions, and has admitted at times he’s not “always there”.

        How do we quantify those? If we go with quantifying team preference (which would be a made up number since we don’t know how much one driver is being favored over the other if at all). Things like preferential treatment with strategy usually goes to the driver in front. And it can sometimes backfire on them as well – Lewis had the first pit stop in Austin but it was lap too early and Vettel in Malaysia 2013 also came in one lap too early for slicks as the track was trying up. In 2013, the team actually ordered Vettel to finish behind Webber, which goes against the team preference idea.

        I know it was also mentioned about micro-mistakes in the race and these should be accounted for. But how do you do that? If you look at Hungary, Hamilton had 2 or 3 huge mistakes, plus a poor start yet finished ahead of his teammate who had a decent start and made one marginal mistake (and could just as easily be entirely on Ricciardo), yet the standings in the race don’t reflect that. Vettel in Bahrain actually made one small mistake – running wide out of the last turn – who would have thought coming back on the track would damage his wing?

        Ultimately mistakes even themselves out over time by where the driver ends up at the end of the race.

    2. It may have been discussed to death, but nothing in the model has been done to try and rectify it. Considering the most important aspect of the model ranking is how a driver performs relative to their team-mate, that is one glaring hole that grossly inflates certain driver’s performances (e.g. Vettel ) and unfairly detracts from others (e.g. Hamilton and Raikkonen). It’s a massive failing of this model.

      Lastly, you mentioned that the question mark over Vettel following 2014 has been answered. I agree, to a large extent it has. But then, by your research, you state reaction times (which is perhaps more crucial in F1 than any other sport) decline after 30. You state physical endurance performance starts to decline around 35 ( this is important as studies have equated F1 drivers fitness levels needing to be akin to astronauts to withstand the G-forces etc). So not only has Vettel been competing against a Raikkonen, that according to these two measures, is past his peak, but he has also been competing with, rather than against, a Raikkonen who is there, merely as a compliant, supporting number 2? When you consider all this, perhaps the question marks surrounding Vettel since 2014 have not entirely been answered.

      1. Neither the data nor common sense support the claims you are making here regarding the effects of imbalanced treatment of teammates on relative performance in the modern era. I’ll leave it at that.

      2. Common sense? Well we do have other posters also voicing concerns about this aspect, not only on your previous blogs but on here too. Maybe they all lack common sense as well……

        Andrew above has intimated the same type of concern regarding Alonso – his quote “Alonso’s ranking reinforces my suspicion that he’s overrated by the model a bit due to the favourable treatment he received at Renault/Ferrari, I don’t think it’s a coincidence that his performance level is a lot lower in his two years at McLaren where he had competitive teammates”……..

        It’s much easier to enhance performance and build a larger gap between a driver and a driver’s team mate in these circumstances of preferential treatment. Much harder when you are having to compete with your team-mate on an equal footing. Your model fails big time on this particular aspect..

      3. The negative comments I see on my articles routinely come from the fanbases of drivers who feel their favorite was too lowly rated. That is transparently the case here, and I have very little time for it.

        Earlier there were claims that the 2015 Ferrari was designed around Vettel. Common sense is apparently in short supply, given the F15T was mostly designed before Alonso left the team. Engineers don’t design cars to specific drivers’ tastes anyway. They design the fastest possible machine within the technical regulations. Matters of driver taste are a very secondary concern, and opportunities for driver feedback are extremely limited today. The main input of the driver is set-up, and that’s available equally to all.

        Top teams have provided drivers with completely equal mechanical equipment in almost all races for 20+ years. If you want to talk about team preference having a meaningful effect, you need to go back to the 1950s-1980s when drivers sometimes had unequal equipment. It’s not a valid excuse for poor driver performance anymore. Since the 1990s, team preference has usually only amounted to team orders when needed for the championship. In cases like Schumacher-Barrichello, Alonso-Massa, and Alonso-Raikkonen, the performance difference between the drivers was so large for that to rarely be needed. Points gained by Schumacher and Alonso by those means sum to very little compared to their overall teammate advantage.

        You have cherry-picked Alonso as an example based on poor performance in what you think are the only two years where he didn’t have preferential treatment (debatable point), but it’s trivial to provide counterexamples. Look at Senna’s low ranking in the overall list. Here is a driver who benefited tremendously from team preference and even unequal equipment for most of his career. Yet many readers of my blog feel he is significantly underrated by the model.

        Finally, there’s the issue of how you would attempt to mathematically implement such a factor in a model, which might be useful in assessing performances in the earlier eras of F1. I have heard literally no useful suggestions from readers on this point.

      4. The car does not have to be built around a specific driver. What i said was is that the “team” can be built around a specific driver, which is an entirely different thing….the “main” driver may get priority on strategy, first choice on updates etc—there are lots of ways of receiving “preferential” treatment other than having a car specifically built around/for a driver!

      5. I think as well it’s one of the skills of an F1 drivers to integrate well with the team, and getting them onto his side. If some of Vettel’s advantage did come to the team being “built around him” then that’s partially down to him being relatively easy to work with. And I’d kind of say that in a way is one of the reasons Alonso has only won 2 world championships. He may be the best in the sport, but his lack of patience and falling out with the team always ends up getting the better of him. So yes Vettel may not be as fast as Alonso but the fact that he’s able to work with the team well probably elevates his considered F1 ability. And I wouldn’t say that’s particularly a flaw in the model as F1 is often just as much about the excitement off the track as it is on the track.

      6. Firstly, not everyone considers Alonso to be the best. Certainly not the current Team Principals. But i take on board your point regarding being able to work well within a team as being a skill. It is indeed. But then i question if this “team-spirit” largely manifests itself when team-mates are not equally competitive. Take Vettel for example.Currently working in tandem and in a harmonious way at Ferrari yet at Red Bull, where Webber was at least initially more competitive, all hell broke lose! Maybe it’s easier to play the team-player when you know your team mates’s main role is a mere supporting act?

      7. As it’s my comment being talked about (fame at last!), I in no way cherry-picked Alonso as an example, I specifically mentioned how Vettel and Hamilton could also be overrated! But with regards to the comparative rankings of these three drivers, surely you can appreciate why people put a lot more store in their performances where they are a lot more closely correlated (e.g. Hamilton and Alonso in the same car, Alonso and Vettel’s comparative performances vs Raikkonen a year apart) as opposed to weakly correlated (e.g. Alonso crushing Piquet/Massa).

        I believe you when you say you have no horse in a particular race, and certainly don’t think that you are biased towards Alonso in any way or have created the model to be so; I’m just saying that Alonso being overrated compared to other current drivers is something I think the model with its current assumptions calculates, in the same way I think it over/underrates other drivers, some of which you agree with in your narrative. He’s certainly not the only one, but he’s arguably the most relevant and emotive example because he’s still racing.

        And disclaimer, I’m a Hamilton fan, so I do have a vested interest! I like to think I can keep my biases in check though 🙂

  7. Ok, so congratulations once again for the article, awesome piece of detail!

    I’d like to ask, could you make a similar article for previous season, please? I could only find this one and 2014’s ratings…

  8. Thanks you! Nice to see Alonso back on the podium. 😉

    Rather annoying that the Torro Rosso guys don’t have a proper rating (and won’t next year too), but that’s the way it rolls.

  9. >”Vettel’s junior career was fairly pedestrian by the standards of drivers who go on to win world drivers’ championships.”

    It really wasn’t. You should check out the junior career (such as it was) of a driver you rate as the best on the grid, Fernando Alonso. It was much less than pedestrian and included coming fourth in Formula 3000 behind a certain Mark Webber. Vettel was strolling to the title in F3.5 in 2007 when he was called up to F1.

    >”Vettel’s four titles all came in clearly the best car”

    They really didn’t. The 2012 RB8 was clearly not “clearly the best car” on the grid that year. There’s an argument to be made it was the best car, but it certainly was far from being “clearly the best”.

    >”Vettel has certainly had incredible cars at his disposal”

    No, actually, he has not. The best of his title-winning cars (2011, 2013) were certainly very good, but even they were not in the top ten of all time F1 cars. The 2010 car was average by the standards of F1 title winning cars. In the 58 years during which there has been a constructors championship, only six times has it been won by a car with a poorer winning percentage than the 2012 RB. You may as well try to argue that the 1985 McLaren or 1977 Ferrari were “incredible cars”.

    1. That is a rather bizarre interpretation of Alonso’s junior career. His International F3000 season was in his second year of single-seaters. Try to imagine a driver stepping into GP2 (the International F3000 equivalent) in only their second year in cars and finishing 4th and then saying it’s a rather pedestrian result. Evidently, you’re not very experienced in assessing junior careers in their proper context. Next you will be telling me Vertstappen’s result in Euro F3 was underwhelming, based on who beat him.

      The RB8 was indeed clearly the best car when accounting for both its speed and reliability. The only car that comes close in 2012 is McLaren, which often had the Red Bull matched for pace at least.

      Your subjective assessments of Vettel’s cars don’t square with the data.

  10. So pretty much the model accurately places each driver higher relatively to his team mate as this reflected also by FIA’s point system (http://www.formula1.com/content/fom-website/en/championship/results/2015-driver-standings.html) but not to ALO vs BUT battle!

    Hilarious analysis 🙂

    1. Often the model ratings of teammates are not in the same order as the championship standings. For obvious reasons.

      1. Well it should be. We are talking about 20 or so races and should be enough to rate teammates over a course of a season.

        For example, ALO was closing on BUT in Italy and then he had a problem – Draw.
        In Singapore, BUT was closing on ALO before problems but you gave it to ALO – Why?? How can you predict the result? Biased opinion maybe? Check the race 5-8 laps before ALO had a problem.

        China BUT was ahead the whole race and then he collided with MAL. ALO overtook him. Why is that given to ALO? Did he have better race pace? I don’t think so. And if involving in some sort of accident takes points away, then why there is draw in Britain? BUT did nothing wrong there.

        For me it’s not that obvious why ALO wins the intra-team battle in your analysis. Why it’s the only pair which is mirrored compared to championship standings?

        Give Italy and China to BUT and run your analysis again unbiased.

        Cheerz!

      2. In China, Button was in an at-fault collision and finished behind. It would obviously not be fair to ignore or even overturn a result that occurred purely through his own actions on track. Indeed, that would be rather biased. I’m trying to imagine a race with Alonso’s and Button’s roles reversed where I award the result to Alonso and your reaction then…

        In Singapore, Alonso was 10.4 seconds ahead at lap 28, due to being consistently quicker in clean air. Alonso then reached the back of Nasr, which caused the gap to reduce to 8.8 seconds by lap 31. That was the sole reason Button was quicker on those laps. Alonso’s gearbox then overheated and he retired. Concluding that Button was capable of not only closing the gap but also passing when he had been slower in clean air all race is obviously not a sensible assessment.

        In Italy, I’m ruling in Button’s favor if anything by ignoring the race, since the timing data across the whole stint and their respective tyre compounds predict Alonso would have been ahead at the finish.

  11. Surely there is a better way of ranking the Toro Rosso pair? Heading into Silverstone GPS analysis had rated the Toro Rosso chassis as second best on the grid, ahead of Red Bull at the time, so I find it exceedingly unlikely that the car was only as good as the average between Lotus/Sauber.
    Would it be possible to rate the Red Bull car solely over the first half of the season (up to and including Hungary), and then use that car rating for the Toro Rosso? Over those races the Toro Rosso probably had a stronger chassis, but it’s surely the most representative gauge we are going to get for now.

    I definitely didn’t expect to see Nasr so highly rated. I thought he had been about as impressive next to Ericsson as Kobayashi had been, so my subjective ranking placed Nasr somewhere alongside Perez (and Verstappen) just outside the Top 10. I still rated him as rookie of the year though.

    1. There isn’t really any satisfactory way to estimate the Toro Rosso performance, because the drivers are rookies of highly uncertain ability. You could definitely assume a higher performance level for the Toro Rosso car than I did, in which case Verstappen and Sainz would be lower in the list than my estimates of 5th and 15th, respectively.

  12. Keisalex · · Reply

    Again: Will we see a, say, 2013 or 12 or earlier year’s drivers rankings?

    1. For now, just the championship titles and select years in the reconstructed history post. There’s a little more detail available in the published paper if you’d like (feel free to email me). I think a full list would be a bit dry for a blog post here.

      1. Will definately do! Thank you very much again, have a Very Merry Christmas everyone ! 😀

  13. Excellent! Thanks indeed for your work!
    My subjective ranking matches well your model so as always it´s a pleasure reading your results and analysis. Best on the web as far as I know.
    Congrats!

  14. His espectacular Mexican Grand Prix proved that your model is correct and Sebastian Vettel is the best F1 driver of 2015.

    1. And you think Hamilton was the best than? His fantastic drive in Hungary problably prove yout point! Or his awesome end of season been beaten by Rosberg? You can’t judge a role season based on one GP!

    2. So I suppose you judge a driver’s season by only one race. OK then. Hamiltons fantastic race in Hungary proves he is the best. Alonso’s Abu Dhabi start crash proves how he’s the best in the last 25+ years. If we start jugding by only one selected race, every single driver had a mediocre season and deserves to be actually last in the model standings.
      Congratulations on your mindblowing comment.

  15. Out of interest, is this the first time the model has rated Vettel/Alonso/Hamilton the top 3 drivers (in any order?)

    1. Actually, they were also top 3 in 2008: 1. Alonso, 2. Vettel, 3. Hamilton. Notably they would also be top 3 in 2012 if you exclude Rosberg (since I suspect his rating is inflated that year by comparison to Schumacher).

  16. […] McLaren were again around a second adrift of Williams, with a base time of 1:26.4 and a degradation rate of 0.56 seconds per lap. Since the Honda engine is still at an intermediate stage of development, this is solid progress from 2015, but they are still some way adrift of the major players. As one would expect, Manor are further back, with a base time of 1:27.0 and a degradation rate of 0.56 seconds per lap. It is, however, impressive to see them within 2 seconds of Williams, given they were scarcely inside 107% last year. […]

  17. […] McLaren probably still have the strongest driver line-up in the sport, you cannot count them out from a podium under favorable circumstances at a favorable track, but it […]

  18. It is mentioned somewhere among the comments that you might consider using match-ups from junior series to ‘fill in the gaps’ for drivers who are less strongly connected to the rest of the F1 field. If you did this, would you consider all the drivers in a one-make formula, like GP2, for example, to effectively be ‘team-mates’? On the one hand they are using the same equipment, but on the other their teams have access to different funding, and may use different strategies.

  19. Karl D. E. Sundin · · Reply

    How are the Toro Rosso drivers ranked now that we have a bit more data on them?

    Also, how is this year’s model championship looking?

    Looking forward to your next post!

    1. Working towards the end of year review and analysis! For now, the Toro Rosso pair are both ranked very highly, but the results are still quite volatile due to the one-sided Sainz-Kvyat match up, so I’ll wait until the end of the season before I trust it.

    2. I wanted to ask that question myself. 🙂
      I fear Kvyat will continue to underperform this season as he shows no signs of improvement, so then the rankings won’t be much more reliable at the end of the season. The Ricciardo-Verstappen battle probably gives more information, even if Ricciardo’s ranking is still quite uncertain (he was paired with Vergne and Kvyat for an important part of his career, while Vergne and Kvyat were teammates in 2014, so the rankings of the three drivers very much depends on their relative performance against a few external teammates and one of them was Vettel in 2014, who clearly underperformed). So it may take a few years before we have a reliable ranking for Ricciardo, Verstappen and Sainz.

      1. Dalligan · ·

        True, it’s interesting as well how much luck has come into this year as well – Ricciardo, Vettel and Hamilton have all had a lot of bad luck this year in comparison to their teammates. They’ve all lost out on a lot of points because of incidents, bad strategy, or their car just randomly breaking down. So maybe this ranking will be a bit less in favour of these guys this time around?
        I imagine Alonso will be back on top this year as well.

  20. […] with age and/or experience. I presented some preliminary work on quantifying age effects in my 2015 review and I’m currently working to build a new model that includes these factors in a satisfactory […]

  21. Can´t wait for the 2016 update on driver rankings! How are Alonso/Button, Hamilton/Rosberg, Vettel/Raikonnen now ranking???
    Kind regards,

  22. […] I did for 2014 and 2015, as well as other seasons historically, I applied my model of driver and team performance to […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: