2017 f1metrics end of season report

2017carsThe 2017 Formula 1 season will be long remembered and debated. Could Vettel have stayed in championship contention with better reliability or fewer driver errors in the critical stages, or was the Mercedes advantage too much for Vettel or any driver to bridge? Is this a step on the road to recovery for Ferrari, or do the later stages of the season foretell yet another temporary-revival-into-implosion?

Besides the spellbindingly quick cars, the key ingredient for the season’s entertainment was the dynamic equilibrium of Ferrari and Mercedes. While most pundits and analysts were tipping Ferrari as the preseason favorites, I wasn’t as convinced. By my numbers, preseason testing was just too close to call; and so it played out as the early season evolved. The low-rake long-wheelbase Mercedes was nigh unbeatable at power tracks and those that demanded high aerodynamic efficiency (low drag to downforce). The high-rake short-wheelbase Ferrari came into its own at circuits with many low or mid speed corners and where total downforce generation was at a premium. It’s rare to have two top cars with such differing characteristics, and the nature of this inter-team battle is analyzed below, including its impact on the outcome of the World Drivers’ Championship.

As in previous years (links for each year: 2014, 2015, 2016), the focus of this article is separating team from driver performances to provide a quantitative estimate of how the drivers would have ranked in 2017 in totally equal cars. While it’s easy to focus on Mercedes and Ferrari, let us not forget that there were other drivers in 2017 who could also have been championship contenders in more competitive and/or reliable machinery. They were relegated to lower priority news in 2017, but have the chance to get full exposure here.

The basis for the model is a driver’s scoring rate in counting races. Counting races are defined as those in which a driver did not suffer a non-driver DNF (mechanical DNF, technical disqualification). An extended scoring system (described here) is used to give fractional points for lower places, so that the model can discriminate between lower placings (e.g., 12th vs. 18th). The scoring rate is transformed into a performance variable, which is then fit with a model that attempts to best explain the variance in terms of driver, team, and season effects. The model is fit to all data from 1950 to the present, and year-by-year rankings are then extracted. The model’s fits are therefore updated in real-time as data become available, which in some cases lead to slight adjustments of rankings in previous years.

The model used for this year’s driver rankings is a substantial upgrade of the previous model. Whereas the previous model (and all models like it) assumed a fixed level of performance across a driver’s career (with possible random variations in form, but no systematic changes), the new model accounts for both the average effects of experience and age on a driver’s performance. This allows the model to make fairer comparisons in instances such as a driver racing against a teammate who is a rookie or well beyond their peak age. In future posts (once the model itself is published in a journal article), I will provide a more detailed overview of the new model and some of the new findings to come out of it, which include updated all-time rankings, model-adjusted championship results, estimates of how much experience and age impact driver performance compared with other factors, future projections for rookies, and several other interesting things we just couldn’t do with the previous model.

Driver rankings

Driver rankings for 2017 are given below, using the model’s points per race (ppr; on a scale of 0 to 10), adjusted for team and season. Interleaved with the rankings are a few analysis sections on other aspects of the sport.

Brendon Hartley, Antonio Giovinazzi, Paul di Resta, and Jenson Button are unrated, because they did not complete the minimum of three counting races required for a season rating. All other 2017 drivers are ranked this year, and given how close some of the top rankings are, I’ve had to go to three decimal places!

In each driver’s profile, I have given their head-to-head results against their main teammate in qualifying (excluding sessions where a driver could not post a competitive time due to mechanical problems), races (counting races only), and points (in races they both started). For reference, I’ve also displayed each driver’s head-to-heads from 2014-2016.

Note that the ppr values should not be directly compared to previous years, as the new model uses a slightly different referencing system.

21. Pierre Gasly (5.705 ppr)

F1 Grand Prix of Malaysia - Practice

Gasly was one of the top 10 in last year’s junior rankings, and seemed destined for a Formula 1 drive as Kvyat’s performances disintegrated. Gasly initially found himself racing in Super Formula, where he came close to winning the title as a rookie, while Red Bull gave the once extremely promising Kvyat a final chance. With Sainz itching to move to Renault, and patience with Kvyat’s erratic performances finally running out, Gasly finally took his place on the Formula 1 grid at the 15th round in Malaysia. For a rookie arriving late in the season, it was a solid performance, although Sainz showed all season that the car (when it wasn’t breaking down) was capable of considerably more.

Late season arrivals are usually difficult for a driver, as they miss out on preseason testing, as well as opportunities to learn and master the car across the year. How much impact does this have? To answer this, I ran an analysis of model-derived driver performances in each Formula 1 season (adjusted for age and experience). I found drivers who started a season later than the eighth round of the championship, and compared to their average performance in their other seasons (requiring a minimum of five starts). The average effect of starting late for a driver at Gasly’s level is about -0.6 ppr, meaning he would move ahead of Palmer if we corrected for that fact. With further adjustment for experience and age effects (i.e., projecting Gasly’s ultimate potential), the model currently sees Gasly as a driver who could be about as strong as Perez.

20. Jolyon Palmer (5.990 ppr)

jolyon-palmer

PAL17A hapless season for Palmer ended four races early, and it was difficult to dispute Renault’s reasoning. At that stage in the season, only 8 of Renault’s 42 points were contributed by Palmer, leaving them 8th in the World Constructors’ Championship, a massive underperformance for the works team. With two drivers scoring at the same rate as Hulkenberg, they would have been 5th, ahead of Williams, Toro Rosso, and Haas. In monetary terms, those three places would be worth circa $20-30 million. Poaching the top scorer from Toro Rosso was therefore a calculated decision, coming as part of the larger Honda-Renault reshuffle.

While it was not surprising to see Palmer, a GP2 journeyman until he won the season on his fourth attempt, beaten by Hulkenberg, an outstanding rookie GP2 champion, the margins were surprising. The median difference of 0.955% in qualifying times was the second largest between teammates in 2017. While some of that difference might be attributed to Renault giving preference to Hulkenberg, Palmer looked comprehensively outclassed.

19. Kimi Raikkonen (6.551 ppr)

rai17

RAI17At age 38, Raikkonen is now heading well into the range of age-related decline for a Formula 1 driver. There will probably always be debates as to whether Raikkonen at his best was at the level of an Alonso or Vettel. With experience and age effects included in each driver’s trajectory, the model is somewhat more positive about Raikkonen’s performances in the period 2004-2007, rating him behind Alonso and Schumacher in those particular years, but not greatly so. This can be understood in terms of the improved comparisons with Heidfeld (when Raikkonen was a rookie) and the model considering the impacts of recent experience and age on Raikkonen’s post-comeback career.

Was he flattered by machinery in 2003, 2005, and 2007? Was he a better driver then than he is today? Yes to both points. As the rest of the grid has generally strengthened since 2007, Raikkonen’s performances have progressively slid further down, to the point where he could now rightly be considered among the weaker drivers on the grid. This can be seen in the graph in Massa’s entry below.

Throughout the season there have been well motivated questions over Raikkonen’s current value as a number two driver for Ferrari. The pros are that he seems willingly consigned to the role, he generates no drama with Vettel, and he is never likely to challenge Vettel for the drivers’ championship. The cons are that he’s rarely fast enough to play even a flanking role for Vettel, he’s not there to win on days when Vettel fails, and he obviously reduces the chances of a Ferrari constructors’ championship (although that was never realistically achievable in 2017). Based on the model estimates, most drivers currently in Formula 1 would score at a superior rate to Raikkonen given the opportunity of a Ferrari drive. Nonetheless, he will continue in that position in 2018.

18. Felipe Massa (6.596 ppr)

massa

MAS17It’s farewell to Felipe again, and I’m afraid there’s no heartwarming sentimentality in the model rankings. Despite clearly outperforming his rookie teammate at most rounds, Massa finds himself just behind Stroll in the model’s ppr rankings. This can be mostly chalked up to Baku, which was a freak event and the only race in 2017 where Williams penetrated the top 5. Massa was running ahead of Stroll there with a guaranteed podium before his car failed. The model excludes the bad luck of Massa not finishing, but doesn’t repay his lost podium. There’s potential room for improving models in this regard in future, although it does become practically difficult to project finishing positions for all non-finishing drivers given their possible interactions. Essentially, it would require linking a driver ranking model with a race simulator model, such as the one I presented previously.

As Massa ends his Formula 1 career, it’s worth reflecting again, now that we have a model that includes age and experience effects, and therefore more realistically captures changes in performance trajectories over time. Below is a graph of his career performances, along with some other drivers of his period who, for one reason or another, never quite delivered on their potential promise.

almost_drivers

Massa’s career is marked by a ‘purple patch’ from 2008-2009, where he was performing at a higher level than before or after. This coincided with a relatively weak 2008 from Raikkonen, allowing Massa to get within a hair’s breadth of the title. Raikkonen’s performance level peaked in 2005, at which time he was approaching the typical performance level of the era’s best drivers, but he did not sustain or improve on this level as others such as Alonso did. Montoya showed early promise similar to Raikkonen, but seemed to hit a wall in 2004, and was easily beaten when he faced Raikkonen as teammates at McLaren. Hulkenberg’s improvement has been gradual, given his huge junior potential, but his trend remains positive. Kubica is a fascinating case, and one to explore in more detail if and when he signs for Williams to replace Massa.

17. Kevin Magnussen (6.822 ppr)

mag

MAG17Magnussen’s 2016 season was fairly disappointing, but far from the worst case scenario. After going 8-8 against the lowly-rated rookie Palmer in races in 2016, there seemed a good chance that a stronger teammate would dominate him in 2017. Instead, we saw a very feisty performance from Magnussen, in which he established himself as one of the most uncompromising racers on the grid. His qualifying performances were not particularly strong, but he was one of the biggest gainers on the first lap of the race, averaging a 1.7 place improvement. Overall, he held his own against Grosjean this year, which is at least enough to maintain a Haas seat for 2018.

16. Daniil Kvyat (6.876 ppr)

Motor Racing - Formula One World Championship - Singapore Grand Prix - Race Day - Singapore, Singapore

KVY17Kvyat at this stage is reminiscent of Alguersuari, another former Red Bull program driver. Alguersuari was the then-youngest Formula 1 driver at 19, dropped by Toro Rosso at 21, and retired from motor racing altogether at 25. Such is the nature of the Red Bull driver program, which is designed to find potential stars and anything less will simply not do. For some drivers, the associated mental strains during the periods of poor performance seem to be simply too much. Kvyat was riding a wave of success from the GP3 championship (easily beating teammate Sainz) to Toro Rosso (showing strong pace against the more experienced Vergne) to Red Bull (outscoring established ace Ricciardo, albeit with some good fortune). Once he was pushed backwards, it seemed to put him into a terminal spiral.

The ultimate outcome for Kvyat has long seemed inevitable, but due to the sudden movement of Sainz to Renault (which necessitated promoting Gasly) and a lack of any other F1-ready replacements in the junior chain, Red Bull were left flatfooted. The Red Bull junior team appears to currently be in a scouting phase rather than a development phase. Richard Verschoor had an excellent first season in single-seaters but apparently didn’t show the progression expected in 2017 and was dropped. Likewise for Niko Kari, who couldn’t make an impression on the 2017 GP3 championship. Others dropped from the team since 2014 include Illott, Camara, Leeds, Lynn, and Stoneman. Top junior talents George Russell and Lando Norris were scooped up by Mercedes and McLaren, respectively. Leclerc — an obvious potential star of the future since 2015 — was wisely signed by Ferrari, as was Giovinazzi.

To replace Kvyat, Toro Rosso needed a driver with a superlicense, and the options now looked thin.

  • They could admit fault and take back one of their rejected drivers, such as Buemi or Vergne, although it’s doubtful either driver would accept the call-up, given what it entailed.
  • They could take a currently unsigned junior driver with solid results but not much F1 star potential (e.g., Sirotkin, Rowland, Deletraz). This would be quite antithetical to the Red Bull junior team.
  • They could try to dislodge Leclerc from Ferrari’s program, but Ferrari would likely drive the price very high given his potential future value.
  • They could gamble and take a driver from another series, such IndyCar (e.g., Power or Dixon) or across from WEC (e.g., Hartley). Arguably, it would be easier for a driver to transition across from the WEC hybrids than from the very physical, lower-tech IndyCars.

Ultimately, they settled on the last option, signing Hartley, a previous Red Bull junior driver who had failed to impress after he reached the Formula 3 Euro and FR3.5 series. On paper, Hartley is no future F1 superstar, but Helmut Marko may well remember that at a similar stage of junior development, Kvyat looked far stronger than Sainz, and we can see how that one played out. Now we will see what Hartley can make of the unexpected opportunity.

15. Lance Stroll (7.052 ppr)

stroll

STR17

As noted in Massa’s entry above, Stroll’s rating is mostly a quirk resulting from the lucky result in Baku. If we award Massa his probable 2nd place in that race and consequently demote Stroll to 4th, Stroll would be ranked dead last in this list with a ppr of 5.582.

Of the two Williams drivers, Massa was undoubtedly the stronger. The median difference of 0.963% in qualifying was the largest between any teammates in 2017. While there were some signs of midseason improvement, the gap ballooned back out in the closing stages. The 2-17 qualifying tally is particularly remarkable given Massa was himself beaten 4-17 by Bottas in qualifying last year, and Bottas was beaten 7-13 by Hamilton this year. If we compound these ratios, Stroll would be expected to outqualify Hamilton less than 1% of the time in sessions where neither have a mechanical problem.

As a result of their weak driver line-up, Williams scored about 40% less points than the model predicts they would have scored in a scenario where all 2017 teams had equally skilled drivers. As it happened, this didn’t actually cost them any WCC positions, as their performance deficit to Force India was too large to bridge, and they were able to just maintain position ahead of other midfield teams. Stroll’s financial contribution to the team was therefore a pure positive in 2017. If Williams find themselves in a closer fight next year, however, he will need to lift his game.

14. Romain Grosjean (7.216 ppr)

grosjean

GRO17

On a good day, with a car in which he is well attuned, Grosjean has proven he can be extremely fast. The problem for Grosjean is that he just doesn’t consistently deliver. This is reflected in the fact that despite convincingly outscoring his teammates across 2014-2017, he has had balanced head-to-heads in races in three of those years. On the good days he scores well, on the bad days he can’t beat Maldonado or Gutierrez. At Haas in 2017, he often seemed frustrated, unable to find his preferred set-up, and seemingly unable to work around the imperfections.

While he has cleaned up his driving from the crashfest of 2012 (where he crashed out of 6 races in 19 starts), Grosjean remains on average one of the most crash-prone drivers on the grid. Below is an updated list from a previous article on historical crash rates, showing how the pool of recently-active drivers (everyone who has raced since 2015, requiring a minimum of 10 starts) ranks in terms of crash rates.

crash_rates_2017.png

The top of this list is mostly populated with rookies. Of the drivers who have raced at least two full seasons, Grosjean ranks behind only Maldonado. Nevertheless, Grosjean remained the better points scorer at Haas for a second year running, this time against a slightly stronger teammate. The boat seems to have probably now sailed on a top-team seat for Grosjean, but he can still deliver valuable performances for the midfield.

13. Marcus Ericsson (7.241 ppr)

ericsson

ERI17

Ericsson got very little credit for a solid season in 2017. Up against one of the Mercedes junior team’s talents, it could have been very one-sided. On paper, Wehrlein scored all the team’s points with his strong drives at Spain and Baku. But both times Ericsson came home 11th and there were important mitigating factors for him.

In Spain, Ericsson’s strategy was not helped by the timing of the virtual safety car, which resulted in him being stuck behind Stroll after his second stop, dropping him away from the battle for 9th and 10th. Probably he was on course for 11th anyway. In Baku, Ericsson followed a team order to allow Wehrlein through for 10th following floor damage to Ericsson’s car from debris — he then acted as rear gunner to protect Wehrlein’s position from Vandoorne. In qualifying and races, Ericsson trailed Wehrlein, overall, but by the respectable tallies of 8-11 and 5-8, respectively.

In 2018, Wehrlein loses his Sauber seat, despite being the slightly quicker of the pair. Ericsson will act as the benchmark for a new highly rated driver: the Ferrari junior Charles Leclerc.

12. Valtteri Bottas (7.384 ppr)

bottas

BOT17Up to Hungary, Bottas seemed a genuine contender for the drivers’ championship, trailing Hamilton by just 19 points, with head-to-heads of 5-6 in qualifying and 5-6 in races. That relative parity was somewhat artificial, as Hamilton’s bad luck in Austria (grid penalty) and Azerbaijan (loose headrest) had cost points and places, although Bottas was not without misfortune in Spain (first lap accident). Hamilton’s storming run from Belgium to the USA put Bottas firmly in the number-two role; five wins to Hamilton while Bottas only twice made the podium. It took outside intervention from Vettel in Mexico to reverse the trend, and then a rare mistake in qualifying from Hamilton in Brazil. Abu Dhabi was a positive end to the season, swinging some momentum back away from the British champion.

Overall, it was a solid year from Bottas, as he experienced his first exposure to a championship battle. He took valuable points from the Ferraris in Russia and Austria; something his opposing number two did not achieve. But Hamilton’s benchmark performances showed how far Bottas was sometimes from extracting the maximum performance of the car, and days like China, where he spun under the safety car, will want to be forgotten quickly. On too many weekends, Hamilton was peerless while Bottas was struggling to beat the Ferraris or Red Bulls. Of the Mercedes team’s 12 wins, just 3 went to Bottas this year.

In terms of qualifying pace, Bottas was considered preseason to be a possible challenger to Hamilton; the reasoning being his impressive 17-4 tally against Massa in 2016. However, Hamilton asserted himself as the faster one-lap driver in 2017, as he has done to every teammate to date (albeit by narrow margins against Alonso and Rosberg). The graph below shows how Bottas performed in qualifying relative to all of Hamilton’s teammates. Each point represents a single race weekend; the fastest time set by each driver was used across the sessions where both drivers set times.

Hamilton_quali

On this metric, Bottas is the weakest of Hamilton’s teammates to date, although still respectably close to Button. It will be interesting to see whether Bottas can get on closer terms with Hamilton in their second season together.


The dynamic equilibrium of Ferrari vs. Mercedes

merc_ferr

Mercedes and Ferrari were the yin and yang of the 2017 season, keeping the competition exciting and unpredictable until the closing stages. Ferrari’s car was an all-rounder, able to bring tyres into their optimal range easily. Mercedes had perhaps the theoretically quicker car, but they couldn’t always access its potential. Ultimately, Mercedes prevailed, as they seemed to progressively solve their problems, while Ferrari stumbled and encountered reliability issues.

There was a perception in 2017 that the Mercedes had additional performance available to it in qualifying that the Ferrari did not. This was supported by Mercedes scoring 15 of 20 available poles. However, much of this difference can probably be correctly attributed to Hamilton’s phenomenal one-lap pace. If we compare Vettel and Bottas in qualifying, the head-to-head favors Vettel 12-7.

To investigate the factors that favored each team, I compiled telemetry data from videos, logging car speed at 1-second increments to obtain descriptive parameters for each track. In each case I used the fastest dry lap available (usually the pole lap).

In the graphs below, we can see how the Mercedes advantage (percentage difference between fastest Mercedes and fastest Ferrari) varied as a function of two informative factors.

  • The percentage of the lap spent on throttle (computed as time where speed was not significantly decreasing.
  • The percentage of the lap spent in the cornering phase at medium speed (computed as time when speed was 150-250km/h and speed changed by no more than ±20km/h at the next time point, implying not in heavy braking or acceleration).

merc_ferrari_factors

Overall, Mercedes was more competitive when more of the lap was spent on throttle, reflecting their strong powerunit. Ferrari was more competitive when more of the lap was spent in medium speed corners, reflecting their strong aero package.

Together, these two factors accounted for 53% of the variance in the Mercedes advantage. Other obvious factors such as temperature and average speed did not provide any meaningful improvement in the fit when included in regression alongside the above two factors.

Going ahead to 2018, it will be interesting to see how Mercedes and Ferrari balance evolution of the existing designs against completely new designs. While their two concepts were closely matched in 2017, it was the first year of a rule cycle, and it is plausible that one concept has a much higher performance ceiling with further development.


11. Esteban Ocon (7.591 ppr)

Ocon

OCO17Ocon’s season was highly impressive on two levels. First, he showed that he could keep pace with a well-established driver, despite having only half a season of prior F1 experience. Second, he showed a clear upward trajectory in his performances across the year. In the first half of the season, Perez beat him convincingly by small margins. In the second half of the year, there was nothing really between them.

Ocon_Perez

Looking at Ocon’s qualifying times relative to Perez across the year, there is a visible upward trend, which approaches statistical significance (p=0.06). A similar upward trajectory across the year in 2014 is what saw Kvyat rapidly promoted to Red Bull ahead of the tried and tested Vergne. If Ocon can continue this promising trajectory, the implication is that he will be stronger than Perez and potentially one of the best drivers on the grid in a few years.

10. Nico Hulkenberg (7.612 ppr)

Hulk

HUL17Hulkenberg seemed particularly at home in the 2017 high-aero cars with lower-deg tyres. As shown in Massa’s entry above, he seems to still be improving over time, and 2017 is rated his peak performance to date, just ahead of 2014 when he beat Perez.

It was a devastatingly one-sided tally against Palmer, including a clean sweep in qualifying (enabled by Palmer’s car failing in qualifying at Spa, the one time he looked convincingly the quicker of the pair). The caveat is that his teammate in 2017 was one of the weakest drivers on the grid. In 2018, we’ll get a chance to see Hulkenberg against a much stronger driver: Sainz.

9. Sergio Perez (7.875 ppr)

Perez

PER17

In terms of rivals, it was a bit of a lonely year for Force India. Clearly slower than the top three teams and clearly quicker than anyone else. What resulted was a furious intra-team rivalry between the two Force India drivers. There was potentially a lot at stake for both drivers. Ocon needed to establish himself relative to an established quick driver, and indirectly relative to Wehrlein to gain continued Mercedes support. Perez wanted to display his potential to a top team by outpacing a highly rated young driver.

While Perez ultimately emerged victorious in this battle, his reputation is not without scars. His advantage over the less experienced Ocon was not really enough to convince anyone that he is a top-level driver. His incendiary relations with his teammate also raised questions over his suitability for a number two role. Would he pull over for Vettel or Hamilton if asked? From what we saw this year, probably not!

For a midfield team such as Force India, Perez remains a perfect match. He has pace comparable to any reasonable alternative, the ability to deliver unconventional strategies (a key method for midfield teams to score points), and financial backing to boot. When podiums are an option, he’s seemingly always there. In fact, out of the drivers who have never won a race, Perez leads his teammates by 6 podiums (7 to 1), the biggest margin in F1 history.

8. Pascal Wehrlein (8.016 ppr)

wehrlein

WEH17This might be the end of the road for Wehrlein’s Formula 1 career, but he can be proud of his performances. Each year he has had to race for the least competitive team on the grid, and each year he has delivered all of his team’s points. The season highlight this year was undoubtedly Spain, where he achieved 8th place. All of this was doubly impressive given Wehrlein’s difficult start to the season and lack of mileage. His neck injury caused him to miss testing and the first two races. Moreover, he was lucky to escape a low-speed accident at Monaco, where his head impacted the crash barrier.

What was perhaps missing from Wehrlein’s season relative to the rival Mercedes junior (Ocon) was the trajectory. Given Ericsson is considered a somewhat easier opponent than Perez, Mercedes observers would have hoped to see Wehrlein further assert himself over his teammate as the season went on. Instead, the gaps remained largely the same.

Weh_EriWill we see Wehrlein in Formula 1 again? Given his potential age-related and experience-related improvements in the next ~3 years, he would be a sensible choice for Williams if they choose not to sign Kubica. Otherwise, without Mercedes backing, it’s difficult to see an obvious route back for him.

7. Stoffel Vandoorne (8.069 ppr)

vandoorne

VAN17Parallels between Vandoorne and Hamilton this year were inevitable. Ten years apart, they each found themselves rookies at McLaren alongside Fernando Alonso. Both came into Formula 1 heralded already as generational talents based on their incredible junior careers. Vandoorne took slightly longer to find a Formula 1 seat, arriving at age 25, relative to Hamilton at age 22. But crucially, Hamilton arrived just before testing restrictions hit Formula 1, meaning he had vastly more mileage than Vandoorne in current cars. Vandoorne was in no way helped by the unreliable Honda powerunit, which further limited his early season mileage. He picked up the undesirable accolade of most grid penalties in 2017, and McLaren-Honda cumulatively had more than the rest of the grid combined.

Hamilton’s rookie season is perhaps the best Formula 1 has ever witnessed, and it will likely remain the benchmark for many years to come. Despite less experience, Hamilton in 2007 was every bit Alonso’s equal. By comparison, Vandoorne’s 2017 season was markedly less impressive. Alonso imposed himself as the clearly stronger McLaren driver. However, Vandoorne displayed signs of improvement across the year, and managed to shade a driver who has regularly dismantled his teammates, including highly-rated non-rookies. The graph below shows Alonso’s qualifying time advantages as percentages over teammates across his Formula 1 career. Each point represents a single race weekend; the fastest time set by each driver was used across the sessions where both drivers set times.

Alonso_quali2From this, we can see that Vandoorne’s performance within 0.44% of Alonso is better than Raikkonen, Grosjean, or Piquet Jr. achieved, and only 0.09% worse than Massa (who was 0.96% faster than Stroll this year) and 0.12% worse than Fisichella. Counting 2017 as effectively his rookie year, Vandoorne is the model’s rookie performer of the year. At peak age and experience, Vandoorne has the potential to be as strong as any of the current top drivers. It will therefore be fascinating to observe his second year alongside Alonso at McLaren-Renault.


The grid penalty system and its alternatives

The frequency and severity of grid penalties has led to questions over the validity of the existing penalty system. Let’s analyze the current system and some available alternatives.

Grid penalties: Currently, drivers can accrue grid penalties for replacements of failed engine components or gearboxes, as well as steward-issued driver penalties. Although some may view this as an artificial way of generating racing, comeback drives are undoubtedly popular with fans. Since the Driver of the Day voting system was introduced, 18 of the 41 winners are drivers who started outside the top 5, and 8 of those had grid penalties.

A potential upside to the grid penalty system is that it is inherently handicapped: a 10-place penalty is a severe penalty for a driver on pole, whereas it is next to no penalty for a driver who qualified 18th. It is therefore more lenient towards manufacturers who are already struggling to be competitive.

Potential downsides to the grid penalty system are that it can lead to overly complicated scrambling of the qualifying order and it can compound issues for drivers who are already suffering reliability problems.

Constructors’ points penalties: A widely-discussed alternative to grid penalties is docking of championship points from teams for part replacements. The argument goes that this way the driver is not affected by the team’s failings. This is different from how Formula 1 normally works; team-driver partnerships win and lose together, and drivers aren’t repaid for mechanical failures, or teams repaid for driver errors. This system is also most likely to hurt teams that are infrequent points scorers, whether the penalty is a percent-based or not. It also has a chance of anticlimactically deciding the constructors’ championship before the race is run; grid penalties impede race results, but do not conclusively decide them. Finally, a points-based penalty system is easily exploited by teams that have a safe position in the constructors’ championship; they can sacrifice unnecessary points to boost a drivers’ competitiveness in the drivers’ championship (e.g., by giving Hamilton fresh engines in every race).

Relaxations on component allowances: It has been suggested that Formula 1 should abandon all attempts at economizing components, thereby removing the need to impose penalties. The argument goes that Formula 1 should be about ultimate car performance, and costs are at an all-time high anyway. As discussed in my previous analysis of Formula 1 finances, cost reduction rules such as those on component usage are ultimately intended to reduce the potential gains available in certain areas, leading to teams spending in less cost-efficient areas. The downstream effect of that is a limit on how much performance the big spenders can gain, and a limit on how much money the large manufacturers are willing to spend; at a certain point, diminishing returns ensure that marginal improvements in performance and marketability are outweighed by sheer costs.

As we have seen in occasional circumstances since 2014, these hybrid engines have significant top-end performance that is rarely exploited in races due to reliability concerns. If top teams were allowed to bring a fresh engine to each race and qualifying session, as they did in the past, the performance gain for the additional cost would surely be a no-brainer. Customers would either need to run at a further disadvantage or renegotiate engine deals with higher engine allocations (on which the FIA imposes no cost cap). In sum, manufacturer teams would benefit.

In summary, a slight relaxation in the number of components allowed per season would certainly be worth exploring, to hopefully reduce the compounding of problems for teams with poor reliability, but a total relaxation seems a bad move. Any limit on components has to be somehow enforced, and relative to a points-based penalty, a grid-based penalty seems quite reasonable, if not preferable.


6. Max Verstappen (8.122 ppr)

verstappen

VER17For the first half of the season, Verstappen seemed cursed. Just about anything that could go wrong did go wrong. It got to the point where some fans were concerned about unequal machinery or Verstappen himself somehow causing the failures! But as I showed recently, we should always be cautious in reaching conclusions about reliability from small samples. By the end of the season, Verstappen actually had fewer mechanical DNFs than his teammate Ricciardo (4 vs. 5), yet still trailed 168-200 in points. Since the model’s only approach to correcting for bad luck is ignoring races with mechanical DNFs, it isn’t going to correct for other factors, and so Verstappen, with fewer points and fewer mechanical DNFs, inevitably trails Ricciardo in the model rankings.

Those who watched the races closely know that due to the way the cards fell, Verstappen’s mechanical DNFs cost more potential points than Ricciardo’s. Several online analyses have been posted already (example), in which people have tried to correct for effects of bad luck. I have given my view with a summary of key incidents below.

Mechanical DNFs

  • Verstappen lost a likely P4 in Bahrain, which would have dropped Ricciardo to P6 (+12 for Verstappen, -2 for Ricciardo); and a likely P1 in Azerbaijan, which would have dropped Ricciardo to P2 (+25, -7).
  • It’s difficult to assess where Ricciardo could have finished in Australia (grid penalty and early retirement) and Russia (early retirement). An optimistic projection based on Verstappen’s pace would be immediately behind Verstappen in those two races for P6 (0,+16).
  • It is similarly difficult to assess where Verstappen could have finished without the clutch failure and Kvyat torpedo in Austria. Since he qualified one place behind Ricciardo, it’s reasonable to assume the same for the race: P4 (+12, 0)
  • In Canada, Verstappen was running P2 and looked quick enough to hold the position when his car failed, which would have dropped Ricciardo to P4 (+18, -3).
  • In Britain, Ricciardo had to start near the back, but recovered to P5, helped by Vettel’s problems. Verstappen’s pace was generally stronger than Ricciardo’s in the race, so I don’t think Ricciardo actually lost points here (0, 0).
  • In Belgium, Verstappen was running ahead of Ricciardo when his car failed. He was therefore on course for P3, which would have dropped Ricciardo to P4 (+15, -3).
  • Ricciardo outqualified Verstappen in the USA and was on track for at least P4, which would have dropped Verstappen to P5. Looking at Red Bull’s race pace and undoing Verstappen’s grid penalty, they likely both could have passed Raikkonen, elevating them to P3 and P4, respectively (0, +15).
  • In Mexico, Ricciardo was up to P7 (his qualifying position) by the time of his engine failure, after starting 16th due to grid penalties. At worst, Ricciardo was on course for P4, and probably P3 without the grid penalty (0, +15).
  • In Brazil, both drivers were running with turned down engines. Ricciardo was outqualified by Verstappen and then took grid penalties. He recovered to one place behind Verstappen, so no actual points loss (0, 0).
  • Ricciardo was on course for P4 in Abu Dhabi, which would have dropped Verstappen to P6 (-2, +12).

Net change: +80 for Verstappen, +43 for Ricciardo

Updated points tally: 248 for Verstappen, 243 for Ricciardo

Accidents

  • Ricciardo was taken out by Verstappen in Hungary. This was clearly Verstappen’s fault. Had Ricciardo survived that contact, he could likely have held P4 given difficulty passing in Hungary, dropping Verstappen to P6 after his penalty (-2, +12).
  • Verstappen was involved in multiple incidents during 2017 that potentially cost points. Some of these were racing incidents. In none of them do I consider him to be primarily to blame, but there was a pattern of entering high-risk positions that could potentially end badly for him. Spain and Italy both fall into this area. I view Singapore as the most straightforward case: Verstappen was taken out while challenging the Ferraris for the lead and it’s difficult to blame anyone but Vettel for this accident since he steered into the path of other cars in a straight line. This race is a difficult one to play out, but had he survived the accident Verstappen was probably on course for about P2, with Ricciardo relegated to P3 (+18, -3).

Net change: +16 for Verstappen, +9 for Ricciardo

Updated points tally: 264 for Verstappen, 252 for Ricciardo

From this detailed analysis, it’s reasonable to conclude that the Red Bull drivers were very closely matched this year, but with maybe a small edge to Verstappen. Although he isn’t quite there in the model rankings for the reasons laid out above, it’s reasonable to conclude that he belongs among the top few drivers this year. Given he is neither at peak age (>50% of age-related improvement remains) or peak experience yet (>20% of experience-related improvement remains), Verstappen may yet become the strongest driver on the grid.

5. Daniel Ricciardo (8.570 ppr)

ricciardo

RIC172017 is actually the first time that Ricciardo has been outqualified across a season by a teammate. His previous results are 5-5 against Liuzzi and 1-0 against Karthikeyan in 2011, 16-4 against Vergne in 2012, 15-4 against Vergne in 2013, 12-7 against Vettel in 2014, 11-7 against Kvyat in 2015, and 11-6 against Verstappen in 2016.

As reviewed in Verstappen’s entry above, it was a close match between the Red Bull drivers this year, with probably a small edge to Verstappen after accounting for misfortune. The overall tally between the two now stands at 18-17 to Verstappen in qualifying, 14-13 to Ricciardo in races, and 420-359 to Ricciardo in points. We should all be very thankful that we get to see this explosive duo go head to head again in 2018. Let’s just hope that Red Bull can deliver a championship contender.

From here on, the ratings are incredibly tight. The next four drivers are so close as to be effectively identical in performance. In fact, as the graph below illustrates, 2017 has the smallest ppr difference between the drivers ranked 1st and 5th of all years since 1950! Differences at the top have in general become smaller since the 1970s, reflecting closer competition and higher standards of driving. But never have the top drivers been this evenly matched.

top5_ppr

4. Lewis Hamilton (8.580 ppr)

hamilton

HAM17Hamilton’s 2017 season was an impressive rebound from 2016 and arguably his strongest season yet. Following his streak of 6 wins in 8 races from Great Britain to the USA, Hamilton seemed destined for not only this year’s title, but also number 1 in the f1metrics model rankings. But through a combination of misfortune in Mexico and some underwhelming tail-end performances reminiscent of the post-title trail in 2015, he slipped slightly. With 2nd in Mexico (i.e., without the Vettel contact) and 1st in Brazil (i.e., without the qualifying crash), he would indeed have topped the f1metrics 2017 rankings and Vettel would have been ranked 4th. Such were the margins this year.

As noted above in Ricciardo’s and Verstappen’s entries, the top 5-6 drivers this year could rightly be considered equals. The numerical differences between them are smaller than anything we could reasonably hold confidence in.

If Hamilton were not already in the conversation for all-time number 1 driver, he has surely put himself there with his cumulative achievements this year. He now trails only two drivers in titles (Schumacher, Fangio), one driver in race wins (Schumacher), and he holds the all-time record for poles. As highlighted below, Hamilton has now also firmly established himself as one of the all-time greatest wet-weather drivers.


The greatest wet-weather drivers

regenmeisters

Three and a half years ago, I published an analysis of historical wet-weather performances. For these analyses, I classified races as either wet or dry. A race was classified as wet if the track surface was wet during any phase of the race (including the start). At the time of that analysis, Button was the currently-active driver with the most wet wins (7 wins), just ahead of Hamilton (6 wins). Since then, Hamilton has increased his tally to 14 wet race wins. In fact, he has won the last 8 consecutive wet races, setting the longest consecutive wet winning streak in the sport’s history.

wet_consec_wins

Undoubtedly, Hamilton was helped by driving extraordinarily dominant cars in those years, but we can gain additional insights by examining his teammate head-to-head records in wet vs. dry conditions.

wet_ham_teammatesAgainst Rosberg, the wet/dry performance difference is extreme, and during this period Hamilton achieved 6 of his 14 wet wins. This record is reminiscent of Senna vs. Prost; Senna finished ahead of Prost 3-0 in the wet, but only 11-9 in the dry. Against his other teammates, Hamilton’s wet weather advantage was less obvious, and he was tied 5-5 with Button, one of the other strongest wet drivers of recent times.

We can conclude that Hamilton’s impressive wet weather record is a combination of being among the strongest overall wet weather drivers, driving strong cars, and having a teammate (Rosberg) who was relatively weak in wet conditions during the period of greatest car dominance. For total wet wins, Hamilton is now 2nd in the all-time list, tied with Senna, and behind Schumacher.

wet_total_wins

As a percentage win record, Senna’s 63.6% is difficult to beat. Ascari is narrowly ahead, but from a very small sample. Hamilton at 42.4% is now ranked 4th all time in this list, ahead of Schumacher, who was at 46.5% before his 2010-2012 comeback.

wet_perc_wins


3. Fernando Alonso (8.738 ppr)

alonso2017

ALO17

2017 was yet another season for Alonso spent toiling in an uncompetitive car, and actually in all respects it was worse than 2016. He undoubtedly remains one of the sport’s top drivers and an all-time great, yet he finished no higher than 6th in a race this year. Some may ultimately consider this predicament his own fault, but for viewers it’s a great shame to not have a driver of Alonso’s quality competing for podiums, wins, and championship titles.

The major highlight for Alonso this year actually came outside of a Formula 1 car, when he participated in the Indy 500. Despite his rookie status in the series, he demonstrated the necessary speed to potentially win the event until (naturally) a Honda failure ended his run. This crossover was a huge shot in the arm for motorsports and generated unprecedented traffic on online forums. The associated explosive growth in subscribers is shown below for the Formula 1 forum on reddit.

alonso_reddit

Hopefully Liberty will take note from this and promote further crossover events, as they are almost always mutually beneficial for the media exposure of both series.

The big question that will be on Alonso’s mind as we head into the offseason is: can Renault engines deliver? He won’t be the only one asking this, as Red Bull’s competitiveness is dependent on the same factor. The data suggest that Renault are not there yet, but they are improving. For a detailed analysis of Renault’s and Honda’s prospects, see the next section.

 


McLaren, Honda, and Renault

Performance in the current racing formula remains strongly dependently on engines. In 2017, the Mercedes powerunit remained the one to beat, but there were positive signs for other engine manufacturers, which is good news given we have essentially stable regulations going into 2018. Below are graphed the percentage differences in qualifying between the fastest time for each engine supplier across the year.

engines_2017

Honda and Renault both show significant downward trends, gaining about 1.6% and 1.0% on Mercedes between the first and last races of the season, respectively. Those figures are also approximately their remaining deficits to Mercedes, although we shouldn’t expect them to make the same gain for a second season in a row, as diminishing returns will slow the rate of progress. Of course, we can’t totally disentangle engine performance from the chassis development at Red Bull (the main Renault representative) and McLaren (the only Honda representative), but we can reasonably assume that most of this progress was on the engine side, since that remains the major area of weakness relative to Mercedes, and we know Mercedes made significant chassis developments across the year too.

Looking at the graph above for 2017, McLaren’s jump from Honda to Renault looks like a significant short-term upgrade, but a less clear decision in the long term. We also should note that Renault’s reliability record in 2017 was poor, which should raise some questions about their ability to continue increasing powerunit performance. We can make more sense of McLaren’s decision by examining the whole 2015-2017 record.

honda_progressMcLaren-Honda appeared to be making rapid progress across 2015 and continued from where they left off in 2016. However, at that point, progress slowed considerably. In 2017, McLaren-Honda were essentially back to square one, beginning the season with a 3-4% deficit to Mercedes, as they had back in early 2015. Viewing the 60-race partnership as a whole, the net rate of improvement is a modest 0.017% per race, or a net improvement of just 1% relative to Mercedes across 3 years. It is understandable that McLaren would become frustrated looking at this rate of progress and the false dawns within.

Naturally, it’s important to ask how much of this problem for McLaren-Honda was on the chassis side vs. the engine side. Most analyses have concluded that the engine was primarily to blame. These factors are difficult to disentangle, but we can make our own estimates by comparing different types of tracks. After removing the within-season trends, here are the tracks of 2017 ranked by the percentage difference between the fastest Mercedes and the fastest McLaren-Honda.

engine_diffs_2017

At the far left we see the least power-sensitive tracks, such as Monaco, Singapore, and Hungary. At the far right, we have the most power-sensitive tracks — those that demand low downforce such as Italy and those with very long straights. Across this range, we see the McLaren-Honda deficit ranging from 1% to 3.5%, a difference of 2.5%. Theoretically, we could imagine a continuum of tracks from completely power dependent (e.g., a straight line) to completely chassis dependent (e.g., a tight radius circle), and by comparing those we could try to isolate the roles of engine and chassis. The tracks on the calendar of course don’t span to the extremes of this range, so we can only conclude that Honda costs McLaren at least 2.5% of the 3.5% deficit at the most power-sensitive tracks. This suggests that with a competitive powerunit, McLaren ought to be at least in the mix for podiums based on their chassis quality.


2. Sebastian Vettel (8.739 ppr)

Vettel

VET17

Should Vettel be ranked ahead of Hamilton? Whew. How to start a heated debate.

Both drivers were undoubtedly brilliant in 2017, but also had flawed weekends. While Hamilton had probably the stronger car on balance across the year, the popular perception was that Vettel had shed more of the available points. Those moments of madness in Singapore and Azerbaijan became symbolic for the entire season.

On careful analysis, it isn’t actually clear which of the championship protagonists gave away more points in 2017, either by careless driving or slow driving. In the table below, I have assessed the main cases where each driver clearly gave away net points to their rival compared to what a ‘perfect’ driver could have achieved on the same weekend. In some cases this was due to obvious error, in other cases due to being significantly outperformed by their teammate.

HAM_VET_table

I have tried to be similarly harsh on both drivers here, but it’s intrinsically a subjective analysis. Some Vettel fans will probably take me to task for including Spain (my view is that the Ferrari was the fastest car that weekend) or Canada (my view is that Vettel’s collision on the first lap was mostly of his own making). We can certainly nitpick this analysis, but the overall conclusion is that both drivers lost a similar amount of net points to their rival relative to what was clearly achievable in their car.

Where we place the two drivers’ performances should then largely depend on how we rate the relative merits of the Mercedes vs. Ferrari. As shown above, the advantage swung to each team depending on track characteristics. Overall, the model ranks the Mercedes the better car when pace and reliability are considered, and by such a margin as to just explain Hamilton’s points lead. But take careful note of just how close the numerical rankings are between drivers 1 and 5 in this list; they are all within any reasonable margin of error. This is not a season with a clear best performing individual.

1. Carlos Sainz (8.766 ppr)

sainz_both

SAI17The Ferrari/Mercedes battle was the year’s obvious focal point, making Sainz a largely overlooked star of 2017. The model is apparently intent on generating controversial discussion this year, so it has placed him number 1 by the slimmest of margins!

It was not a perfect season from anyone, Sainz included. For Vettel and Hamilton, there were several forgettable and costly weekends apiece, as tabulated in Vettel’s entry above. Sainz’s early season showed signs of overdriving as he spun in China (followed by a wonderful recovery drive), took out Stroll in Bahrain, and took out Massa in Canada. Those were the three low points. Through no fault of his own, he was also taken out by Kvyat in Britain and spun trying to avoid Kvyat in Azerbaijan.

In qualifying, there was actually little between Sainz and Kvyat this year, but in races Sainz was a monster. His Toro Rosso teammates never cracked the top 8, while he placed there in 7 of his 10 finishes. The only race in which both Toro Rossos finished and Sainz was behind Kvyat was Italy, where Sainz was affected by grid penalties.

toro_rosso_positions

Sainz’s mean finishing position for Toro Rosso was 8.1 vs. 12.6 for his teammates; a remarkable difference of 4.5 positions. For reference, Hulkenberg and Palmer differed by an average of “only” 2.5 positions this year.

In reading these data, we have to naturally question whether this was Sainz performing spectacularly or Kvyat — a previously quite promising driver — completely losing the plot. Let’s look at this question in more detail, as it gives some insights into how model ratings are generated and shows why my decision to post annual rankings is always a bit of a gamble.

While new data are coming in, the rankings of recent years remain significantly in flux. The rankings I post for each season are usually shuffled slightly a year or two later. As an example, see how the 2014 rankings have changed since I published them, due to new data from 2015-2017 and an improved model.

2014_driver_tableMost of the changes are minor tweaks, but for a few drivers (especially those who had relatively few teammate links at the time), the changes are significant. Perez, Hulkenberg, and Vettel are big gainers, while Bottas, Vergne, Raikkonen, Kvyat, and Ericsson have had their 2014 performances reappraised downwards in light of new evidence. In the same fashion, how Sainz performs next year against Hulkenberg will retrospectively modify his season rankings for 2015-2017, and unless he beats Hulkenberg by an impressive margin, he will most likely move to a lower position in the 2017 top 5, given how fine are the margins.

A particular uncertainty arises with the rankings of Sainz/Kvyat due to insularity of the Red Bull driver program. Toro Rosso drivers tend to either get promoted to Red Bull or leave the sport, making it hard to directly compare them to others. As shown in the network graph below, the cluster of drivers containing Ricciardo, Verstappen, Sainz, Kvyat, and Vergne is only tenuously connected to the rest of the grid.

driver_network_2017cThe main outward connection for this group is the 2014 Ricciardo-Vettel teammate battle. Ricciardo convincingly won that battle, leading to impressive rankings for all within the cluster, but many have questioned whether 2014 was a typical season for Vettel. The ranking of the Red Bull cluster as a whole therefore has to be treated with some scepticism. I have noticed, for example, that the all-time rankings of drivers in this cluster are presently quite sensitive to Vettel’s gains and losses vs. Raikkonen, since these serve as another major anchor point.

The new Sainz-Hulkenberg comparison (illustrated as a red dashed line) is precisely what we need: a connection to another well-connected driver who is distant from the Red Bull driver network. The little we saw of Sainz vs. Hulkenberg in 2017 suggested they are competitive with one another, but Renault’s poor reliability undermined any serious comparisons. It’s worth noting that had Sainz finished 9th in Mexico and Abu Dhabi rather than suffering mechanical failures, his season ranking would fall to 4th behind Vettel, Alonso, and Hamilton, as the model would gain certainty in a smaller performance gap between Sainz and Hulkenberg.

The 2018 season will therefore serve as a superb opportunity for improving confidence in all the Red Bull program driver rankings, and I will be sure to revisit Sainz’s 2017 season ranking at the end of next year. Enjoy the offseason!

Advertisements

54 comments

  1. Andrew M · · Reply

    Very interesting as always. The addition of age related/rookie effects obviously has a large impact for some drivers (I think it brings Perez and Hulk to more reasonable positions than previous years for example), and like you hint I think Sainz will be brought down to earth next year – I think Kvyat’s performances fell through a trap door in 2016-17 after being demoted.

    Hamilton vs Vettel is interesting, but I think it’s clear that Hamilton could have done more in Mexico/Brazil/AD if he needed to for the title. Baku also hurt him as he lost a certain win through no fault of his own.

    Just out of interest, how is Singapore counted for everyone who crashed out? Is the race just not counted for Vettel, Verstappen, Alonso etc?

    1. Thank you. Crashes are always counted by the model as a driver DNF (i.e., a zero points race) regardless of whose fault it was. That can certainly be unfair in some cases, but the alternative is trying to subjectively assign blame for accidents, which is very murky area I’d rather avoid in a model.

      1. Thanks, that’s different to how I thought it was actually, and includes quite a stiff penalty in Vettel’s score already. Of course with the top 5 so closely matched lots and lots of variables could effect the final outcome.

        Very interested to see more details of the updated model and how it effects various rankings. A lot of Kimi fans will be disheartened to see him as such a shadow of his former self…

      2. Another thought about Sainz – he probably benefitted slightly from bailing out from Toro Rosso a few races before the year end, meaning those left had to take all the penalties and start at the back as they ran of PUs.

      3. Agreed, that definitely didn’t hurt him! There are a lot of factors contributing to Sainz taking the number 1 rank, and with any one of those changed he would probably be 5th or 6th, due to how close the rankings are this year. It’s a surprising result on paper, but I think it’s clear why he appears there when we consider the modeling assumptions and the form of the available data (e.g., the teammate links). I wouldn’t personally have put him number 1, but somewhere in the top 7 for sure.

  2. As always,
    Thank you so much for your unique, novel, scientific and in-depth analysis!!!
    I enjoy it so much!
    Your findings and analysis totally fits with my opinion on drivers. I´m slightly surprised by Vettel being up there with Alonso, but that is the result.
    Also, congratulations for the new work on adding driver performance variable across time.
    As always,
    BIG THANK YOU!

  3. as always a pleasure to read. the cluster diagram at the end is fascinating and really shows how the model works. however, one nitpick, you say renault would have beaten toro rosso with 2 drivers of hulkenberg’s quality – they did beat toro rosso!

    1. Ha, quite right! I think I wrote that a few races before the end of the season when the Sainz decision was being made.

  4. Thomas Turrell-Croft · · Reply

    I’ve been looking forwards to this post, thank you for all of your insight.

    I’m really impressed with the work you have done on the model.

  5. I’ve been checking this site daily and got quite the rush to see that this post is now live. Brilliant stuff, as always.

    In Mark Hughes’ race report on Monaco, he puts forward a well-reasoned case outlining why it’s extremely likely that Ferrari robbed Raikkonen of the win. What would reversing Seb and Kimi’s finishing order in Monaco do the ranking?

    1. I love his reports! I checked your hypothetical and Vettel would go down to 3rd (8.61 ppr), while Raikkonen would go up to 18th (6.83 ppr).

      My own view is that Vettel was quick enough to either undercut or overcut Raikkonen in that race, so Ferrari could have maintained Raikkonen’s lead with careful planning, but only if they favored him winning.

      1. Interesting on both counts – thanks!

  6. This is another very well done post, and one I was looking forward to, but I would like to take the chance, since it’s hard to get an answer from you in this blog till you make a new post, to ask you: did you fix yet the fact that indianapolis 2003 wet race was missing from the list? That should be another win for schumacher and the 36% win rate if I recall is the same as I saw in the old topic.

  7. […] View Reddit by whatthefat – View Source […]

  8. Hi Andrew,

    Here’s a video response to my concerns of what I see as a peculiar points inflation for 2017 compared to 2016:

    Kimi Raikkonen
    2017 (6.551 ppr) 19th
    2016 (6.51 ppr) 8th (????)
    2015 (5.79 ppr) 12th
    2014 (6.92 ppr) 8th

    Fernando Alonso
    2017 (8.738 ppr) 3rd
    2016 (7.92 ppr) 1st
    2015 (7.93 ppr) 2nd
    2014 (9.13 ppr) 1st

    Felipe Massa
    2017 (6.596 ppr) 18th
    2016 (4.91 ppr) 16th
    2015 (5.75 ppr) 13th
    2014 (5.93 ppr) 13th

    Valtteri Bottas
    2017 (7.384 ppr) 12th
    2016 (6.13 ppr) 10th
    2015 (6.14 ppr) 9th
    2014 (7.13 ppr) 4th

    Sebastian Vettel
    2017 (8.739 ppr) 2nd
    2016 (7.221 ppr) 4th
    2015 (8.30 ppr) 1st
    2014 (6.16 ppr) 12th

    Carlos Sainz
    2017 (8.766 ppr) 1st
    2016 (7.26 ppr) 3rd
    2015 (5.20 ppr?) 15th

    Lewis Hamilton
    2017 (8.580 ppr) 4th
    2016 (6.98 ppr) 6th
    2015 (7.91 ppr) 3rd
    2014 (8.23 ppr) 2nd

    Daniel Ricciardo
    2017 (8.570 ppr) 5th
    2016 (6.98 ppr) 5th
    2015 (6.72 ppr) 8th
    2014 (7.97 ppr) 3rd

    Max Verstappen
    2017 (8.122 ppr) 6th
    2016 (7.26 ppr) 2nd
    2015 (7.20 ppr?) 5th

    Romain Grosjean
    2017 (7.216 ppr) 14th
    2016 (5.06 ppr) 14th
    2015 (5.95 ppr) 10th
    2014 (6.30 ppr) 9th

    Marcus Ericsson
    2017 (7.241 ppr) 13th
    2016 (5.42 ppr) 13th
    2015 (5.28 ppr) 14th
    2014 (5.66 ppr) 15th

    Daniil Kvyat
    2017 (6.876 ppr) 16th
    2016 (5.03 ppr) 15th
    2015 (6.81 ppr) 6th
    2014 (6.18 ppr) 11th

    1. Andrew M · · Reply

      “Note that the ppr values should not be directly compared to previous years, as the new model uses a slightly different referencing system.“

    2. At the beginning of the article I say: “Note that the ppr values should not be directly compared to previous years, as the new model uses a slightly different referencing system.”

      There is actually no inflation between 2016 and 2017 within the new referencing system.

      1. Apologies, but what an extraordinary set of results. I don’t think anyone (mathematical or not) had Sainz down as their driver of the year. Excluding Gasly who did five races only, I think the only thing we can agree on Palmer is being the worst driver with Hulkenberg, Perez and Ocon slap bang in the middle!

        Sorry for being a salty Kimi fan, could not believe what happened in Spain (DNF lap 1 thanks to Bottas), Monaco (team orders?), Baku (hit by Bottas and ran over Force India debris), Canada (brake issues), Hungary (team orders?), Singapore (DNF lap 1 thanks to Vettel) & Malaysia (DNS due to turbo failure).

  9. Thank you for this very detailed post. It must have cost you a lot of time to do so much research.

    I had one question about your score metric. While the pprs are ideal to rank the drivers, how should they be interpreted? I know the ppr is a non-linear function and that the pprs calculated for this year cannot be compared with earlier pprs, but the actual numbers are quite meaningless to me. How much better would Vettel or Hamilton be than, say, Palmer in the same car according to the model? How many points would Palmer have scored in a Mercedes and how many points would Hamilton have scored in a Sauber?

    By the way, I think that this new model is a substantial improvement over the previous one. I can’t wait to read the updated all-time driver ranking post. Good luck with the research!

  10. A few remarks on what you wrote about Ericsson’s Spanish GP:
    “In Spain, Ericsson was right behind Wehrlein at the time of their pit-stops. Ericsson’s stop occurred just before a virtual safety car, dropping him by ~15 seconds and 5 places relative to those that stopped during it. Without this, he was probably on course for 9th.”

    I’m afraid this is incorrect.
    1. Ericsson was over 22 seconds behind Wehrlein before his pit stop, running in 13th place
    (I think you’re confusing this with Ericsson’s first pit stop on lap 18, when he was just 6 seconds behind his team mate – but the virtual safety car had nothing to do with that round of pit stops, as it was only deployed on lap 33; additionally, the fact that Ericcson had to pit on lap 18 and 32, while Wehrlein was only prompted to pit for the first time on lap 33 due to the VSC, despite using the same compound as Ericsson and posting the same lap times, was absolutely crucial for their different results).
    2. Ericsson pitted from P13, but found himself in 14th place after the VSC ended, having lost a place to Stroll (therefore effectively losing a single place)
    3. Ericsson didn’t lose ~15 seconds compared to the cars that stopped under the VSC. Between lap 31 (Ericsson’s last lap before entering the pits; not counting lap 32, as the finish line is crossed with the speed limiter in the pit lane, which already affects the lap time before the actual pit stop takes place) and lap 35 (first time crossing the line after the VSC ended), he lost:
    – 5.3 seconds to Wehrlein
    – 5.9 seconds to Stroll
    – 7.3 seconds to Hülkenberg
    – 9 seconds to Kvyat
    – 9.4 seconds to Massa
    (drivers who pitted on lap 33)
    – 8.8 seconds to Sainz
    – 12.1 seconds to Ocon
    – 12.7 seconds to Grosjean
    – 18.3 seconds to Pérez
    (drivers who pitted on lap 34)

    Out of all those drivers, only Grosjean and Kvyat were really relevant for Ericsson’s chances at scoring points (the other drivers were either too far ahead (PER 66 secs, OCO 55 secs, HUL 32 secs, WEH 27 secs, SAI 25 secs) or finished behind him (MAS, STR)).
    Kvyat was 3.6 seconds ahead before Ericsson pitted, and finished lap 64 (Ericsson’s final lap) 12.6 ahead. => Ericsson lost neither track position nor would he have been able to catch Kvyat without the time lost in the pits.
    Grosjean was 14.7 seconds ahead before the pit stop and finished the race 10.7 seconds ahead. => Ericsson didn’t lose track position and would only have had a chance to finish ahead of Grosjean if you assume that a pace advantage of 2 seconds over the last 30 laps would’ve sufficed to overtake Grosjean on one of the least overtaking-friendly tracks of the season.

    So, I have to disagree. Ericsson wasn’t ‘probably on course for 9th’ – he was on course for 11th, with a tiny chance of fighting for 10th.

    1. You are quite right about this, I’ll correct the article.

  11. Superb analysis, Andy.

  12. I’m a little confused about the age/experience thing. You say it’s been Incorporated into the model, which suggests that Verstappen’s score has been boosted because of his youth, but your analysis of Verstappen suggests this isn’t true.
    Does age/experience effect a drivers score here? Or is it merely that you can predict future scores better?

    1. I will explain in more detail when I discuss the model in a future post, as I realize I haven’t been super clear on this. To briefly explain,

      * The previous model assumed each driver had a flat performance trend across their career; there might have been year-to-year fluctuations, but these weren’t described by any trend in the model. Hence, when the model compared say a rookie to a driver at their career peak, it assumed they were both operating equally close to their full potential. In, for example, the case of Nico Rosberg vs. Michael Schumacher, this led the model to have an inflated estimate of Rosberg’s peak performance, since it didn’t consider the fact that Schumacher was likely not performing near his peak performance due to age and experience.

      * The new model takes into account age and experience of teammates across each year of a driver’s career when it tries to estimate the driver’s peak performance. Being beaten by a much more experienced driver as a rookie, for example, is therefore no longer as severe a penalty.

      * In each year, including 2017, the model still reports the driver’s absolute performance. It’s not making some correction for their current age to, for example, predict how Verstappen might have performed if he were 27. It can be used to make such projections, but those are not the basis of the ppr values presented. They are only correcting for team and season, i.e., asking how the drivers as they actually performed in 2017 would have performed relatively in the same equipment.

  13. The ranking tries to be an objective alternative to subjective opinion out there. When a number of influential subjective opinions agree though, it quickly gains heft. The Autosport Team Principals poll just awarded Hamilton 233 of a possible 250 points, the highest percentage ever given to any driver in that poll, which is in its 10th year. It was a pretty easy call this year, when you consider the closeness of the cars, and which drivers in the title fight performed at crunch moments, and which didn’t. Whichever TP had Hamilton 3rd is having a laugh (who are we kidding? It was obviously Horner).

    Many of the positions in this year’s F1Metrics ranking just simply don’t pass the smell test. Sainz in #1 … running into Stroll in Bahrain & wiping out Massa in Canada, pretty much remove him from any Best Driver consideration. No one in F1 would have Sainz as the top driver, not even Renault.

    Vandoorne in 7th is baffling. He had a frankly awful first half. I was expecting big things from him, and he disappointed. Binning it after the SC period in Monaco, or the amateur door closing on Massa in Spain were really bad. He’s only as high as he is because Alonso continually rates well in the model. Alonso is that high only because he gets a 6th and then had less counting races to spread the points from that good result over. Alonso had 4 retirements after 80% of race laps run this year, most while running outside the points. Perhaps others had similar, but I doubt it.

    Ericsson is another that snags a ranking higher than what any reasonable person would have assigned him, based on his results.

    I think any model should hone in on the “usual” performance of the drivers, by taking the middle 80% (or whatever percentage you want), and discarding both the best and worst 10% results. Or weighting the “middle 80%” as 50% alongside a 50% weighting for all the results. That should mitigate the situations where a team’s car is especially suited to a certain track, and 1 driver gets a great result while the other has a mechanical DNF (like Stroll & Massa in Baku).

    Perhaps also weighting more recent years higher than the results from seasons over a decade ago, would make sense for a “current” ranking.

    There’s a perfect ranking system out there somewhere.

    1. I don’t agree that subjective consensus implies correctness. I think that has been highlighted a number of times in previous articles.

      Having said that, we shouldn’t expect any model to always be correct; moreover, one in which all the results feel correct to an expert is likely incorrect, since experts themselves are liable to bias. Identifying the anomalous cases, where even I think the model is at fault, is certainly the path to iterative model improvement. Indeed, much of this article is devoted to discussion of just that, e.g., in the cases of Stroll/Massa and Sainz/Kvyat.

      I don’t agree regarding Vandoorne or Ericsson. I think they both have been quite underrated by experts and most fans this year. Alonso’s a higher benchmark than I think people realize.

      1. The thing is that humans can make judgements that your model simply can’t. The team principal’s likely factored in the pressure of the title race into their decisions. Obviously performing under that pressure, when a championship is at stake, deserves higher marks than driving in a midfield car where the battle is simply to beat your teammate. Of course, only a few are under that sort of pressure in any given season, and mainly because they have a competitive car. We cannot know how Sainz would’ve performed in the heat of a title battle. It’s clear that the mistakes of his I mentioned initially, would be put under a far greater microscope, if he was in a title fight. Sainz instead gets the benefit of being judged for what he is: an up-and-coming driver, but one that’s still raw and prone to mistakes.

        As for Alonso, he is definitely an all-time great, and a solid benchmark. However, it’s a bit of a running joke now as to how he ranks so highly year-on-year in your model, by one of the posters above on another F1 forum. Discounting mechanical DNF’s should serve as protection from harshly hurting a ranking; it shouldn’t serve as a benefit to that ranking, which I believe it does with Alonso. If you included the positional points for Alonso, for the races where he retired after 80% of the race, it would bring down his ranking. Alonso ran 15th at Monza, from laps 34-50, then retired the car with 2 laps to go. He wasn’t going to finish higher than 15th in that race. How should that be treated as equal to Hamilton dominating the entire weekend at Malaysia ’16, and then retiring from the lead with engine trouble? It shouldn’t. StatsF1.com has position by lap for races going all the way back to the start of F1, so why not use that? It goes back to the building blocks chosen … if you put garbage in, you’re gonna get garbage out.

        Vandoorne at 7th, and Alonso 3rd, ahead of both Hamilton & Vettel, are both simply bizarre. F1 Fanatic’s Keith Collantine is doing his rankings currently, and has Ericsson at #18, Stoffel at #14, and Sainz at #10. Those are sensible positions for those drivers, considering the seasons they’ve had. Even with Vettel’s mistakes at crucial times this year, I’ll be surprised if he has Alonso ahead of him. The highest Alonso will get in that ranking will be 4th I would bet, behind HAM, VET, VER, but perhaps ahead of RIC. Let’s remember that Alonso was trailing Vandoorne in the standings after 90% of the season … this despite Vandoorne being a rookie, and Alonso getting any upgrades first. Again, that’s a calculation any human can make, that the model can’t.

      2. Humans can take into account other factors, but different humans will weigh those factors differently, and the average of many experts may still be a biased or inaccurate answer. Simply taking more factors into account does not imply a superior model.

        I don’t agree with your other views. I think you are applying circular logic (if a model doesn’t agree with what you have decided is correct, it can’t be correct). But this discussion is not new ground for us.

      3. I believe you should think about how you rate performance yourself, when watching an F1 race. Then try to incorporate that into the model, to remove your subjective biases from it. The “middle 80%” is a good idea, which should capture the “usual” performance of drivers, and protect from a horrible race harshly bringing down a ranking, or a fluke result (or the case of one driver snagging a great result at a track suited to their car, while their teammate DNF’s at the same race) inflating a ranking where it’s not warranted.

        As a follow-up, F1 Fanatic did have Alonso #4, though ESPN has him at #3 (ahead of VET in that one).

        If you were to rank the drivers this year, you yourself, who would be your top 5?

      4. Incorporating how I personally rank drivers’ performances is not a good way of minimizing my personal bias. The best approach in terms of maintaining objectivity is building minimalistic models (i.e., minimal number of parameters) and only adding new parameters as they are shown to be of additional useful information (e.g., age and experience in the latest model). That’s the approach I have taken, and I expect to iteratively improve the model in this fashion in future.

        If I was to purely subjectively rank the drivers myself this year, the top four would certainly be between Verstappen, Alonso, Hamilton, and Vettel. I’m not sure how I would order them, as they were so close in performance. I think Hamilton and Vettel had the most bad weekends of those four, but they were also in a championship fight, as you point out.

        Ricciardo would likely be number 5 in my list, and Sainz in number 6. Then some ordering of Hulkenberg, Vandoorne, Perez, and Bottas in positions 7-10. I would not rank Wehrlein or Ericsson quite as highly as the model does — those two and Sainz are not well connected to the driver population, so their rankings are quite uncertain at present (and I think slightly too high).

      5. Alan Wong · ·

        @KRB I think it’s more of a joke that people’s opinions on “driver performance” wildly oscillate year to year when most of the performance is down to the variations in car performance. Why even bring up the point that alonso being highly ranked in the models is a “running joke”? Is it unreasonable to expect that a driver who has consistently great still be great, in spite of the car’s performance? Doesn’t that stand more to reason than assuming alonso’s performance fell off because he’s now racing in a deficient car with poor reliability?

    2. The most obvious retort is to make your own ranking system if you so wish…

      I don’t think it’s beneficial to use current positions when a driver retires, for all sorts of reasons, not least because they might be out if sync on pitstops. You could take that into account, but what happens if it rains after they retire and the order gets totally flipped anyway? Or what happens if someone has an engine failure in qualifying and another on lap 1? Or if they have a problem that steadily gets worse? It doesn’t really solve anything, it just adds another variable and leaves things more open to subjective analysis.

      As for Alonso, the model punishes him doubly for Singapore! (Denied a great result and gets a DNF through no fault of his own.) The fact that Vandoorne is still a rookie does mean Alonso’s score here is more variable than most though.

  14. andrewthun · · Reply

    Thank you for sharing this seasons analisys. For me the standout drivers were Alonso, Sainz, Hülkenberg, Hamilton, Ricciardo and Vettel, not neccessarily in this order, but except for Hülkenberg, I can see all of them where I thought I would. Vettel’s efficiency for me is however a bit controversial. He caused a lot of trouble, made so many mistakes, but most of the time recovered a huge load of points at the end of the race. I don’t think he would have been a worthy champion this year, but he delivers exactly that point-maximizing performance, regardless of what he has done earlier in that particular race harming his and others chances. My other concern is Massa, who might be well over his peak, but I felt that he had a particulary strong year, and lost so many points due to being taken out, due to mechanical failures, due to handled by his team as a test dummy in tricky conditions to measure if the track is dry/wet enough to change to the appropriate tyres that Stroll would be benefited from. It’s really a joke that Stroll ended up almost with the same points tally as Massa. However, Stroll was also able to maximize his scorings despite constantly lagging behind Massa, and F1 is (unfortunately?) still about bringing home the points.

    Looking forward for the detailed explanation of the age and experience effects.

  15. Fascinating stuff. As you mentioned a lot of drivers are close on ppr this year, and that’s probably even more the case in all time driver rankings. I imagine with the improvement of the model, the ranking will keep shuffling. Any chance of getting a glimpse of current per-year driver ratings for the past seasons?

  16. Joshua V. · · Reply

    I’ve been reading these for a while now, you have a very interesting website. One thing that makes me wonder is what the GOAT list would look like if it were subjected to updated model and the new data for present drivers, or would that only really affect Fernando Alonso?

  17. I must admit I have no idea how the ppr works, given the non-linearity of points given, especially with the extended points system you use. Shouldn’t Stroll’s ranking above Massa, who clearly dominated him over the course of the season, serve as an immediate red flag indicator that surely something can be done to improve the system?

    1. It is discussed in the article as a robustness issue, so yes, I clearly agree.

      It’s not an issue specific to this model. It will appear in any model or ranking system (including the existing WDC system) that rates performance based on points scoring.

      1. It makes sense that the issue would always appear, I was just wondering if the current points system is the best choice. (it might be, I don’t know) For example, I’ve tried entering 20 pts for a win all the way to 1 pt for a last place, counting Stroll’s Monaco classification as retirement, and got 11,65 points for Massa and 10,07 for Stroll.

      2. There is definitely scope for improving points systems in motorsport. This is actually a question I’ve recently been investigating, so I’ll hopefully have a post on that in the near future.

  18. I look forward to reading it. What I had in mind was just changing the points system used when modelling driver performance. The nonlinear nature of F1 points system favors higher points finishes which, I imagine, is meant to encourage drivers to take more risk. Stroll almost outscoring faster Massa in F1 largely on the account of his one podium finish would thus indicate the right points system was chosen for that purpose. There’s no avoiding the occasional paradox, but I imagine a points system used when ranking drivers should be more reflective of their head to head score, which stands at 26-6, if google hasn’t deceived me.

  19. Talking about halo now driver height and weight seem to be big factors. It’d be great to see some stats about these, or even include them in the model.
    Great posts btw!

    1. Cool idea! I’ll try to explore that in future.

  20. […] de la fiabilidad. Un análisis matemático le puso como mejor piloto del 2016, y el tercero de este año (claro que es el mismo modelo que dice que, en su cénit, Heinz-Harald Frentzen fue mejor que […]

  21. […] was one of the weaker drivers on the F1 grid; my performance model rates him outside the top 15 in all years of his career. In 2004 he finished only 8th in the […]

  22. Thanks a lot for the post.
    I was wondering, do you publish your data somewhere?

    Cheers

  23. […] In this article, I will first provide an analysis of the times set during testing, focusing on what can be derived from long runs. I will then provide a preview of teammate battles for the season, making predictions using my driver ranking model. […]

  24. I may have misunderstood how the season ratings are worked out, but my impression is that it is an updated career peak ranking factoring in the new information. If this is the case, would it make more sense to rate a season by how much it has changed the driver’s career peak, rather than by the new peak itself?

    1. Hi Adam. The way the model works is that it computes a season performance value for each driver for each year of their career. This season performance estimate depends on the driver’s estimated peak ability, age, experience, as well as any year-to-year fluctuations in performance. Each year, as new data come in, there are updates to each driver’s estimated peak ability, but the rankings I present are specifically performance in the current year.

      You could indeed rank drivers within a season by how much their estimated peak ability went up or down, and that would be interesting. The interpretation would be very different, however. The drivers who are going to see the biggest changes, typically, are inexperienced drivers, since their career estimates are the most volatile. So you may end up with something like Gasly right at the top, Hartley right at the bottom, and all the most experienced drivers somewhere in the middle of the list.

      1. Thanks Andrew – if these are season specific that makes complete sense!

  25. […] (i) The adjusted driver performance rankings (in ppr) for each season, including the driver’s predicted hypothetical performance at that age and with that level of experience. This is a ranking of absolute driver performances, taking teams out of the equation. In other words, how the model predicts the drivers of that season would have relatively performed in equal machinery. This is the same approach I use for my end of season driver rankings each year. […]

  26. Can you post a graph showing the performance of the three one time WDCs (Button, Raikkonen and Rosberg) similar to the one you posted on the Massa entry? Quite interested in seeing that. Also, how is the ranking effected after the results of this season? Sainz and Vandoorne, for example have underperformed quite a bit.

  27. […] to the second part of this five-part series, in which I apply the f1metrics model of driver and team performance to simulating historical hypothetical situations. Consider this a quantitative approach to tackling […]

  28. Francisco Cansado Carvalho · · Reply

    Thank you

  29. […] model had misplaced certainty (90% likelihood) that Sainz would beat Hulkenberg. As I explained in last year’s report, Sainz appeared to have an inflated rating due to a lack of teammate connections out of the insular […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: