2024 Update:
This post incorporates race data from the 2022 and 2023 seasons. It has been thoroughly rewritten and now features new, consistent graphs1 compared to the previous version from June 2022 (Analysis: Super-Shoes Have No Impact On Elite Men’s Race Times). For further background on the debate, see discussions on LetsRun.com from July 2022, September 2023, and January 2024.
Jan. 25, 2024 Update:
LetsRun user found a bug in the Python script, see Tech Notes below.4
Racing and training in carbon-fiber plate and foam ‘super-shoes’ has been a rewarding and undeniably faster experience for the author. However, their use has sparked considerable controversy in competitive running circles. These shoes are often credited with giving runners an unfair advantage over competitors who do not use them, complicating comparisons between contemporary records and those from the past. A key challenge in interpreting statistical race data for signs of such an advantage is that average race times generally improve over time due to a variety of factors. This study aims to visually contextualize these improvements alongside the expected gains from using super-shoes, thereby assessing the true impact of this innovative footwear technology.
Numerous studies and anecdotal accounts show runners have achieved major gains in running efficiency and speed when wearing super-shoes. Naturally, concerns about the integrity of the sport have mostly centered around the record-breaking performances of elite competitors, rather than the marks being set at one’s local 5K and similar events. Consequently, this analysis focuses on the top 100 runners each year in long-distance events, rather than on broader runner categories. These are not the top-100 times, but the best time in a given year of the top-100 runners in their particular events. As winners and top finishers of their respective races, super-shoes should have provided a massive boost to all who were wearing them, with an effect on times that should be easily discernible in yearly statistical comparisons.
This analysis reveals that elite men have shown minimal improvement in race times beyond the existing trends since the introduction of super-shoe technology in 2017, except possibly in the Road 10K event. In contrast, for elite women, there has been a notable decrease in times across all four events, aligning with the projected 2% performance increase attributed to the shoes. These projections are based on a straightforward ‘best fit’ linear regression, which measures the prevailing trend since 2001. It’s worth noting that alternative trend projection methods might yield different outcomes. Despite the significant results for women, the discrepancy with the Men’s results (apart from the Road 10K) casts some doubt on the overall impact of super-shoes on race times.
Data Sources
The events studied for the years 2001-20232 were the: 10,000 Meters, 10km Road race, Half Marathon, and Marathon, based on data from World Athletics. For each event, the average time of the top 100 runners was calculated for each year and placed on a graph, illustrating the progression of times before and after the adoption of super-shoes in 2017. (Note: There were a few instances of super-shoes appearing before 2017, but their use did not become statistically significant until that year.) Based on the sources outlined below, the percentage of the top-100 runners wearing super-shoes was estimated to as follows:
For women:
11% in 2017
38% in 2018
59% in 2019
88% in 2020
92% in 2021-2023
For men:
13% in 2017
45% in 2018
65% in 2019
88% in 2020
92% in 2021-2023
The initial estimates for the first three years were derived from a 2020 Cornell study, ‘An Observational Study of the Effect of Nike Vaporfly Shoes on Marathon Performance‘. This study meticulously examined race photographs of 578 runners across 22 well-known marathons in the US and Canada, recording their times and the types of shoes they wore. They found that roughly 9%, 31%, and 47% of women runners were wearing Vaporflys in 2017, 2018, and 2019, respectively, compared to 12%, 41%, and 59% for men. Naturally, shoes worn by the top-100 distance runners globally would have differed significantly from those worn by the mixed group of American elite and amateur runners in the Cornell study. To gauge this difference, I examined race photographs of the top 100 women in the 2019 marathon event. Out of the 87 identifiable, 51 were wearing super-shoes, equating to 59% — about a quarter more than the 47% found in the Cornell study.
A second major source was a Running World article titled ‘What were people wearing on their feet for the World Half Marathon Champs?‘, which precisely counted the shoes all 116 male competitors at the World Athletics Half Marathon Championships in Gdynia, 2020. The researchers estimated that approximately 90% of the men were wearing carbon-plated shoes. Given that a few might not fall under the category of ‘super-shoes’ as defined here,3 the estimate was adjusted to 88%. It was then assumed that the percentage of women wearing super-shoes had reached similar proportions by 2020, and that the composition of athletes in the top-100 mirrored those at Gdynia. Consequently, for 2021, the usage of super-shoes among the elite was assumed to have slightly increased to 92%, and to have remained steady at that rate through 2022 and 2023. These estimates, based on various measurements and spot-checks of other sources, are intended to be conservative. As long as the percentages of elite runners wearing super-shoes are not significantly lower, the conclusions should remain valid.
The Cornell study also sought to quantify how much a runner’s time might improve after switching to Vaporfly shoes. While the range varied considerably among individuals, on average, they found that men’s times improved by about 2.1% (±0.7%) and women’s times by about 1.4% (±0.8%). However, these figures are at odds with other studies suggesting that women benefited more, not less, from super-shoes. For instance, Senefeld’s 2021 study, ‘Technological advances in elite marathon performance‘, discovered that the “the magnitude of improvements in performance for males (2.0%) and females (2.6%) are similar to the 2% faster performance predicted using models based on metabolic savings in running.”
If the performance advantage of super-shoes is only 1.4% for women or lower, it becomes easier to attribute any reduction in times to the shoes. Conversely, if the performance advantage for men is not 2% but rather 2.6% (which would be astronomical), it becomes challenging to reconcile these modest performance gains with the predictions of shoe technology. The original research on super-shoe performance by Wouter Hoogkamer et al. in 2017, concluded that : “a 4% average energetic savings observed should translate to ~ 3.4% improvement in running velocity at marathon world record pace (20.59 km/h).” For simplicity, and because the 2% figure is most frequently cited by critics, the following graphs for both men and women have been calculated using that figure.
The Graphs
Each graph can be interpreted in four main ways:
- The Primary Blue Line: This line, marked with diamond-shaped points, is simply the average time of the top-100 athletes worldwide in each event for the indicated years.
- The Black and Red Trend Line: This is a ‘best fit’ linear regression based on the times from 2001 to 2016, differing from the previous analysis which started the trend line from the slowest year. This new approach may not capture the latest trends as precisely but offers greater consistency across graphs. The red segment illustrates the projected trend post-2016, continuing the slope established in the earlier years.
- The Orange Line: Marked with star data points, this line indicates the hypothetical average times from 2017 to 2021 if the 2% performance increase attributed to super-shoes were subtracted from racers’ times. This adjustment accounts for the increasing percentage of runners wearing super-shoes each year. If super-shoes had a significant impact, the orange line should align closely with the red trend line after 2016.
- The Green Triangle: This marker represents the projected average time for the top-100 runners in 2023, calculated based on the red trend line and assuming that 92% of runners were wearing super-shoes. If super-shoes were providing a substantial boost, the expectation would be for the blue line to intersect the triangle or at least approach it closely.
Technical Notes:
1The scale of the Y-axis is now consistently set at 12% (±6%) of the average time for each event. This standardization allows for accurate visual comparisons between graphs, and each block in the grid represents 30 seconds. The source code used to generate the graphs is available in the ‘Graphing: Python Source Code’ section below.
2The data for the years 2001-2021 were initially obtained from World Athletics in 2022. Since then, these lists have been updated by World Athletics as athlete performances and event results continue to be reassessed for inclusion in the top rankings. For instance, in the Women’s Road 10K for 2021, approximately five of the top 100 runners had changed by 2024, resulting in a new average time about 5 seconds slower. Similarly, in the Men’s Road 10K for 2021, about ten runners were replaced, leading to a new current average about 2 seconds slower. These time differences do not significantly impact this analysis. The alterations seem largely due to certain races being excluded from the category (e.g., Madrid’s San Silvestre Vallecana 10K), rather than issues like doping violations. It’s anticipated that the results for 2022-2023 will also undergo future revisions.
3‘Super-shoes’ are defined here as any carbon-and-foam running shoe demonstrated in lab settings to enhance running efficiency. This category includes primarily Nike’s Vaporfly and Alphafly models, Asics’ Metaspeed Sky, and to a lesser extent, Adidas’ Adios Pro models, Saucony’s Endorphin Pro models, among others.
4LetsRun user “Hard2Find” found a bug in the Python script which was calculating the trend line on data points from 2001-2023 instead of 2001-2016. The graphs have been regenerated below. The glitch did not affect the main “Average Times” line or “Subtract 2%” line, but it did change the slope of the trend line and position of the green “Projected 2023 Time”. For the Men’s events, there is very little difference, but for the Women’s events, it places their 2023 times for all events right in line with a 2% performance increase, instead of merely approaching it, which is noteworthy.
Graphing: Python Source Code+
Consider first the men's performances:
Men's 10,000 Meter & 10K Road Races
In the 10,000 Meters event (on the track), the actual average time in 2023 was 27:34, aligning closely with the trend predicted before the widespread use of super-shoes. This similarity makes it challenging to attribute the improved performance solely to the advent of super-shoes. Additionally, the orange line, which models men's performances without the super-shoes' advantage, displays a significant increase, deviating from the trend observed in the years preceding their introduction.
In the Men's 10K Road race, the trend to 2021 was similar to the Men's 10,000 Meters. Yet, the average times in 2022 and 2023 decreased notably, not quite reaching the 2% improvement indicated by the green triangle, but still representing a major reduction in times. As observed in nearly all the graphs, there was a noticeable spike upwards in 2020, largely attributable to the numerous race cancellations during the pandemic and the resulting smaller pool of recorded times.
Men's Half Marathon
In the Men's Half Marathon, the average time in 2023, projected by the trend from 2001 to 2016, nearly matched the actual observed time of 59:44. This finding contrasts sharply with the 58:30 time projected by the super-shoe model, suggesting that the influence of super-shoes may not be as pronounced as anticipated. Notably, the average time dipping below 60 minutes in 2022 is a significant milestone. However, this achievement aligns directly with the incremental improvements observed over the past twenty-one years, rather than indicating a sudden leap in performance attributable to super-shoe technology.
Men's Marathon
In the Men's Marathon, the observed improvement of approximately 10 seconds per year aligns with the 5 seconds per year enhancement seen in the Half Marathon. The 2023 result of 2:05:27, though lower than the predicted trend line, falls significantly short of the 2% improvement benchmark, suggesting only a modest impact of super-shoes on marathon performance.
Considering the historical data, it becomes challenging to identify a significant or seismic shift in men's distance running times in the seven years following the widespread adoption of super-shoes. The evidence suggests that while there may be marginal improvements, they do not constitute the groundbreaking changes one might have expected from such technological advancements.
The overall Women's race times exhibit considerably greater improvement and variation compared to the Men's times:
Women's 10,000 Meter & 10K Road Races
In this updated analysis, the starting point of the trend line is now anchored at the data from 2001 instead of 2010. This adjustment results in a flat slope, making the super-shoe projected time more attainable. Nonetheless, the women's performances in the past three years are remarkable, reaching a 2% improvement below the established trend.
The trajectory observed in the women's Road 10K race results for 2022 and 2023 is particularly striking, as it extends below the time projected by our super-shoe performance model. This close proximity to the model's projection suggests that shoe technology may have played a role in these recent advancements.
Women's Half Marathon
n the Women's Half Marathon, there has been a remarkable average improvement of nearly 19 seconds per year from 2006 to 2017. This trend continued with impressive times recorded in 2018-2019 and again in 2022-2023 when times were in line with a 2% performance increase. With a range of times (12%) now consistently applied across all graphs, the steep slope of improvement in the Women's Half Marathon is readily apparent when compared to other events. However, despite the rapid progress in recent years, the rate of improvement since the adoption of super-shoes in 2017 is roughly equivalent to that of the preceding decade.
Women's Marathon
In the Women's Marathon, an average decrease in times was observed to be 15 seconds per year through 2016, and the last two years, like the Half Marathon, were exceptional and well within our 2% super-shoe projection. Particularly noteworthy is the dramatic reduction of two and a half minutes in the average time among the top-100 runners from 2021 to 2022. This substantial drop is somewhat perplexing, considering the assumption that super-shoe utilization remained consistent between these two years.
Discussion
It is well-established through multiple studies that carbon fiber plates combined with thick, lightweight foam in super-shoes can improve the metabolic efficiency of runners in laboratory settings. Consequently, one might expect the greatest benefits in marathon events, where managing energy deficits is critical. Yet, while the last two years have shown suggestive trends in the Women's Marathon, the Men's marathon event does not display any clear correlation with super-shoe use. Conversely, in the 10K Road race, which does not deplete energy stores to the same extent as a marathon, we observed significant improvements in both Women's and Men's times, hinting at a possible advantage from super-shoes. Given the widespread claims of universal improvements in running economy due to these shoes, it seems paradoxical that their impact would be event-specific, especially in longer distances. If the effect was as significant as described, it should be distinctly evident in all long-distance races and for all athletes. However, none of the Men's results reached the 2% performance increase benchmark when accounting for the gradual improvements over time that have occurred in all events before 2017. The Women have arguably reached the 2% improvement threshold, although this interpretation is dependent on how prevailing trends are calculated, and there may well be other factors that are pushing them to these extraordinary times.
So, what factors might be contributing to the decrease in times, if not solely the shoes? Several possibilities emerge:
1. Previously, I suggested a potential placebo effect, particularly in the women's 10K Road race and the notable year of 2019. However, the strong results in 2022 and 2023 lead me to reconsider this hypothesis. It now seems less likely that psychological factors alone are driving the decrease in times.
2. A more plausible explanation for the gradual improvement in average times might lie in the increasing number of elite runners participating. A larger, more diverse field inevitably leads to a lower top-100 average time. The pandemic year of 2020 illustrated this point, as a reduced pool of racers correlated with slower averages. An analysis of the nationalities in the top-100 lists from 2001 and 2023 reveals a substantial increase in runners from countries like Kenya, Ethiopia, and Uganda. This influx of new contenders, especially from Ethiopia, could be a significant factor in the downward trend of times.
3. Another consideration is that while super-shoes may enhance running economy (which appears likely), their impact on the elite-of-the-elite top 100 runners may be less pronounced. These athletes, owing to their rigorous training and innate abilities, might not gain as much from improvements in running economy. If true, this outcome would be favorable, allowing amateur runners to enjoy the benefits of better times and potentially fewer injuries through super-shoes, while preserving the integrity of elite competition and historical record comparisons.
4. There is also speculation that individual responses to super-shoes vary greatly, with some runners experiencing remarkable performance boosts while others see only minimal benefits. This variability could lead to significant disparities in outcomes among elite athletes, and perhaps even explain the large gap between Men's and Women's improvements of recent years.
Uncertainties
While this analysis presents a comprehensive overview of the impact of super-shoes on elite running times, several uncertainties could potentially affect its validity, in addition to those already mentioned:
Real-World Performance Increase: There is a possibility that the actual performance improvement afforded by super-shoes in real-world conditions is less than the widely cited 2%. This raises a pertinent question: if the improvement is indeed marginal, does it still constitute a significant advantage or become less objectionable?
Estimation of Usage Among Elite Runners: Accurately measuring the percentage of elite runners wearing super-shoes in races presents a challenge, particularly for the years 2022 and 2023, where the assumption is that 92% of athletes used them. The true figure could be substantially higher or lower, and it also hinges on how one defines 'super-shoes.'
Accuracy of Calculations: The development of a new Python script to generate the graphs for this updated analysis allowed for extensive re-testing and verification of calculations, however, there still remains the possibility of errors or inaccuracies in the computational process.
Race Course and Conditions: Another consideration is the potential impact of the race courses and conditions over the past five years. Variations in these factors could have influenced race times, either slowing down or speeding up performances, and thereby obscuring the true effects of super-shoes.
Further Study
Further investigation into the dynamics of elite long-distance running could yield valuable insights. It should be relatively simple to examine whether the numbers of elite long-distance runners, particularly women, have increased significantly over the last 23 years. This could be accomplished perhaps by examining the registration records at World Athletics, or through anti-doping controls, to understand the correlation between the proliferation of elite athletes and average race times.
Another intriguing avenue for study concerns 'super-spikes' and their impact on shorter track events. Establishing a link between enhanced performance times and shoe technology in these events would require reliable data on the prevalence of super-spikes from 2017 to the present, which seems currently lacking, as well as a laboratory-informed baseline of performance improvement for such shoes.
Furthermore, comparing the results of the world's top-100 runners with those at more local or regional levels could provide interesting contrasts. It's possible that in environments with fewer world-class competitors, the influence of super-shoes might be more pronounced, revealing different aspects of their impact on running performance.
Comments, questions, and corrections are welcome below (moderated for spam).
