Seeding Inaccuracy on the PPA Tour

Due to the influx of new talent on the heels of the MLP/PPA merger, seeding inaccuracy has jumped by over 30% in 2023. Comparing seed inaccuracy of PPA veteran teams to former APP teams reveals a significant and widening gap with former APP teams under-seeded and PPA veteran teams increasingly over-seeded.

Even if you’re not going to read it all, scroll down to look at the pretty graphs.

The damage of the inaccurate seeds is pervasive, costing former APP and PPA players real dollars and reducing the quality of play in quarterfinals, semifinals, and finals at each event.

We propose a number of solutions that could nip this growing problem in the bud. Given the unique circumstances of the MLP/PPA partnership, finding a way to give these transferring players more accurate seeds just makes sense.

Without any changes, we must instead wait until mid-2024 or 2025 for a natural mathematical equilibrium, meanwhile the tour—as great as it has been this year—will continue to underperform its full potential.

Intro

On November 9, 2022 the MLP/PPA merger rocked the pickleball world, shifting the landscape of pro pickleball. After a year of working with the APP Tour, MLP agreed to partner with the PPA Tour bringing some of the world’s best players back to the popular team-based events.

However, the shift put many APP-affiliated pro players in a predicament. With little time and thin on details, they had to make a big decision. If they wanted to participate in MLP (Major League Pickleball), those eligible for a PPA Gold Card had to commit primarily to the PPA Tour for twice the prize money, or partially to the PPA Tour for 50% of the prize money. The lion’s share of eligible players chose to throw their lot in with the PPA Tour—many of whom had predominantly played APP events in 2022.

Among all the difficulties, one downside these players may not have considered was the uphill battle that awaited them in terms of seeding.

On his Tennis Sucks podcast, Travis Rettenmaier expressed concern over the inherent inaccuracy of the seeds this year on the PPA Tour:

“It’s not that I don’t like it, I just think it’s unfair and it’s illogical.”

But does the actual data back up the intuitive sense that there is a problem here? Read on to find out!

Why Seeding Matters

According to ChatGPT, the purpose of seeding players in a sports tournament is to create a more balanced and fair competition. Seeding involves ranking the players or teams based on their previous performance, skill level, and other factors, and then placing them in the tournament bracket accordingly. This helps to ensure that the strongest players or teams do not face each other too early in the tournament, and that the best players or teams have a greater chance of making it to the later rounds.

Accurate seeding is good for everyone. Veteran tour players can ease into their day to some degree without having to worry about facing their strongest competition until the later rounds. Fans can be more assured they will get to see their favorite pros competing and not getting knocked out in the first round. Broadcasters can be more assured that the best, most popular players will be competing against each other in the later rounds, ensuring the satisfaction that not only did the best team win, but the highest-level, most entertaining play occurred in the finals or semifinals.

Questions to Answer

Now halfway into the new pro pickleball landscape of 2023, we were curious how things are going. Here are some of the questions we will answer:

• How accurate are the PPA Tour seeds this year compared to last year?
• Is there a gap in the seeding inaccuracy of PPA veteran teams compared to former APP teams? Is it growing or shrinking?
• What about the seeds for “hybrid” teams with all the new partnerships between PPA veterans and former APP players?
• How accurate are the seeds for teams who are/were unaffiliated with a specific tour last year?

The Data

We analyzed over 1,700 seeded teams across 21 events from the past 13 months of PPA tournaments. We then compared their seeds with where they ended up finishing in the tournament.

If a team seeded 10th finished 5th, then they were under-seeded by five (-5). If a team seeded 1st finished 4th, they were over-seeded by three (3). If a team seeded 11th finished on the 9th-12th plateau, then we consider that they were accurately seeded (0).

The Results

Combining both over-seeding and under-seeding into a single metric, using absolute values, yields the following graph of PPA doubles seeding inaccuracy over time. Prior to the announcement of the MLP-PPA partnership, it hovered around 3.0, meaning that on average, the seeds were inaccurate by an average of three spots in either direction. However, since the MLP/PPA merger, seeds have become increasingly inaccurate, ballooning to an average of more than four spots off.

Compared to the last half of 2022 (3.1), PPA Tour seeds in 2023 are off by 4.1 spots per team on average—31% less accurate.

PPA vs Former APP

Given the influx of new APP talent—many of whom rarely or never played PPA events in 2022, we could logically expect PPA veterans to be somewhat over-seeded and former APP players to be somewhat under-seeded, and this is indeed the case.

Former APP teams are out-performing their seeds (under-seeded), while PPA teams have been under-performing their seeds (over-seeded) by much more than usual.

It is important to note, that this graph is not an indication of how good the respective teams are, but only how well they are performing when compared to their assigned seed.

A Growing Gap

Not only have PPA teams been over-seeded in 2023, but they are becoming more and more over-seeded as the year progresses, while former APP teams have been consistently hovering at around 2.0 spots under-seeded.

Isolating the gap between the two, yields the following graph.

As you can see, not only is there a significant gap, but the gap is growing, not shrinking. While initially surprising, upon further reflection it makes sense for the data to show this. Seeding accuracy as calculated for this article will apparently get worse before it gets “better.” We believe there are a few reasons for this:

• Seeds for APP teams in early 2023 were clearly worse than they are today, however the poor seeds resulted in poor results overall as former APP teams faced difficult PPA opponents in early rounds.
• Many APP teams were still quite new and uncomfortable as they settled into their new life with the PPA Tour early on
• The top PPA teams feature some of the best players in the world, and it takes time to adjust to facing that level of speed and intensity on a regular basis and start posting improved results.

Despite the caveats, it is alarming that this gap is expanding (as shown by the trend line). On a long enough timeline, we should eventually achieve equilibrium and consistently accurate seeds, but the current trajectory suggests a longer timeline than we might have otherwise guessed. Not just one year (the period for PPA points rankings), but potentially much, much longer before the ill effect of these seeding inaccuracies are laundered out of the system.

More Detail

We can also layer in the seeding inaccuracy for “hybrid” teams (those with one Former APP player and one PPA veteran) as well as teams of unknown affiliation (primarily new or local pros giving it a shot).

This chart provides a fuller picture of the dynamics at play with seeding teams from these different categories. As we might expect, unknown teams are generally under-seeded since they are less known, and it is easy for them to over-perform by winning just one game. Similarly, we would expect hybrid team seeding inaccuracy to appear between PPA and former APP teams and that is largely the case.

Notably, at most events this year, former APP teams have been even more under-seeded than unknown/unaffiliated teams.

Worse Than It Looks

It is surprising that the early 2023 tournaments don’t show a huge gap in seeding inaccuracy between PPA and former APP teams. While confusing at first, consider the following example…

Imagine a fictional team arrives on tour. No one knows them yet, and they’ve never played professionally, but let’s suppose they are truly the 8th best team on tour from day one and they never improve or decline. Even though they are the 8th best team they will not get the #8 seed until they have earned that spot via the points system.

Event One, Seed #32: They face the #1 seed and lose as expected in the first round. They manage to win two games in the backdraw earning a few points and landing on the 13th-16th plateau. ($0)

Event Two, Seed #31: They face the #2 seed, losing in the first round. They win just one game in the backdraw, landing on the 17th-24th plateau. ($0)

Event Three, Seed #30: They face the #3 seed, losing in the first round. They win three games in the backdraw, including one upset, landing on the 9th-12th plateau. ($0)

Event Four, Seed #27: They face the #6 seed, losing in the first round. They win four games in the backdraw, finishing 7th-8th. ($0)

Event Five, Seed #22: They beat the #11 seed in round one, losing to the #6 seed in round two. They win two games in the backdraw, losing to the #5 seed, landing on the 9th-12th plateau. ($550)

Event Six, Seed #19: They win three games, knocking off the #3 seed and making the semifinals, taking 4th place. ($2,200)

Event Seven, Seed #15 …

After seven tournaments in this contrived but plausible example, the team we KNOW is 8th-best on tour, still only has a #15 seed. We can see that getting results that reflect your skill level is difficult, takes time, and the deck is stacked against you. Despite being under-seeded relative to results in each tournament, they are not able to consistently land on their true 7th-8th plateau since they are consistently facing top-seven seeds early in the main draw and backdraw.

In those first three events, comparing their seed to the results tells us that they were under-seeded by more than 17 spots on average (-17.5), however, we know as the 8th-best team, that they were truly under-seeded by 24 spots on average (-24).

A Problem Until 2025?

This is the uphill battle that awaits pro pickleball players at the highest level, and it is a battle former APP players are currently fighting for the second time in their careers. PPA points go back one year (according to their website), but as you can see, the negative impact of poor seeds will linger for many months longer than that due to the headwind of poor seeding early on. That means the former APP players’ second seeding battle will last well into 2024…or longer?

On the Tennis Sucks podcast, Travis stated that PPA points go back 18 months, and co-host Graham confirmed this. Though we cannot find this on the PPA website (or elsewhere), it is possible that pro players have access to more detailed documents than the rest of us. If true, seeding inaccuracy for many players will be mathematically impacted until 2025.

Striking Where It $tings

PPA prize money payouts are based primarily on how deep you go in the main draw. According to the PPA website, the only prize money available in the backdraw is for the winner of the backdraw (5th place). For the purposes of assessing seeding accuracy, we are taking into account the performance of teams in the backdraw. However, teams don’t get paid for wins in the backdraw unless they win all backdraw games, earning 5th place. Thus, facing a top-5 team too early in the main draw due to seeding inaccuracy is even more detrimental in terms of prize money.

As the true 8th-best team in our example, they should, on average, make the quarterfinals, with a payout of $1,100 for a typical event. However, due to the tough draw, they went home empty handed until they finally broke through in the last couple events.

Add to this the fact that many sponsorship contracts also include bonuses for where pros finish in the tournament, and the real cost to players goes beyond just the lost prize money.

Staksrud/Tellez

Take the example of Staksrud/Tellez—both Gold-Card-committed former APP players. In our opinion, they are at worst the 5th-best team on the PPA Tour right now, and with all the top teams present would be best seeded at #5. Both individuals played in a handful of PPA events in 2022, so luckily for them, they didn’t come into the season completely empty-handed in terms of PPA points.

Jan 15: Masters, Seed #20: They beat the #13 seed, but lose to the #4 seed, landing on the 10th-24th plateau (backdraw suspended).

Feb 5: Desert Ridge Open, Seed #23: They win two games, losing to the #2 seed in the quarterfinals. After winning one more game in the backdraw, they lose, finishing 7th-8th.

Feb 19: Arizona Grand Slam, Seed #13: They win one game, losing to the #4 seed in the second round 11-9 in the third game. Winning three games in the backdraw, they again finish on 7th-8th.

Feb 27: Indoor Nat’l Championships, Seed #11: They win four games in the main draw, knocking off the #3 seed and losing the finals 11-7 in the third game, finishing 2nd.

Mar 12: Florida Open, Seed #10: They lose two in a row, finishing on the 25th-32nd plateau. While they typically play better, it is notable that they were pretty unlucky drawing two significantly under-seeded teams.

Mar 19: Austin Showdown, Seed #9: Two wins, then a loss to the #1 seed send them to the backdraw where they win out, finishing 5th.

Apr 8: Red Rock Open, Seed #8: Two wins, then a loss to the #1 seed. They battle through the backdraw winning 5th place.

Apr 23: Newport Beach Shootout, Seed #11: Three wins, knocking off the #6 & #3 seeds, on their way to a semifinal loss to the #2 seed before winning the bronze match, finishing 3rd.

May 7: North Carolina Open, Seed #7: Two wins, then a loss to the #2 seed in the quarterfinals sends them to the backdraw where they knock off the #3 & #4 seeds, winning 5th place.

May 21: Atlanta Open, Seed #7: One win then, a tough loss to the #10 seed (another underseeded, former APP team) 12-10 in the third game. After making a run in the backdraw, they lose to the #11 seed, finishing 6th.

Jun 4: Texas Open, Seed #9: Two wins, then a loss to the #1 seed in the quarterfinals sends them to the backdraw where they win out, again taking 5th.

Eleven events into the season, and Staksrud/Tellez are still consistently under-seeded. In their case it appears to have had a direct negative impact on their results for at least the first three events of the year. Without the breakthrough silver medal in late February upsetting the #3 seed 11-5 in the third game in quarters, they would probably still be suffering from double-digit seeds.

Just this past weekend, as the #9 seed, they faced the best team in the world—the #1-seeded Johns brothers—earlier than they should have in the quarterfinals. Staksrud/Tellez ended up losing in three, but clearly gave the Johns brothers their toughest competition of the tournament as the only team to win a game against them (11-3 in the second).

Not only does facing the #1 seed so early cost them money and points, but there is a trickle-down effect—the #8 seed should’ve had a softer opponent for a chance to make the quarterfinals, also costing them both money and points. In addition, the Johns brothers shouldn’t have had such a stiff quarterfinal. Staksrud/Tellez under-seeded at #9 is good for no one.

Bad for Everyone

What is the impact of all this? Certainly it would feel insulting to be severely under-seeded as a multi-year professional, but are the former APP players the only ones negatively impacted by this year’s seeding inaccuracies?

No.

Inaccurate seeds are bad for everyone—PPA veterans, former APP players, new up-and-coming pros, fans, broadcasters, and the PPA Tour overall.

As a PPA veteran top-10 team, facing another top-10 team masquerading as a #23 seed in the first or second round is not helpful and could prematurely send you home without a dime. Even if you win, having an unnecessarily exhausting match out of the gates could put you at a huge disadvantage.

Not only that, but it also means that at least one true top-10 team gets knocked out earlier than they should, reducing the quality of competition, excellence, and skill left for the semifinals and finals (bad for fans, broadcasters, PPA Tour, pickleball in general).

While the impact of inaccurate seeds may be more obviously damaging to former APP players, at the end of the day, no one wins.

Conclusions and Solutions

The injection of new talent to the PPA Tour this year has been a game-changer—creating new, exciting partnerships, matchups, and all-out entertainment. Given how often PPA tournaments “went chalk” in 2022, it’s understandable that the PPA Tour may be a bit hesitant to act.

How could this chaotic seeding problem be fixed?

On his podcast, Travis Rettenmaier suggested that the time-window for seeding based on points could be shortened or DUPR ratings could be used to re-seed players more accurately.

In addition to those ideas, more come to mind:

• Accept newly contracted players’ 2022 APP points on some multiplier (say 60-80%)

• Re-seed newly contracted players based on Pickleball World Rankings points

• Expand any newly contracted players’ 2022 PPA points (if any) as though they had entered more tournaments

• Appoint a small committee of statisticians, tour officials, PPA veteran players, and former APP players to recommend a viable new solution

The heart of any sport is the excellence of the world’s best players—for pickleball, this is the PPA Tour. As guardians of pickleball, protecting the sport and its best players from the pervasive damage of inaccurate seeds is worthy of time and attention for the sake of all involved. Given the unique circumstances of the MLP/PPA partnership, finding a way to give these transferring players more accurate seeds just makes sense.

Without any changes, we must instead wait until mid-2024 or 2025 for a natural mathematical equilibrium, meanwhile the tour—as great as it has been this year—will continue to underperform its full potential.

Real Clear Stats (RCS) is a leader in Pickleball Stats collection, communication, and development capable of tracking 250 individual players stats live in real time with an eye toward broadcast integration.
In addition to sharing stats from occasional major matches on the tours, RCS meticulously tracks every hit at MLP events and aggregates the data to fairly compare all 48 players both in Premier and Challenger Levels.
RCS also works with a growing list of MLP teams to help them gain strategic advantages during events as well as assistance when drafting or trading players.