Bug Reporting for Games with PvP Matchmaking

PvP matchmaking is the system that pairs players into competitive matches, ideally against opponents of similar skill, and it is one of the most consequential systems in a competitive game, because the quality of the matches it makes determines whether the game feels fair and worth playing. It is also a hard, distributed, statistical system where bugs hide: matchmaking that pairs players badly, that takes too long, that forms unbalanced matches, that handles parties or ratings wrong. Like other server-side systems, matchmaking bugs depend on the matchmaking context and the resulting match. Tracking PvP matchmaking bugs means capturing the players' ratings and parties, the matchmaking decision, and the match it formed.

Matchmaking is a consequential, statistical system

PvP matchmaking takes the pool of players queuing and pairs them into matches, trying to make matches that are fair, balanced, and quick to form, usually based on a skill rating that estimates each player's level. This is a consequential system, since the matches it makes are the core of the competitive experience, and a statistical, distributed one, balancing match quality against queue time across a fluctuating pool of players.

The consequence is that matchmaking bugs are serious and varied: pairings that are badly unbalanced, putting players against far stronger or weaker opponents, queues that take too long or fail to find matches, ratings that update wrong, parties that are handled unfairly, matches that form with the wrong players. These hurt the competitive experience directly, since unfair or broken matches drive players away. Understanding that matchmaking is a consequential, statistical system frames the bug tracking: capture the matchmaking context and the match formed, since a matchmaking bug is about a decision the system made given the players in the pool.

Capture the matchmaking context

The core context for a matchmaking bug is the matchmaking context, the player's skill rating, their party if any, their queue, the region, and the relevant pool conditions, since a matchmaking bug is about a decision the system made given these inputs, and reproducing or understanding it requires knowing what the system was working with. Capture the matchmaking context when a bug is reported.

A report of a matchmaking bug, a wildly unbalanced match, an endless queue, becomes interpretable when you know the player's rating, party, region, and queue conditions, since the matchmaking decision was made from those inputs, and a bug may be specific to a rating range, a party situation, or a thin pool in a region. The matchmaking context is the input from which the matchmaking decision, and any bug in it, emerged. Capturing the matchmaking context is the foundation, providing the inputs the matchmaking system worked from, against which a bad pairing, a long queue, or an unfair match can be diagnosed.

Capture the match it formed

The output of matchmaking is the match, so capture the match it formed, the players placed in the match and their ratings, the team composition, the balance of the resulting match, since a matchmaking bug, especially a fairness or balance bug, is judged by the match it produced, and the match composition is what reveals whether matchmaking did its job. Capture the resulting match when a bug is reported.

A report that a match was unfair becomes diagnosable when you can see the players and ratings in the match, revealing whether the matchmaking genuinely produced an unbalanced pairing and why, perhaps a thin pool forcing a wide rating spread, a party of mismatched players, a rating that was wrong. The match composition is the matchmaking's output, the thing the player experienced as fair or unfair. Capturing the match it formed, alongside the matchmaking context, is what lets you diagnose matchmaking fairness and balance bugs, seeing the inputs and the output together and judging whether the decision was wrong and why.

Watch the parties, ratings, and queue

Parties are a major matchmaking complexity and bug source, since matchmaking a party, a group queuing together, is harder than matchmaking individuals, the party's combined skill, the fairness of pairing a party against solos or other parties, and party-handling bugs are common. Capture the party context for matchmaking bugs involving groups, since the bug may be in how the party was rated or matched.

And watch the ratings and the queue, since the skill rating drives matchmaking and a rating bug, a rating that updates wrong, that misestimates a player, propagates into bad matches, while queue bugs, queues that take too long, that fail to find a match, that form matches with the wrong players, are their own category. Capture the rating and queue context for these. Watching the parties, ratings, and queue, the party-handling, the rating updates, and the queue behavior, covers the main matchmaking bug sources, where the statistical, distributed nature of matchmaking produces the unfairness, the rating errors, and the queue failures that are the system's characteristic bugs.

Setting it up with Bugnet

Players experience matchmaking bugs as unfair or broken matches, so add an in-game report option for match quality and attach the matchmaking context, the player's rating and party, and the resulting match composition as custom fields, ideally with the server-side matchmaking decision logged. Bugnet stores them so a matchmaking bug arrives with the context and match needed to see what the system worked from and what it produced, letting you judge whether a pairing was genuinely unfair and why.

Group identical reports into occurrence counts, watching whether complaints cluster around a rating range, a region, or a party situation, which would point at a matchmaking problem there. Because matchmaking is statistical and its quality determines the competitive experience, the aggregated reports are especially valuable, surfacing the systematic matchmaking issues, the rating ranges that get bad matches, the regions with thin pools, the party situations that are unfair, that any single match would not reveal, keeping the matchmaking fair across the player pool, which is exactly what a competitive game lives or dies on.

Watch the aggregate and tune fairness

Because matchmaking is statistical, the most important view is the aggregate, the distribution of match quality across many matches, not any single match, since matchmaking is judged by how fair its matches are on average and a single bad match may be unavoidable in a thin pool while a pattern of bad matches is a real bug. Watch the aggregate match quality, using your captured reports and server data together, to find the systematic fairness problems.

And tune fairness deliberately, since matchmaking trades off match quality against queue time and the right balance depends on your player pool, and the captured reports tell you where the current tuning is failing players, the rating ranges or regions where matches are unfair or queues too long. Pair the reports with your matchmaking metrics, which quantify the match quality the reports describe qualitatively. Together they let you keep PvP matchmaking fair and responsive across your player pool, tuning the statistical system against the real distribution of players and matches, which is the only way to make matchmaking that feels fair, since matchmaking quality is the foundation of a competitive game's health.

Matchmaking is a consequential, statistical system. Capture the matchmaking context and the match formed, and tune fairness against the aggregate.