The Questions of MatchMaking: Wait Time VS Match Quality

About the author:

Bruce Brown currently leads the team responsible for building Amazon GameLift‘s FlexMatch matchmaking service.  He has been in the software industry for 12 years including time spent at Microsoft on the Xbox Live Cloud Compute and Xbox Multiplayer teams, as well as at Riot Games where he worked on the League of Legends Live Gameplay and Personalization teams. Currently, he is playing Zelda: Breath of the Wild with his 6 year old son.

The Questions of Matchmaking

As a modern game developer, you are likely aware of the core dichotomy that besets matchmaking in multiplayer games: player wait time versus match quality. You likely have had to make this tradeoff at some point in the past, and you probably agree that it’s one of the toughest, and potentially least rewarding, tradeoffs you have to make. No matter how painstakingly you strike the balance between these two opposing requirements, it can be almost impossible to satisfy 100% of your global player base.

Now, this blog isn’t going to be about me telling you “the right choice” to make a horde of happy players via matchmaking. Nobody knows your game better than you, and no one is more equipped to make this decision for your players. During my time at Xbox Live, I worked with developers to improve their matchmaking, and in every single case the outcome was different. Every game is unique, with different goals, different challenges, and different players. You are the expert in this equation. So although I can’t provide you with an answer that is one size fits all, what I can attempt to provide you with is my framework of questions you can ask when calculating the choice that is right for your game, and ultimately your players.

The Waiting Game

Until someone figures out how to defy the laws of physics, perfect matches made instantly aren’t going to happen. So, what can we do in the meantime? Well, here is what we know:

1. Players are going to have to wait for a match
2. Players aren’t going to wait forever.

So, the question I like to ask is, “Just how long are my players willing to wait for a match?”

This question forces you to view things from the perspective of your player. It also pushes you to think methodically, using one half of the problem (wait time) as the subject, and using the other half (match quality) as a dial to tune into your player’s resonant matchmaking frequency, zeroing in on the timeframe they’re willing to wait. So how do we measure a player’s tolerance to wait?

Questions of Time

There are a number of avenues of exploration, but the metrics required to answer this question ultimately boil down to two categories:

1. Player data – things you can measure about your matches
2. Player feedback – what your players tell you about their matches

If your game is live, you probably have this data in spades. You also know what your game’s goals for success are, and how these translate to delivered player experience. If your game is pre-release, these goals can act as a proxy for the data and feedback.

With data, feedback, and goals in hand, you now have a wealth of insight into player behavior at your disposal. This can be used to calculate the wait time sweet spot by asking some of the following questions:

• How long is a match in your game?

If your matches are short (single digit minutes), your players are likely to want to get back into a match as quickly as possible. Longer matches (tens of minutes) might mean your players might be willing to wait for the most competitive match possible.

• On average, how many matches do your players play in a single gaming session?

If your players only sticks around for one or two matches per session, then they might be willing to wait longer for the best possible match. If they play many matches in one session, they may prefer quantity over quality.

• Are your players casual/ranked/somewhere in between?  Do they fall into multiple segments?  Do they sometimes cross segments (casual/ranked)? (Okay, that was three variables in one.)

Being able to segment players can be critical for issuing timely matches. Segmenting allows you to pre-group players into selection pools for more patient as against less patient players, and preventing you from searching across your entire player base for every match request. Your players will also want a competitive match, so it’s important to be able to pre-group players into pools so you can make smarter matches. But, on the other hand, you have to balance out the fact that players generally will only wait so long (at which point you had better relax your criteria and get them playing).

Collecting answers to these questions will enable you to define the answer to the wait time half of the equation. Now let’s look at what we can ask about match quality.

Looking for a Fair Fight

When you attempt to group players into matches, you likely start with a set of criteria that you measure players against, such as latency or skill.  You then compare those criteria across players through some set of heuristics via some algorithm. I’m going to call the set of criteria “attributes”, the comparison mechanism “rules”, and the algorithm the “rules engine”. So you pass player attributes into your rules engine which uses your rules to compare those attributes to find a match.  Okay, great – so what?

Well, the number of attributes, the complexity of the rules, and the efficiency of your rules engine are limiting factors to how fast you can match your players (given a constant population of players). These factors also define match quality in your game. If you have lots of rules and attributes, you likely are trying to hone in on a very specific quality of experience that you want for your players. On the other hand, if you only care about latency when considering match quality, you only have one rule, and competition quality isn’t your highest priority.

No matter what your rules of match quality are, you want to give these constraints every advantage you can to make a match as fast as possible. You also probably want to be able to relax your rules over time so that you can be sure that your players eventually find a match, even if it isn’t the best quality.  To achieve this, here are some questions to ask about match quality:

• Which rules are rejecting players most often?
• How long does it take to match players within a segment, on average?
• Which players are taking the longest to match?  Why?
• How long do you want to wait before expanding your search and relaxing your rules?  Which rule(s) do you want to relax?  What metrics are informing these choices?

Answering these match quality questions requires a system that can make these measurements and present that data to you in a logical, clear way. And this is what I meant earlier in proposing my approach to the eternal waiting time/match quality dichotomy – you can use your deliberated answers to fine tune our assessment of just how long your players are willing to wait before being placed in a match.

Trial and a Lot Less Error

Hopefully this framework of questions will not only help you to find the right balance between player wait time and match quality for your game, but will also convince you that you are the smartest matchmaking system around. Your expertise with your player knowledge and data ultimately means you determine how successful your matchmaking is for your players.

As such, the Matchmaking tools you surround yourself with should work with you to answer these questions. At Amazon GameLift, we built FlexMatch with these questions in mind to support your expertise, emitting the data you need to answer these questions, and providing the flexibility to make the process of matching players suit your needs for your game and your players.

Thanks, and happy matching!