Glicko, the Rating System That Admits It Might Be Wrong

Open a fresh account on lichess, play one game, and your rating shows up as 1500?.

Most people read that question mark as “provisional, ignore this for now” and move on. It is more specific than that. In the lichess source it is a single comparison: a rating is provisional when the player’s deviation is at least 110. The question mark is not a UI hint, it is a number leaking through the interface, and that number is the entire reason Glicko exists.

I went down this road because I wanted a ranked ladder for a side project and assumed rating was a solved problem. Take Elo, pick a K-factor, ship it. That assumption survived about two days.

What Elo actually gives you

Arpad Elo was a physics professor and a chess master, and the system he designed for the USCF in 1960 (adopted by FIDE in 1970) does one job very well: it turns a sequence of win/loss results into a single comparable number. You expect a result based on the rating gap, you compare it to what happened, and you move both players by the surprise.

The only tuning knob is K, the maximum a game can move you. FIDE currently runs K=40 for new players, 20 after 30 games, 10 once you pass 2400. That staircase is the tell. K is doing two unrelated jobs at once: it is how fast the system learns, and it is how much a single result matters. Those are not the same thing, and Elo has one dial for both.

So the K schedule exists to fake something the model doesn’t have. A new player needs a big K because the system knows nothing about them. A 2400 player needs a small K because the system knows a lot. Both statements are about confidence, and confidence is not a value Elo stores anywhere.

Everything awkward about Elo falls out of that gap:

• Two players rated 1500, one with 5 games and one with 5,000, are treated as identical opponents.
• Someone who stops playing for three years keeps a rating the system has no business standing behind.
• “Provisional” periods, rating floors, and games-played thresholds are all bolted on afterwards to patch holes the model can’t express.

Elo also assumed performance was normally distributed. That turned out to fit the data worse than a logistic distribution, which is what most implementations actually use today. It is a good reminder that the system everyone calls “Elo” is already a modified Elo.

A rating with error bars

Mark Glickman’s answer, worked out in his 1993 Harvard dissertation and published as the Glicko system through the nineties, is almost embarrassingly direct: store the confidence.

Every player gets a second number, the rating deviation, or RD. It is a standard deviation on the rating estimate, which means the pair together describes an interval rather than a point. A player at 1500 with an RD of 50 is a claim that their true strength is very likely somewhere between 1400 and 1600. The same 1500 with an RD of 350, the value a brand new player starts at, is barely a claim at all.

The same rating, three different claims. The interval is roughly the rating plus or minus two RDs.

RD moves in two directions. It shrinks when you play, because results are evidence. It grows while you sit idle, because the system’s belief decays toward “who knows, it has been a year.”

Then comes the part that makes Glicko more than Elo with a decoration attached: RD feeds back into the update itself, on both sides of the board.

Your own RD controls how far a result can move you. High RD means the system doesn’t trust your number, so it moves freely. Low RD means your rating has been earned over hundreds of games and one loss should barely dent it. That is the FIDE K-staircase, except derived rather than declared.

Your opponent’s RD controls how much the result teaches. Beating someone whose rating is a shrug tells the system almost nothing, so it barely moves you. Beating an established 1700 is real information and moves you accordingly. Elo has no way to express “this result was less informative than that one,” and once you have seen the distinction you notice its absence everywhere.

This is the property I’d defend hardest if I had to cut features. Rating-your-confidence and confidence-weighting-the-evidence are the same mechanism used twice, which is why the system stays small. Glicko’s original description is three steps.

Rating periods, the assumption everyone breaks

Here is where implementations go wrong, and I include my first attempt.

Glicko does not process games one at a time. It processes a rating period: a batch of games that the math treats as having happened simultaneously. Everyone enters the period with a rating, RD, and volatility, the results are observed, and new values come out the other end. Glickman’s guidance in the Glicko-2 paper is blunt about the size of that batch:

The Glicko-2 system works best when the number of games in a rating period is moderate to large, say an average of at least 10-15 games per player in a rating period. The length of time for a rating period is at the discretion of the administrator.

An online ladder wants the opposite of this. Nobody finishes a match and waits until Sunday to find out what it did to their rating. So the near-universal choice is to run one rating period per game, which is exactly the regime the paper warns about. That is not a small deviation either, because it changes what the system is estimating. With batching, “how did this player do against this field” is one question answered once. Per game, it becomes a chain of single-observation updates, each one confidently re-estimating a player’s strength from a sample of one.

Lichess takes the interesting middle path. It updates after every game, in the order played, but it keeps the notion of elapsed time by making rating periods fractional. The constant in their source is 0.21436 rating periods per day, with a comment explaining exactly why that number:

// Chosen so a typical player's RD goes from 60 -> 110 in 1 year
val ratingPeriodsPerDay = 0.21436d

Read that again, because it is a product decision disguised as a constant. 110 is the provisional threshold. The number was picked so that a typical active player who walks away comes back to a question mark after exactly one year. The paper gives you a decay mechanism; it does not tell you when a rating should stop being believed. That call belongs to whoever runs the ladder.

If you take one thing from this post: decide your rating period deliberately and write down why. Period length controls learning speed, inactivity decay, and how gameable the system is, all at once. Leaving it at “one period per game because that’s what the library did” means those three properties were chosen for you.

Glicko-2 and the volatility knob

Glicko-2, published by Glickman in 2001, adds a third per-player number: volatility. Where RD asks “how sure are we,” volatility asks “how erratic has this player been.”

The distinction is real. Consider two players with identical ratings and identical RDs. One grinds out results exactly in line with expectation, week after week. The other beats a 2000 on Tuesday and loses to an 1100 on Thursday. The first player’s rating is a decent predictor. The second player’s rating is a number the results keep disagreeing with, and volatility is how the system notices.

A player with high volatility gets larger rating swings, on the theory that a rating that keeps being wrong should be allowed to move faster. New players start at 0.06 by default.

Volatility is constrained by a system constant, τ, that you set once for the whole ladder and never per player. From the paper:

Reasonable choices are between 0.3 and 1.2, though the system should be tested to decide which value results in greatest predictive accuracy. Smaller values of τ prevent the volatility measures from changing by large amounts, which in turn prevent enormous changes in ratings based on very improbable results.

“Should be tested” is doing a lot of work in that sentence, and in practice nobody tests it. τ gets copied from whatever example the library README used.

There is a small piece of implementation history worth knowing. The volatility step is the only part of Glicko-2 that needs iterative root-finding, and the original procedure occasionally failed to converge from a bad starting value. Glickman replaced it in February 2012 with one based on the Illinois algorithm, a variant of regula falsi. In his simulations the new version took a median of 5 iterations, with a maximum of 19 across 10,000 runs. If you are porting an implementation written before 2012, check which version of Step 5 it inherited.

Farming volatility

Volatility has an obvious problem once you say it out loud: it is an amplifier, and it is controlled by the player.

The attack was documented against Pokémon GO’s Battle League leaderboard in 2020 in a write-up called Farming Volatility, and it is uncomfortably clean:

The enabler is the matchmaker, not the formula. Once you have dropped far enough, your opponents are people you can beat at will, so manufacturing an arbitrary win/loss pattern costs nothing but time. From there you are not gaming the rating, you are feeding the volatility estimator a sequence you constructed.

The Elo-MMR paper (Ebtekar and Liu, WWW 2021) measured this on a simulated TopCoder ladder: the volatility-farming player finished 523 rating points ahead of an otherwise identical honest player, gaining nearly 1,000 points over the final 15 contests alone. That is not a rounding error, that is the top of a leaderboard.

Chess wasn’t spared the question either. Lichess issue #7862, opened on 5 January 2021, is titled “Glicko2 may be flawed” and says exactly this: throw games to inflate volatility, then play honestly and collect the amplified gains.

Glicko-2’s volatility assumes players are trying to win. That assumption holds in rated chess tournaments, which is what it was built for, and dissolves on an anonymous ladder with free rematches and no entry cost. Before you adopt Glicko-2, ask whether losing is cheap in your system. If it is, volatility is an attack surface and you need to bound it.

The mitigations are all variations on removing one leg of the loop. Cap volatility so the amplifier saturates. Put a floor under rating so tanking is expensive and slow to undo. Keep matchmaking wide enough that a deranked account still faces uncertain opponents, which is the step that breaks the “free results” precondition. Or take the academic route and use a system designed to be strategyproof, which is what Elo-MMR argues for.

What a production implementation actually looks like

The most useful hour I spent on this was reading lichess’s rating module instead of another explainer. It is Glicko-2 as published, wrapped in about a dozen guard rails that no paper mentions:

val minRating = IntRating(400)
val maxRating = IntRating(4000)

val minDeviation              = 45
val variantRankableDeviation  = 65
val standardRankableDeviation = 75
val maxDeviation              = 500d

// past this, it might not stabilize ever again
val maxVolatility     = 0.1d
val defaultVolatility = 0.09d

// rating that can be lost or gained with a single game
val maxRatingDelta = 700

val tau = 0.75d

Put next to the paper’s defaults, almost nothing matches:

That comment on maxVolatility — past this, it might not stabilize ever again — is the honest one. Someone watched a live system wander into a state it couldn’t recover from and nailed a board over it. You will not find that in a journal, and you will absolutely need it.

The display rules are the other half, and they are pure product:

The leaderboard gate is the one I’d copy first. A ranked list that admits any RD is a list of whoever got the luckiest recent sample, and you get the same pathology TrueSkill avoids by displaying a conservative lower bound instead of the raw mean. Requiring an RD of 75 or better says, in effect: the board is for players the system is actually sure about.

And the RD floor solves a problem I did not see coming. Without one, a veteran’s RD drifts so low that the system essentially stops updating them: it has decided it knows their strength and stops listening. Floor the RD and the system keeps a permanent minimum willingness to be surprised.

Where Glicko stops fitting

Glicko is a paired-comparison model. Two competitors, one outcome, repeat. If that is not your shape, the fit is bad in ways no parameter tuning fixes.

Teams are the common case. A 5v5 result is one bit of information about ten people, and attributing it to individuals is a modelling question Glicko doesn’t answer. The usual hack, rating each player against the average enemy rating, works about as well as it sounds: fine in the aggregate, unfair in every specific case you’ll get complaints about.

Free-for-all is worse. A 100-player battle royale finishing in a full ranking is not a paired comparison at all, and decomposing it into thousands of pairwise results is both wrong and expensive.

TrueSkill, which Microsoft deployed on Xbox Live in 2005 after training it on the Halo 2 beta, was built for exactly this. It models skill as a Gaussian per player, sums team performances, and handles multiway rankings natively; leaderboards show a conservative estimate rather than the mean. TrueSkill 2 followed in 2018 with experience, quitting behaviour, and individual stats folded in.

So why is Glicko the one you find everywhere — lichess, Chess.com, Pokémon Showdown, Online-Go, TETR.IO, Guild Wars 2, Splatoon, and a long tail of smaller ladders? Two reasons, and neither is mathematical.

It is public domain. Glickman placed both Glicko and Glicko-2 there deliberately, and TrueSkill is patented. For a small team, “can I use this without a lawyer” outranks several percentage points of predictive accuracy.

And it is small enough to implement in an afternoon without believing in magic. Three numbers per player, a handful of steps, one iterative loop. When ratings look wrong in production you can actually trace why, which is not true of every Bayesian system I have read the source of.

For 1v1 with honest players, Glicko-2 is the right default. For teams or free-for-all, look at TrueSkill or OpenSkill (a permissively licensed implementation of the Weng-Lin models) before you try to make paired comparison do something it wasn’t built for.

What I’d carry into my own system

The ladder I was building is 1v1, so I landed on Glicko-2. But the parts I ended up caring about were not the ones I expected.

Store the uncertainty, but display something else. Rating and RD are what the system reasons with; what a player sees can be a tier, a rank name, a conservative lower bound, anything you can change later without a migration. Once players can read the raw number they will optimise against it, and at that point you have lost the ability to fix it.

Gate the leaderboard on confidence, at an RD threshold of 75 or whatever the equivalent is in your numbers. It costs nothing and removes the entire category of “who is this person with 6 games at the top of the board.”

Cap everything that can run away: volatility, single-game delta, rating floor, RD floor and ceiling. Every one of those caps is an admission that the model has regimes where it misbehaves, and every one is cheaper than finding out which regime at 2am.

Decide the rating period on purpose. Per game is a legitimate choice, and it is also a different system from the one in the paper. The difference should be deliberate rather than inherited from a library default.

And assume someone will try to lose. Ask what a player who wants to lose can extract from your system. If the answer involves an amplifier they control, bound it before launch instead of after the leaderboard is already wrong.

What I find genuinely good about Glicko is that it is an old idea applied honestly: a measurement without an error bar is not a measurement. Elo gave us a number. Glicko gave us a number plus how much to trust it, and then spent the rest of the design making the trust do real work.

The rest — the caps, the floors, the fractional periods, the anti-tanking rules — is what happens when a statistical model meets people who want to win. The paper does not have those parts. The source code does.

References

• Glicko rating system — Wikipedia
• Mark Glickman’s Glicko page: Glicko, Glicko-2 and Glicko-Boost papers
• Glickman (1999), Parameter Estimation in Large Dynamic Paired Comparison Experiments, JRSS-C 48(3)
• The Glicko-2 system (PDF)
• lichess-org/lila — modules/rating/src/main/Glicko.scala
• lichess issue #7862, “Glicko2 may be flawed”
• Ebtekar & Liu, Elo-MMR: A Rating System for Massive Multiplayer Competitions (WWW 2021)
• TrueSkill — Wikipedia
• Elo rating system — Wikipedia