Your average customer does not exist. This might sound like a provocation, but it is a lesson we have learned the hard way at CCP Games while working on EVE Online, a game known for its complexity, its passionate user base, and its remarkably diverse customer behaviors.
A product team reviews their latest retention report. Day-30 retention sits at a healthy 15%. The numbers look good. Beneath that single number, however, lies a hidden reality. Some customer groups retain at over 15%, while others churn at alarming rates below 5%. The “average” masks the success stories along with the warning signs. When we aggregate heterogeneous customer populations into a single metric, we often end up optimizing for a customer that does not exist, completely missing the ones who matter. We call this the averaging trap. Escaping it has become central to how we approach customer analytics, early engagement modeling, and lifetime value prediction.
Why Averages Lie
Averages feel safe. They summarize complexity into a single number that is easy to report, track, and act on, though that simplicity comes at a cost, especially when your customer base is not as homogeneous as the metric implies.
While investigating the initial journeys of players who logged in for the first time, we noticed something strange. Players who skipped the tutorial had more than double the Day-7 retention of those who completed it. The naive interpretation? “Tutorials hurt retention.”
The real story was hiding in plain sight. Our highest-retention “new players” were not new at all. They were experienced players creating secondary accounts (alt accounts), veterans who skipped tutorials because they already knew the game. Meanwhile, genuinely new players, the ones who needed onboarding, were struggling and churning. The average blended these two populations into a single misleading number. Alt accounts are not a problem. Multi-boxing with secondary characters is a legitimate playstyle that keeps our most engaged players invested. But when we measure “new player retention” without accounting for them, we end up optimizing for the wrong audience.
The Acquisition Quality Illusion
This kind of distortion does not stop at retention. Consider a marketing campaign that brings in a wave of new sign-ups. Early engagement looks strong, and the team celebrates. How do you know whether these customers will generate revenue in the future?
Not all new customers are equal. Some will convert, spend, and become long-term players. Others will try the game for a few days, then leave and never return. If you can only measure campaign success by waiting months for lifetime value data to materialize, the budget is already spent, and the opportunity to course-correct is gone. Revenue is the primary signal, though it is not the only one we care about. Healthy ecosystem participation,things like in-game market trading, joining alliances, and participating in large-scale conflicts,often predicts long-term engagement along with eventual monetization. The question is not just “did we acquire users?” It is “did we acquire the right users?”
The Segmentation Blind Spot
The same trap appears in product decisions with a different twist. Imagine you are analyzing which in-game activities correlate with long-term engagement. The data suggests that players who participate early in complex economic activities like trading, manufacturing, and market speculation have significantly higher retention.
The tempting conclusion is to push more players toward economic gameplay earlier. The correlation might be backward, though. Players who engage in complex systems are often already committed; they have decided to invest in the game. For players still on the fence, forcing them into systems they are not ready for could overwhelm rather than engage them. Without understanding where different customer segments are in their journey, you risk designing interventions that backfire. In each case, the problem is not bad data. It is mixing populations that behave fundamentally differently, then drawing conclusions from the blend.
Breaking the Average
If averages hide the truth, the first step is asking a better question: who are we actually looking at? In our case, the answer started with a simple distinction. Not all “new accounts” represent new players. A significant portion of first-time logins come from experienced players creating secondary accounts. These players behave differently from day one. They skip tutorials, navigate confidently, and engage with advanced systems immediately. Treating them the same as genuine newcomers pollutes every downstream metric.
Using behavioral signals from the first two hours of playtime, such as session patterns, exploration behavior, economic activity, and tutorial engagement, we developed a classifier that identifies true new players early in their journey. Once we could separate true new players from returning veterans, retention metrics told a completely different story. Conversion rates, engagement patterns, and lifetime value all looked different when viewed through this lens. The “average” had not changed; now we could see what it was made of.
Understanding Player Journeys
Knowing who your players are is essential, though it is only part of the picture. The next question is: how do they experience your product? Even among true new players, behaviors vary dramatically. Some dive into missions and find their footing quickly. Others explore the interface, open windows, and browse menus, yet never quite figure out what to do next. Some get stuck after completing the tutorial, paralyzed by too many choices, while others abandon activities mid-way after hitting a difficulty spike.
To understand these patterns, we looked at the sequence of events in a player’s first hours. Think of it as an extension of classic market basket analysis, where timing and order matter. A player who skips the tutorial and immediately joins a fleet is telling a very different story than one who completes the tutorial and then wanders without direction. By clustering these event sequences, we identified distinct early-game archetypes. Some clusters were dominated by returning veterans, while others revealed new players who were trying to engage yet getting lost along the way.
From Patterns to Predictions
Understanding player types and journeys is valuable, but the real power comes from predicting outcomes before they happen. If you can identify which players are likely to convert, you can act while there is still time to influence the outcome. Marketing can allocate budget to campaigns that attract high-potential customers, and product teams can prioritize interventions for at-risk segments.
We have built models that predict both conversion likelihood and long-term revenue based on early behavioral signals. The inputs combine in-game behavior, engagement patterns, and contextual signals, while the outputs feed directly into decisions about acquisition and onboarding. Predicting “average” lifetime value across a mixed population is far less useful than predictions that account for player segments. A model that knows whether it is looking at a true new player can make sharper, more actionable predictions than one trained on blended data.
The Real Challenge
The hardest part of this work is not the data science; it is the organizational change. A segmentation that lives in a notebook does not help anyone, just like a prediction that arrives too late will not change decisions.
Stakeholders want simple KPIs, a single number they can track and report. That instinct is understandable: simplicity scales. However, a single number often hides more than it reveals. The challenge is finding ways to preserve insight without overwhelming the audience, surfacing complexity when it matters while keeping the story clear. This requires alignment between data science and product teams: shared definitions, shared priorities, and a willingness to iterate together. The models are never “done”; they are a living part of how the organization makes decisions.
Escaping the Trap
The “average player” is a convenient fiction, useful for quick summaries, but dangerous for real decisions. Better insights come from asking sharper questions: Who are we actually looking at? How do they experience our product? What can we predict about their future?
At CCP Games, escaping the averaging trap has meant building systems that distinguish true new players from returning veterans, mapping the journeys that lead to engagement, and predicting outcomes while there is still time to influence them.
At the Data Innovation Summit, we will walk through the systems we built, from early classification models to journey mapping and real-time predictions, sharing what worked, what did not, and what we are still figuring out.
About the Authors
Kajetan Sygula is a Senior Data Scientist at CCP Games with 10 years of experience applying machine learning to real-world business problems. For the past four years, he has worked in the gaming industry, focusing on practical ML solutions such as churn prediction, early conversion signals, behavioral clustering, and revenue modeling. With a background in finance and IT management, Kajetan takes a business-first approach to data science, turning insights into actionable strategies that drive measurable results.
Snorri Árnason is the Executive Producer of EVE Online and EVE Vanguard at CCP Games, where he leads the strategic evolution of one of gaming’s most iconic player-driven universes. Since joining CCP in 2007, he has leveraged his background in industrial engineering and financial planning to move through senior production and director roles, assuming the helm of the EVE franchise in 2025. With an Executive MBA and an M.Sc., Snorri focuses on empowering player agency and balancing innovation within a living economy.
*The views and opinions expressed by the authors do not necessarily state or reflect the views or positions of Hyperight.com or any entities they represent.
