Since day one, being data-obsessed has been embedded in Dream11’s DNA. Walk into “The Stadium”—our office—and you’ll see data at the heart of every decision we make.
As a user-first sports technology company, we run hundreds of experiments every day to craft world-class experiences. But in the real world, perfectly clean experiments aren’t always possible. To overcome this and stay ahead of the curve, we built our own AI-powered Causal Inference Platform. By leveraging observational data, the platform uncovers true causal impact—enabling sharper, faster, and more confident business decisions.
Here’s a closer look at Dream11’s AI-driven data journey.
Why A/B Testing Is the Gold Standard
In traditional A/B testing, users are randomly assigned to either the current product version (Control Group, CG) or a new variant (Treatment Group, TG). The winning version is determined by comparing performance across key metrics such as revenue and engagement.
The power of A/B testing lies in randomization. By eliminating pre-existing differences between groups, it ensures a true apples-to-apples comparison. Any shift in metrics can be directly attributed to the product change—independent of seasonality, user behavior biases, or external influences. This clear cause-and-effect relationship is what makes A/B testing the gold standard for measuring impact.
The Real-World Challenges of Experimentation
Despite its rigor, running clean A/B experiments is often impractical. Common challenges include:
1. Degraded User Experience
Some scenarios can’t be ethically tested. For example, we cannot intentionally trigger deposit failures for a treatment group just to measure their impact.
2. Overlapping Experiments
When treatment groups overlap across experiments, isolating the impact of any one change becomes nearly impossible.
3. Selection Bias
Consider an email campaign: not everyone in the treatment group will open the email. Comparing “users who opened the email” to those in the control group introduces skew and undermines validity.
4. Ignoring Individual Differences
Treatment effects vary across users. A one-size-fits-all conclusion overlooks meaningful differences—what works for power users may not work for new users.
5. Time Constraints
Experiments require time. Often, business decisions must rely on insights derived from existing data.
Enter AI-Powered Causal Inference
When controlled experiments aren’t feasible, Causal Inference bridges the gap. By combining AI with observational data, it answers a critical question:
Did X (a feature or event) truly cause a change in Y (a business outcome)—and by how much?
Case Study: Measuring the True Cost of Deposit Failures
We wanted to understand how a deposit failure affects a user’s future spending.
We analyzed two groups of users who had attempted at least one deposit:
Treatment Group (TG): Experienced a deposit delay or failure but did not report it.
Control Group (CG): Experienced no deposit issues.
Since users weren’t randomly assigned, inherent differences existed—such as tenure on the platform. These differences, known as confounders, can distort outcomes.
We observed that the TG spent less than the CG. But was this drop truly caused by the failure? Or were newer users simply less likely to report issues and spend less overall?
To isolate the true impact, we applied causal inference.
The Four-Step Causal Inference Framework
1. Identify & Define Groups
Clearly define treatment and control groups using observational data.
2. Observe Outcomes
Track business metrics over a defined period (e.g., spending in Week 4 after the deposit attempt).
3. Estimate Counterfactuals (The AI Engine)
Using advanced AI algorithms, we estimate what would have happened if users in the treatment group had not experienced a failure. These hypothetical outcomes are called counterfactuals.
4. Compare Like-for-Like
By comparing actual outcomes with predicted counterfactuals for the same users, we control for confounders and isolate the true causal effect.
Applying this framework across different failure categories allowed us to precisely quantify impact severity. As a result, teams could prioritize engineering resources effectively—without ever running a harmful A/B test.
Building Dream11’s In-House Causal Inference Platform
Causal questions arise daily across product, marketing, and analytics—spanning mega contests, rewards, brand campaigns, and notification unsubscriptions.
Although Causal AI has been recognized as a major emerging technology trend, existing tools fell short of our needs. They lacked intuitive, self-serve interfaces for non-technical users and struggled to scale to Dream11’s data volumes, which often exceed hundreds of gigabytes.
To bridge this gap, we built a custom Causal Inference Platform integrated with our ML infrastructure, powered by distributed computing frameworks like Ray.
Key Capabilities
Self-Service UI: Designed for product managers, analysts, and marketers with minimal coding requirements.
Flexible Problem Handling: Supports binary, multi-class, and continuous treatment scenarios.
Massive Scalability: Optimized to handle high-volume datasets efficiently.
Segmented Insights: Delivers granular results across user cohorts (e.g., new users vs. power users).
Robust Validation: Combines visual diagnostics with advanced techniques such as Matching, Meta-Learners, Double Machine Learning (DML), and Doubly Robust (DR) Learners.
Conclusion
Causal Inference unlocks cause-and-effect relationships hidden within existing data—turning observational insights into strategic advantage.
At Dream11, our in-house Causal Inference Platform has become a strategic compass, empowering teams to make faster, more confident, and deeply data-driven decisions. In a rapidly evolving digital ecosystem, Causal AI isn’t just an alternative to A/B testing—it’s a competitive edge.
