Senior Data Scientist- Experimentation & Causal Inference

Senior Data Scientist – Experimentation & Causal Inference

Headcount: 1

Level: P6

Location: US R&D Center (collaborating with global teams across APAC)

About the Role

We are seeking a Senior Data Scientist – Experimentation & Causal Inference to serve as the statistical authority behind our rapidly expanding experimentation platform. This role is responsible for ensuring that every experiment we run is trustworthy, sensitive, and correctly interpreted.

You will own the methodology layer of experimentation—not the infrastructure. While our platform engineering team builds and operates the experimentation platform, you will design, validate, and continuously improve the statistical frameworks that power it.

You will be the expert the organization relies on when asking:

"Can we trust this result?"

More importantly, you will build the methodologies and automated validation frameworks that answer that question before it needs to be asked.

Team Overview

You will join a multidisciplinary team of data scientists and platform engineers within the Big Data organization, reporting directly to the Head of Big Data.

The experimentation platform engineering team will implement your statistical specifications into production systems, allowing you to focus on methodology, experimentation quality, and causal inference.

What You'll Do

1. Own Experiment Quality & Guardrail Frameworks

• Design and maintain automated anomaly detection systems for live experiments, including:
• Sample Ratio Mismatch (SRM) detection
• Traffic allocation validation
• Experiment integrity monitoring
• Define alert thresholds and circuit-breaking criteria to identify or stop compromised experiments before they influence business decisions.
• Develop and validate guardrail metrics that balance:
• Sensitivity
• Directionality
• Interpretability
• Establish experimentation standards and best practices across the organization.

2. Improve Experiment Sensitivity Through Variance Reduction

• Design and implement variance reduction methodologies such as:
• CUPED
• Covariate adjustment techniques
• Pre-experiment normalization methods
• Reduce metric noise caused by historical user behavior and external factors.
• Enable faster and more reliable detection of treatment effects, particularly in:
• Low-traffic environments
• High-priority product launches
• Time-sensitive business decisions
• Drive measurable improvements in experimentation efficiency, with a target of reducing average experiment duration from 14 days to 9 days and significantly increasing experimentation throughput.

3. Lead Continuous A/A Testing & Data Quality Monitoring

• Design and operate continuous A/A testing programs as a permanent health check for the experimentation ecosystem.
• Validate the integrity of the end-to-end measurement pipeline, including:
• Client-side instrumentation
• Event collection systems
• Message queues
• Real-time data warehouses
• Monitor:
• Baseline metric stability
• Pipeline reliability
• Instrumentation quality
• Define what constitutes a statistically healthy experimentation environment and provide specifications for engineering teams to operationalize monitoring.

4. Partner on Experiment Design & Causal Inference

• Collaborate with Product, Engineering, Growth, and Leadership teams on:
• Experimental design
• Sample size calculations
• Power analysis
• Metric selection
• Duration estimation
• Result interpretation
• Provide guidance on causal inference methodologies when randomized experimentation is not feasible, including:
• Difference-in-Differences (DiD)
• Synthetic Control
• Regression Discontinuity Design (RDD)
• Ensure business decisions are supported by rigorous statistical reasoning.

5. Enable Scalable Self-Service Experimentation

• Translate statistical methodologies into:
• Technical specifications
• Validation scripts
• Decision frameworks
• Best-practice guidelines
• Empower experiment owners to self-serve while maintaining scientific rigor and consistency.

Required Qualifications

Education

• Master's or PhD in:
• Statistics
• Biostatistics
• Economics (Econometrics)
• Computer Science
• Mathematics
• Or a related quantitative discipline

Experience

• 3+ years of industry experience designing and analyzing online controlled experiments at scale within a technology company.
• Demonstrated experience working with large-scale digital experimentation programs involving meaningful user traffic.
• Experience beyond survey-based or clinical trial experimentation is strongly preferred.

Statistical Expertise

Strong foundation in:

• Hypothesis testing
• Power analysis
• Multiple testing correction
• Sequential testing
• Experimental design

Hands-on experience with at least two of the following:

• CUPED or other variance reduction techniques
• Sample Ratio Mismatch (SRM) detection
• Continuous experimentation quality monitoring
• Experiment health diagnostics

Technical Skills

• Advanced Python proficiency, including experience with:
• SciPy
• Statsmodels
• NumPy
• Equivalent statistical libraries
• Strong SQL skills, including:
• Window functions
• Common Table Expressions (CTEs)
• Data validation and quality analysis
• Experience building simulations and statistical validation frameworks.

Communication & Collaboration

• Ability to explain complex statistical concepts to technical and non-technical stakeholders.
• Strong capability to translate business questions into rigorous experimental methodologies.
• Comfortable writing clear specifications that engineers can implement in production systems.

AI-Native Workflow

• Extensive hands-on experience using AI-assisted development tools such as:
• Claude Code
• OpenClaw
• Cursor
• Similar AI coding assistants
• Proven ability to leverage AI tools to:
• Accelerate analysis
• Automate repetitive workflows
• Improve productivity
• Build self-service tooling

Language Requirements

• Fluent in both English and Chinese.
• Ability to collaborate effectively with engineering, product, and data teams across APAC and North America.
• Comfortable serving as a communication bridge between the US R&D Center and regional teams.

Working Hours Flexibility

• This role requires occasional cross-time-zone collaboration with teams across:
• Singapore
• Dubai
• Other APAC locations (UTC+4 to UTC+8)
• Candidates should be comfortable joining important meetings during early mornings or evenings when necessary.

Preferred Qualifications

• Experience with Bayesian experimentation frameworks.
• Familiarity with multi-armed bandit methodologies.
• Understanding of streaming data ecosystems and their impact on experimentation validity, including:
• Kafka
• Flink
• Real-time analytics architectures
• Publications or conference presentations related to:
• Experimentation
• Causal inference
• Statistical methodology
• Machine learning evaluation
• Experience in:
• Fintech
• Cryptocurrency
• Trading platforms
• Other high-scale consumer technology environments

Why Join Us?

Define Experimentation Standards at Massive Scale

You will establish the statistical foundations for experimentation across a platform serving 80M+ users globally. Your methodologies will directly influence product strategy and business decisions.

Immediate and Measurable Impact

With more than 120 active experiments and ambitious growth plans, your work will be deployed quickly and deliver visible business outcomes.

Science-Driven Decision Making

We run experiments to discover the right answers—not to validate predetermined decisions. You will have direct visibility with leadership when experimental findings challenge conventional thinking.

High-Caliber, Low-Bureaucracy Environment

Work alongside a focused team of data scientists and engineers where responsibilities are clearly defined:

• You own the science.
• Engineers own the platform.
• Great ideas move quickly into production.

This is a rare opportunity to shape the experimentation culture, methodology, and decision-making framework of a global-scale technology company.