Staff Software Engineer, Simulator Evaluation

Waymo’s simulator is one of the most complex virtual environments ever built. It blends deterministic logic, physical dynamics, and state-of-the-art Generative AI to create a training ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge: How do you mathematically prove that a virtual world is "real"?

We are looking for a Staff Software Engineer to act as the Technical Architect for this domain.  You will work at the intersection of software engineering and AI, ensuring that our simulated worlds—whether driven by explicit rules or foundation models—provide a trustworthy representation of reality.

In this Staff-level role, you will report to a Senior Staff Software Engineering Manager and act as a Technical Lead, bridging the gap between deep technical metrics and high-level product strategy.

You will:

• Architect the Eval Rubric: You will define the "Definition of Done" for simulation realism. You will look ahead at product goals (e.g., launching in snow, highway driving) and architect the evaluation roadmap that ensures our simulation fidelity matures in lockstep with onboard needs.
• The "Critic" for the System: You will design the comprehensive mathematical frameworks that validate our hybrid world. You will decide how we balance distinct evaluation needs—from verifying logical rules and dynamics to measuring the distribution quality of generative AI models.
• Build at Scale: You will lead the design of large-scale, extensible evaluation platforms (C++/Python). You ensure our metric pipelines are not just scripts, but robust distributed systems capable of providing clear, reproducible signals on petabytes of data.
• Strategic, Cross-functional Leadership You will act as the technical bridge between organizations. You will partner closely with AI research and other simulation teams, as the eval workflows you build will drive rapid innovation and research roadmaps.

You have:

• System-Level Engineering:
• 8+ years of industry experience, with a focus on building complex data systems, evaluation platforms, or back-end infrastructure.
• Expertise in designing systems that scale (C++, Python, distributed computing), with a strong focus on API design and maintainability.
• Advanced Quantitative Intuition:
• You don't just calculate metrics; you design frameworks. You can debate the merits of different statistical approaches (e.g., determining the right confidence intervals for safety-critical validation) and apply them to complex, non-deterministic systems.
• You have experience designing and implementing evaluation frameworks for complex systems or machine learning models.
• Product-Aware Leadership:
• Experience creating technical strategies that span multiple teams. You can translate high-level product requirements into concrete engineering problems (e.g., "To launch in snow, we need X specific friction metrics by Q2").

We prefer:

• Background in fields that blend code, math, and simulation: Autonomous Vehicles, Algorithmic Trading, AdTech/Search Ranking, Machine Learning, or Robotics.
• Familiarity with the validation of Generative AI (LLMs, Diffusion models) and/or classical simulation systems (Agent-based modeling, heuristics).
• Experience driving technical roadmaps for large-scale systems or validation frameworks.
• Experience guiding a team or system through a major architectural shift.