Valiant Harbor International is a CVE Service-Disabled Veteran Owned Small Business that specializes in technical, programmatic, acquisition, compliance, and financial services for Government science and technology, research and development, and technological programs. At Valiant Harbor International, we emphasize our intense focus on helping federal government agencies identify and address organizational challenges to tailor and integrate specific solutions to solve their most difficult problems. Success is defined by our ability to meet our customers’ needs quickly, efficiently, and effectively.

Valiant Harbor International is seeking a detail-oriented Health Data Scientist to provide technical leadership in data design, evaluation strategy, and clinical realism for a multi-stakeholder program advancing AI diagnostics for rare diseases supporting the Advanced Research Projects Agency for Health (ARPA-H), Proactive Health Office (PHO). This is a Science and Engineering and Technical Advisor (SETA) role focused on ensuring that datasets, benchmarks, and evaluation methods are scientifically rigorous, clinically grounded, and aligned with real-world use.

In this role, you will operate at the intersection of health data, AI evaluation, and clinical practice, providing expert guidance to program leadership and external performers. You will help define what success means for AI diagnostics in rare disease contexts and ensure that technical progress translates into meaningful clinical insight and patient impact. This role is ideal for someone who combines deep experience with healthcare data and AI evaluation with the ability to critically assess methodologies, anticipate failure modes, and translate clinical realities into measurable technical criteria.

Key Responsibilities

• Lead and advise on the design, preparation, and validation of healthcare datasets used across the program, including EHR, genomic, and multimodal data.
• Define and assess evaluation strategies and benchmarks that are clinically realistic, methodologically sound, and resistant to shortcut learning or leakage.
• Review technical proposals, model submissions, and evaluation results from external teams, providing expert feedback on validity, risks, and interpretability.
• Assess data quality, representativeness, and bias, and advise on mitigation strategies appropriate to rare disease contexts.
• Evaluate alignment between technical performance metrics and clinical decision-making needs, working closely with clinical SMEs.
• Identify gaps between benchmark performance and real-world applicability, and advise on corrective actions.
• Serve as a technical bridge between AI developers, clinical experts, and program leadership.
• Support the program manager with technical assessments, recommendations, and decision support related to data and evaluation.
• Track progress of external partners and identify data- or evaluation-related risks that could impact program outcomes.
• Produce high-quality written reports, analyses, and presentations that communicate technical findings to diverse audiences.

This Position Is Suited For Candidates Who

• Think deeply about measurement, validity, and scientific rigor.
• Are comfortable exercising judgment in ambiguous or under-specified evaluation contexts.
• Maintain high standards for data quality and methodological integrity.
• Communicate clearly across clinical, technical, and programmatic audiences.
• Operate with ownership and accountability in high-stakes environments.
• Are motivated by advancing AI systems that deliver real clinical and patient impact, not just strong metrics.

Required Qualifications

• PhD in Biomedical Informatics, Data Science, Biostatistics, or a related field.
• 5+ years of experience working with healthcare or biomedical data in applied or evaluative roles.
• Hands-on experience with EHR, genomic, or multimodal healthcare datasets.
• Deep understanding of AI/ML evaluation methods, including benchmarking, robustness, and sources of bias or leakage.
• Demonstrated ability to critically review and assess technical work across health data and AI domains.
• Strong understanding of data governance, privacy, and compliance in healthcare contexts.
• Excellent written and verbal communication skills, with experience producing high-quality technical materials.

Preferred Qualifications

• Domain expertise in rare disease, diagnostics, or clinical decision support.
• Prior experience in technical advisory, evaluation, or SETA-style roles.
• Experience collaborating closely with clinicians and healthcare stakeholders.
• Experience working in fast-paced, multi-stakeholder, or early-stage environments.

Clearance Requirements: Ability to obtain a Public Trust.

Salary Range: Negotiable.

EEO Statement: Valiant Harbor International, LLC is an Equal Opportunity/Affirmative Action employer. Valiant Harbor International prohibits discrimination with respect to the hiring or promotion of individuals, conditions of employment, disciplinary and discharge practices, or any other aspect of employment on the basis of sex, race, color, age, national origin, religion, disability, marital status, sexual orientation, gender identity, pregnancy, veteran status, or any other protected class. If you are an individual with a disability and require a reasonable accommodation to complete any part of the application process, or are limited in the ability or unable to access or use this online application process and need an alternative method for applying, you may contact (202) 417-6705 for assistance.

This is a full time position