Data Scientist, Bailey AI

What This Role Is

Bailey is a clinical AI system that orchestrates LLMs, manages health safety evaluation, and integrates with clinical data to help people navigate their health. Three data scientists build it today, and a new ML/Platform Engineer is standing up the deployment and monitoring infrastructure underneath it.

This role is the analytical half of that infrastructure. As drift detection, safety-weighted quality scoring, and distributional monitoring come online, someone needs to own the statistics behind them: what counts as drift, how confidence calibration is measured, how outputs compare against ground truth in our FHIR data. You'll also go deeper on health scoring analysis (population percentile and distribution work) and support retrieval/embedding evaluation as the team's RAG work matures.

This is real, scoped analytical ownership, not busywork. You'll be mentored by three senior data scientists, but mentorship here means a safety net and a growth path — not supervision. You're expected to scope your own analysis, exercise judgment, and know when and who to ask, not to wait for a spec.

What You'll Actually Do

Four real, currently-queued workstreams — not a generic junior-DS template:

• Evaluation science support (pairs with the new ML/Platform Engineer): once the platform engineer builds the pipelines, you own the analytical side — defining drift thresholds, the statistical work behind safety-weighted quality scoring, confidence calibration analysis, and ground-truth comparisons against FHIR data.
• Health scoring depth: a second set of hands on body-system scoring analysis — population percentile/distribution work and outlier handling in the Databricks batching pipeline — currently handled side-of-desk by Sean alone.
• RAG/embeddings evaluation (pairs with Kenan): retrieval quality analysis and embedding evaluation supporting Kenan's move toward embeddings/vector stores, without requiring him to hand off ownership.
• Clinical skills library analysis: usage frequency analysis, overlap/dedup signal, and risk-tier classification accuracy for the clinical skills library.

You won't be handed all four on day one — expect to ramp in through one or two with a senior DS pairing closely, then take on more as you build context.

What You Bring

These are the things that would be hard to build on the job at the pace this role demands:

• Real statistics and machine learning fundamentals, already in hand. Not developing basics — you can already reason about distributions, calibration, and evaluation design without a tutorial.
• Real Python ability, evidenced by your own work. A GitHub (or equivalent) that reflects code you personally wrote and can explain line-by-line — not output from a coding agent presented as personal skill.
• An understanding of systems and control flow. Coding agents are widely used now, but you can only direct one well if you understand the system you're asking it to change. You should be able to catch an agent when it's wrong, not just accept its output.
• Genuine independence in ambiguity. The problems in this role (drift thresholds, safety-weighted scoring, FHIR ground-truth comparisons) aren't fully defined yet. You should be comfortable scoping your own analysis — and just as comfortable asking a sharp, targeted question when you're stuck, and knowing who on the team to ask.
• Ambition and ownership. You treat a knowledge gap as something to close, not something to avoid, and you feel accountable for outcomes, not just tasks.

What Would Make You Particularly Effective Here

Not requirements — things that would shorten your ramp or deepen your impact:

• Experience directing a coding agent as part of your own workflow (nice to have, not decisive — and its absence isn't a red flag)
• Exposure to clinical data standards (FHIR, HL7) or health information systems
• Coursework or project experience with embeddings, vector search, or retrieval evaluation
• Familiarity with Databricks, Spark, or other batch-processing pipelines
• Healthcare or other regulated-industry experience

About the Team

You'll join a small data science team (Sean, Kenan, Zack) that builds Bailey's AI capabilities: models, integrations, clinical skills, and orchestration. A new ML/Platform Engineer is joining alongside this role to own production infrastructure. You'll have direct senior mentorship on every workstream above, real ownership over your analysis, and visibility into work that's genuinely still being defined — not a finished playbook you're executing against.