Data Scientist / ML Engineer

Available full-time · January 2027

Arjun Varma

Arjun Varma

I build forecasting, decision-support, and agentic systems, from data and evaluation to deployed products.

Data science intern at Novo Nordisk. M.S. Data Science at Columbia. Previously, three years building production ML and analytics at ZS.

01Selected projects

  1. DEUCE's match forecast dashboardProduct screenshot · DEUCE
    ForecastingDesigned & built solo

    DEUCE Tennis Forecast

    Calibrated tennis forecasts, player analysis, and tournament simulations for ATP and WTA, with a daily modeling pipeline.

    87,957 matches in walk-forward evaluation

    Maintained product · data refreshes hourly · model retrains daily

    Evidence & design

    Across 87,957 walk-forward matches, the ensemble reached 0.1950 ATP and 0.2017 WTA Brier scores; the product serves forecasts for both tours.

    87,957 matches in walk-forward evaluation

    Measured evaluation

    45,831 ATP and 42,126 WTA scored matches, evaluated chronologically

    Evaluation methodology notes
    0.1950 ATP Brier score

    Measured evaluation

    five-seed calibrated XGBoost ensemble; 69.6% walk-forward accuracy

    Evaluation methodology notes
    0.2017 WTA Brier score

    Measured evaluation

    five-seed calibrated XGBoost ensemble; 68.5% walk-forward accuracy

    Evaluation methodology notes

    Scope. The closing market remains stronger on the odds-matched subset: 0.201 Brier versus the model's 0.203. The full evaluation and that subset are different comparisons.

    forecast inputs

    Four families of match evidence feed one calibrated forecast. Select a signal to see what it adds.

    walk-forward
    01
    selected signal
    Surface Elo

    A separate strength signal for each court surface keeps hard, clay, and grass performance from collapsing into one rating.

    modelXGBoost
    calibratedwin probability

    simulated: point → game → set → match → draw

  2. Recorded output · NYC listings

    Do superhosts get better review scores than other hosts?

    Yes. Superhosts average 4.89 vs 4.61 (Δ=0.28, p<0.001, Welch's t-test).

    AgentsDesigned & built solo

    Airbnb Data Analyst Agent

    A question becomes validated SQL, a chart, and a cited answer, with failed queries and their corrections visible in the trace.

    5 agents on a typed message bus · architecture

    Evidence & design

    A live demo answers multi-step questions with SQL, charts, source-row citations, and inspectable traces. The recorded example shows recovery from an invalid column.

    5 agents on a typed message bus

    Architecture

    planner · SQL · validator · chart · narrator — every step inspectable and replayable

    Agent architecture notes
    ×3 retry budget, exponential backoff

    System contract

    validator-triggered on tool errors and intent mismatch

    Recorded example & checks
    Required source-row citations

    System contract

    narrator contract; not a measured citation-success rate

    Citation contract notes

    Scope. The citation requirement is a system contract, not a measured 100% success rate. Recorded runs demonstrate behavior on specific questions, not every possible query.

    Full recorded trace →
    recorded failure → recoverystep 1/9
    planner0.82s

    Compare avg review_scores_rating for superhosts vs non-superhosts; check statistical significance before claiming a difference.

  3. Packet structure · architecture

    Statement
    Demand letter
    Exhibit index
    Filing guide

    Uploaded evidence → shared case record → packet

    AgentsCollaborative project

    ClaimReady

    A planner and four specialists turn contracts, invoices, emails, and screenshots into structured NYC small-claims packets.

    4 specialist agents · architecture

    Evidence & design

    A deployed workflow produces a structured packet from uploaded documents, supported by multimodal extraction and legal retrieval.

    4 specialist agents

    Architecture

    Extractor, Defendant Resolver, Jurisdiction Checker, and Drafter

    Repository architecture
    6 legal-corpus documents

    Corpus scope

    Curated legal corpus documented in the repository; not an evaluation result

    Repository corpus notes
    4 packet documents

    Architecture

    Documented packet contents; not a measured success rate

    Repository output contract

    Scope. The public portfolio describes the shared system without an individual ownership breakdown or a measured end-to-end success rate.

    specialist pipeline

    Four specialists build one shared case record. Select a stage to see what it adds.

    CaseFacts
    stage 01adds to CaseFacts

    Extractor

    Reads uploaded text and images, then converts the evidence into validated CaseFacts rather than passing free-form reasoning downstream.

    evidence / tool

    contracts · invoices · emails · screenshots

    fields added
    contractperformancebreachexhibits
    statement
    demand letter
    exhibit index
    filing guide

    NYC · unpaid-services contract claims · document generation only

  4. Evaluation schematic · proprietary work

    Available history
    45 days masked
    30-day target

    Keep near-diagnosis information out of the model.

    Production MLAdvanced Data Science Associate Consultant

    Biliary Tract Cancer Early Detection

    A production cancer-detection model built around delayed claims, leakage-safe evaluation, and interpretable patient scores.

    ~45d earlier identification

    Proprietary client work · public website and source code unavailable

    Evidence & design

    The deployed pipeline scores 250M patient-claims monthly and identifies likely BTC patients approximately 45 days earlier than the claims-lag baseline.

    ~45d earlier identification

    Measured evaluation

    vs. the standard 45-day claims-lag baseline, leakage-masked backtest

    Evaluation methodology notes
    250M patient-claims scored / month

    Production scale

    automated monthly refresh, in production

    Production methodology notes
    PMSA '25 methodology presented

    Recognition

    client funded replication across other tumors and brands

    Reported impact notes

    Scope. This is proprietary client work. Patient data, source code, and numerical precision/recall results are not public; the case study explains the methodology and reported production scope.

    evaluation window

    Drag the index date. The model sees history only up to 45 days before it, then is scored on the next 30.

    leakage safe
    available history
    45 days masked
    next 30 days
    index date
    01Trainonly information available live
    02Maskremove near-diagnosis leakage
    03Predictscore the following 30 days

02Experience

Novo Nordisk

Jun 2026 — present

Plainsboro, NJ

Data Science Intern — Commercial Data Science

Improved propensity scoring and next-best-engagement models, then built an LLM decision-support layer that turns behavioral, access, competitive, and adoption signals into explainable field guidance.

Role details

Data Science Intern — Commercial Data Science

Jun 2026 — present

  • Improved propensity scoring and next-best-engagement models, then built an LLM decision-support layer that turns behavioral, access, competitive, and adoption signals into explainable field guidance.
  • Integrated longitudinal claims with unmet-need, market-opportunity, and early-adoption signals to prioritize ~500 of 10,000+ providers nationwide.
  • Achieved 5.2x top-decile lift with an XGBoost propensity model using provider profile, prescribing, access, and engagement features; combined predictions with rule-based signals into explainable priority tiers and a nationwide planning dashboard.
  • Supporting pre-launch analytics for investigational denecimig (Mim8), informing HCP targeting and field readiness.

ZS Associates

Feb 2022 — Jun 2025

Pune

Advanced Data Science Associate Consultant

Shipped the biliary tract cancer early-detection model — 250M patient-claims scored monthly, in production.

Role details

Advanced Data Science Associate Consultant

Feb 2025 — Jun 2025

  • Shipped the biliary tract cancer early-detection model — 250M patient-claims scored monthly, in production.
  • Reduced weekly decision cycles from days to minutes for 100+ stakeholders across a $10B portfolio by translating 5+ high-volume sources into a reusable Spark/SQL data warehouse, product KPIs, and self-service analyses.
  • Productionized models with MLflow tracking, feature and prediction drift monitoring, and CI validation to reduce silent failures.

Decision Analytics Associate Consultant

Jul 2024 — Jan 2025

  • Built and deployed positive-unlabeled learning models that recovered missing categorical labels in sparse medical transaction data, expanding analytical coverage from ~40% to 95% across segments and territories.
  • Raised first-pass analytical quality above 99% and saved ~50 hours per month by leading a five-member modernization of legacy business rules into validated, reusable logic.

Decision Analytics Associate

Feb 2022 — Jun 2024

  • Established audit-ready real-world evidence modeling and segmentation across $4B+ in products by designing cohort and entity features over millions of incomplete and miscoded records.
  • Engineered PySpark/SQL ETL across healthcare sources and defined patient inclusion and exclusion logic robust to missing and miscoded fields.

Promoted to Associate Consultant in 4 cycles (typical: 5). Expert Associate and Insight Illuminator awards.

Columbia University

2025 — present

New York

Graduate Teaching Assistant

TA for Business Analytics II (Foundations of AI) and Hollywood & Big Data at Columbia Business School.

Role details

Graduate Teaching Assistant

2025 — present

  • TA for Business Analytics II (Foundations of AI) and Hollywood & Big Data at Columbia Business School.

03Inside an agent run

A recorded question, the answer it produced, and the checks along the way. From the Airbnb Data Analyst Agent.

Full case study →

Recorded NYC listings example

Do superhosts get better review scores than other hosts?

Answer from this run

Yes. Superhosts average 4.89 vs 4.61 (Δ=0.28, p<0.001, Welch's t-test). Source: listings.review_scores_rating × listings.host_is_superhost, n=44,519.

Recorded duration: 11.4s · Cost: $0.014 · 9 steps

Inspect the recorded steps
recorded failure → recoverystep 1/9
planner0.82s

Compare avg review_scores_rating for superhosts vs non-superhosts; check statistical significance before claiming a difference.

04All projects

  • DEUCE Tennis Forecast

    2026Forecasting

    A live ATP and WTA forecasting platform that combines surface-aware Elo, opponent-adjusted serve and return modeling, calibrated XGBoost, and tournament simulation.

    Designed & built solo

    Python · XGBoost · Elo · Markov models · Monte Carlo · Next.js · GitHub Actions · Firebase

  • Airbnb Data Analyst Agent

    2026Agents

    A reusable multi-agent analytics copilot that plans, writes, validates, charts, and narrates SQL analysis, with source-row citations and inspectable traces.

    Designed & built solo

    FastAPI · LangChain · DuckDB / Postgres / Snowflake · OpenAI function calling · matplotlib · pytest

  • ClaimReady

    2026Agents

    Planner-led workflow with four specialists that turns contracts, invoices, emails, and screenshots into structured NYC small-claims packets with multimodal extraction, legal retrieval, and typed handoffs.

    Collaborative project

    OpenAI Agents SDK · FastAPI · Next.js · GCP

  • Biliary Tract Cancer Early Detection

    2024 — 2025Production ML

    A production model that flags likely BTC patients ~45 days before claims data confirms them — scoring 250M patient-claims every month.

    Advanced Data Science Associate Consultant

    PySpark · XGBoost · SHAP · K-means / GMM · NLP clustering · MLflow

    Proprietary client work · public website and source code unavailable

  • Filing Intelligence RAG

    2026Retrieval

    Evidence-first research across 4,967 indexed passages from 128 filings, earnings decks, and call transcripts spanning 15 companies. Deterministic retrieval, bounded context, and page/line citations open directly in the source PDF; 72 automated checks cover retrieval, authentication, API contracts, and production builds.

    Product and RAG engineering

    Python · Next.js · TypeScript · FastAPI · ChromaDB · Vertex AI · GCP Cloud Run

  • GAFFER: Live World Cup Forecasting Platform

    2026Forecasting

    Forecasts the 2026 World Cup from ~49K international matches using Elo, a time-weighted Dixon-Coles goal model, squad-value blending, and 50,000 Monte Carlo runs. Walk-forward evaluation across 8,136 matches since 2018 reached 0.887 log loss versus a 1.05 baseline; frozen pre-match predictions are graded against Kalshi as the hourly pipeline refreshes.

    Python · pandas · NumPy · Next.js · GitHub Actions · Firebase

  • ClassPulse

    2026LLM products

    Live classroom theme extraction — students answer via QR, an LLM clusters responses into themed cards every 10s over SSE, with a 5-model fallback chain.

    FastAPI · React · SSE · OpenRouter

  • SeanceAI

    2025Agents

    Conversations with 60+ historical figures under era-appropriate knowledge boundaries; Dinner-Party mode runs 2–5 figure multi-agent dialogue.

    Flask · OpenRouter · SSE

  • Citation Format Checker

    2026LLM products

    Narrow-scope chatbot that flags APA 7 / MLA 9 / Chicago 17 violations with rule-IDs and quoted evidence; three-method eval suite, 30+ test cases.

    Vertex AI · FastAPI · Cloud Run

05About

My work spans forecasting, experimentation, healthcare analytics, and agent workflows. I build the data pipelines, evaluation, and interfaces around the model.

Forecasting & evaluation

Chronological backtests, calibration, and uncertainty.

DEUCE Tennis Forecast →

Agent workflows

Typed tools, validation, and recovery from failed queries.

Airbnb Data Analyst Agent →

Production ML

Claims pipelines, leakage controls, and interpretable models.

BTC Early Detection →

Retrieval & citations

Scoped retrieval and answers grounded in source documents.

Filing Intelligence RAG →
All skills
Programming & ML
Python · SQL · R · pandas · NumPy · scikit-learn · PyTorch · XGBoost · SHAP
Modeling & Experimentation
Propensity scoring · A/B testing · Experiment design · Hypothesis testing · Causal inference · Ranking / recommendation · Probabilistic forecasting · Monte Carlo simulation
Evaluation & Monitoring
Walk-forward validation · Brier score · Multiclass log loss · Regression suites · Feature drift · Prediction drift · Data quality validation
Applied AI
LLMs / RAG · LangChain · Agentic workflows · Multi-agent orchestration · LLM evaluation · Guardrails · FastAPI · REST APIs
Data & MLOps
Spark / PySpark · Databricks · MLflow · Production ML · ETL / ELT · Data warehousing · Docker · CI / CD · GitHub Actions
Databases & Cloud
Postgres · Snowflake · DuckDB · ChromaDB · AWS S3 · AWS EMR · AWS Athena · AWS SageMaker · GCP Cloud Run

Columbia University

M.S. Data Science

Aug 2025 — Dec 2026

TA — Business Analytics II (Foundations of AI) · Hollywood & Big Data, Columbia Business School

Vellore Institute of Technology

B.Tech, Electronics & Communication Engineering

Jul 2018 — May 2022

Special Achiever Award · Merit Scholarship