Airbnb Data Analyst Agent

A reusable multi-agent analytics copilot that plans, writes, validates, charts, and narrates SQL analysis — every number cited back to source rows.

Live website GitHub

Org: Columbia · Agentic AI for Analytics
Role: Designed & built solo
Period: 2026
Status: live
Stack: FastAPI · LangChain · DuckDB / Postgres / Snowflake · OpenAI function calling · matplotlib · pytest
Links: Live demo ↗GitHub ↗

5^*1

agents on a typed message bus

planner · SQL · validator · chart · narrator — every step inspectable and replayable

×3^*2

retry budget, exponential backoff

validator-triggered on tool errors and intent mismatch

100%^*3

numbers cited to source rows

narrator contract — uncited claims don't ship

01/ CONTEXT

The problem

Analysts spend hours translating business questions into SQL, pulling data, checking it, and reformatting the answer into something a stakeholder can consume. Most LLM "text-to-SQL" demos hallucinate schema, fail silently on bad queries, or skip the last-mile steps that actually matter: a chart, a citation, a sanity check.

The live Airbnb implementation is the demonstration surface for a reusable analytics copilot: it decomposes the question, writes SQL that actually runs, validates the result before showing it, plots the data, and cites the rows it used. The same adapter contract supports DuckDB, Postgres, and Snowflake.

02/ CONSTRAINT

The constraints

Text-to-SQL models hallucinate column names, misuse joins, and produce queries that run but return the wrong answer
Single-prompt approaches have no way to recover from tool errors or mid-query course corrections
Charts need semantic intent ("show trend over time"), not just "make a chart of this dataframe"
Auditability is table stakes in any real analytics context — every number shown must trace back to source rows
Latency and cost must stay bounded even as the agent retries and self-critiques

03/ APPROACH

The loop

Instead of one prompt doing everything badly, five specialized agents each own one contract:

Planner — decomposes the natural-language question into a plan of sub-queries and tool calls
SQL Agent — writes SQL against the live warehouse schema, with access to db.schema() and db.query(sql)
Validator — dry-runs the SQL first, audits row counts, nulls, and types; self-critiques when the result doesn't match the planner's stated intent; retries on tool error (×3, exponential backoff)
Chart Agent — calls plot.auto(df, intent) to render the right visualization for the question, not just a visualization
Narrator — composes the final answer, citing every number back to specific source rows

All five communicate over a typed message bus, so every intermediate step is inspectable and replayable — the trace below is rendered from exactly that bus.

The five-agent pipeline. Each box is a separate agent with its own contract; the message bus carries {plan, sql, df, chart, answer, error}.

04/ SYSTEM

The build

Every capability is a typed tool with an explicit contract — the agents can only act through these:

Tool	Signature
`db.schema()`	`→ Table[]`
`db.query(sql)`	`→ DataFrame`
`df.describe(df)`	`→ Stats`
`plot.auto(df, intent)`	`→ PNG`
`web.search(q)`	`→ Link[]`

The warehouse layer is pluggable: DuckDB for the live demo (NYC Airbnb data), with Postgres and Snowflake adapters behind the same interface. FastAPI serves the API; the whole system is exercised by a pytest eval harness on every commit.

05/ EVIDENCE

Evidence

Guardrails are only real if you can watch them fire. Below are recorded runs from the system — step through them. The first run includes a hallucinated column name, the database error it caused, and the validator catching and fixing it.

What keeps the system honest in production:

SQL dry-run + row-count sanity check before any query result is trusted
Null / type audit before any chart is rendered
Self-critique when the SQL result doesn't match the planner's stated intent
Golden Q/A regression suite — every commit runs canonical questions and checks the answers
Per-query latency and cost breakdown, so behavior stays measurable as prompts evolve

agent.trace · recorded run

❯ Do superhosts get better review scores than other hosts?

planner820ms

Compare avg review_scores_rating for superhosts vs non-superhosts; check statistical significance before claiming a difference.

sql[fail]1.1s

SELECT superhost_flag, AVG(review_scores_rating) AS avg_rating,
       COUNT(*) AS n FROM listings
WHERE review_scores_rating IS NOT NULL
GROUP BY superhost_flag;

db[fail]90ms

ERROR: column "superhost_flag" does not exist

validator[retry]940ms

Tool error caught. Re-read schema via db.schema() — column is host_is_superhost. Retrying (attempt 1/3, backoff 0.5s).

sql1.4s

SELECT host_is_superhost, AVG(review_scores_rating) AS avg_rating,
       COUNT(*) AS n FROM listings
WHERE review_scores_rating IS NOT NULL
GROUP BY host_is_superhost;

db160ms

returned 2 rows · n(true)=18,402 · n(false)=26,117

validator2.1s

Null audit OK · sample sizes healthy · running Welch's t-test on the two groups.

chart1.7s

Rendered bar chart — superhost 4.89 vs other 4.61 (0–5 scale).

narrator3.1s

Yes. Superhosts average 4.89 vs 4.61 (Δ=0.28, p<0.001, Welch's t-test). Source: listings.review_scores_rating × listings.host_is_superhost, n=44,519.

run complete · 11.4s end-to-end · $0.014 · 9 bus messages

step 9/9

Recorded runs from the live system, replayed from the message bus. Use the controls to step through — including the failed query and its retry.

06/ IMPACT

Impact

Handles multi-step analytics questions end-to-end with auditable traces
Every answer is grounded — users can drill into the exact rows the agent cited
Regression evals plus per-query latency/cost tracking keep production behavior measurable
Designed to plug into any warehouse with a SQL-compatible adapter

Questions about this project? The chat answers from these case studies and links back to them.