Podcast Answer Engine

Podcast-grounded answers, with verifiable citations.

Note: these project notes were drafted with AI assistance.

Ask any podcast a question in plain English. Get a cited prose answer with timestamps pointing to the exact moments speakers said the relevant things. The wedge: every claim is validated against its source chunk before it's shown. Failed claims get dropped. If everything fails, the system says "not in the corpus" instead of guessing.

Pipeline

  user query
      │
      ▼
  ┌─────────────────────┐
  │  query parser       │  small LLM, structured output
  │  topic / speaker /  │  resolves "latest", aliases,
  │  episode / time     │  temporal references
  └──────────┬──────────┘
             │ filters
             ▼
  ┌─────────────────────┐     ┌────────────────────┐
  │  hybrid retrieval   │◄────│  local index       │
  │  bm25 + dense + RR  │     │  chunks + meta     │
  └──────────┬──────────┘     └────────────────────┘
             │ top-K chunks
             ▼
  ┌─────────────────────┐
  │  grounded synthesis │  larger LLM, must cite
  │  prose + [chunk_id] │  only retrieved chunks
  └──────────┬──────────┘
             │ candidate answer
             ▼
  ┌─────────────────────┐
  │  validation gate    │  per-claim checks:
  │  • citation exists  │   - chunk in retrieval set?
  │  • entailment       │   - claim ⊨ chunk?
  │  • literal tokens   │   - names/numbers present?
  └──────────┬──────────┘
             │
             ▼
  cited answer  ──or──  "not in the corpus"

Design choices

• Two LLMs, separated. A small fast model parses the question. A larger model writes the answer. Each does one job.
• Three retrievers, blended. Keyword for exact phrases, dense for paraphrase, reranker for question-conditioned relevance. No single strategy wins; the blend does.
• Universal transcript shape. Speaker-labeled, timestamped turns with episode metadata. Adding a podcast is "drop a file in a folder."
• Validation as a gate, not a vibe. Programmatic per-claim checks against retrieved evidence. Synthesis is structurally prevented from inventing citations.

Status

Working POC. Six episodes across two podcasts. End-to-end pipeline runs locally. Cross-episode synthesis verified: a single hiring-advice query surfaced takes from a top-tier VC, a SaaS founder, and a positioning expert across three different episodes in one shot.

Surfaces

• CLI. Primary surface today.
• MCP-style tool for AI assistants. Coding agents call it directly during chat sessions.
• Agent harness with a polished terminal UI. Streaming output, slash commands, themes. Next milestone.

All three are thin wrappers over the same backend. Same retrieval, same synthesis, same validation.

Next

Wider corpus. More tests on the validation gate. It has to hold as the index scales. That's the interesting part.