Stable foundation deployed Next: evaluation confidence Ideas parked deliberately

Project update roadmap

A lightweight product/engineering map for the /project-update-test workflow. The aim is to keep useful future ideas visible without pretending they are all urgent.

What is already in place

Context and trend baseline

Saved project context, milestones, health history, risk lineage, official snapshots, and deterministic project selection.

Project knowledge layer

Manual/background knowledge, approved-report ingestion hooks, MiniLM embeddings, semantic/keyword retrieval, queue status, and backfill tooling.

Report pipeline integration

Retrieved knowledge is available to the analyser as supplemental context, never as transcript evidence. Semantic milestone and risk matching are now in place.

Ask project memory

A scoped Ask endpoint and UI can retrieve relevant project memory, with retrieval-only fallback when generation is unavailable.

Roadmap

Next

Golden evaluation pack

Build a small set of repeatable project-update cases so we can measure quality instead of relying on one-off smoke tests.

  • Renamed milestones and reworded risks.
  • No context / first report.
  • Background-doc retrieval.
  • DB unavailable / graceful degradation.
  • Duplicate transcript handling.
Next

Docs and ops notes

Document the moving parts so the system is easier to run and safer to change later.

  • Embedding worker and queue status.
  • Ask endpoint and generated/retrieval-only modes.
  • Venv dependency note for psycopg2.
  • Backfill script usage, but do not run historical backfill until real approved reports exist.
Later

Generated Ask answer validation

The generated mode exists, but live validation should use disposable fake memory first so real project memory is not sent externally just for testing.

Idea / parked

Auto-compaction and retention policy

Not needed yet. Keep official/manual/background knowledge indefinitely, keep real approved-report memory for now, and clean up test junk aggressively. Revisit auto-compaction only if retrieval becomes noisy or the knowledge store grows enough to need lifecycle management.

  • Possible future rule: keep latest 10–20 approved-report chunks.
  • Summarise older reports into a single historical memory item before archiving raw chunks.
  • Never auto-archive official/manual background docs.