Regulation agent
FDA CFR retrieval and section-aware evidence
AI engineering · Research MVP
Regulatory Multi-Agent RAG System
A source-aware FDA research assistant built to keep regulations, guidance, and supplemental web evidence distinct while making retrieval, citations, and review steps inspectable.
Architecture
Route by source, validate before review
A structured router dispatches to only the needed source specialists. Regulation and guidance agents use the shared retrieval stack; web research is supplemental. Their attributed evidence is synthesized once, then citation-checked and evaluated.
- Question Structured multi-label routing
Guidance agent
FDA guidance retrieval with source provenance
Web research agent
Supplemental search and extracted web evidence
- Merge evidence and synthesize Assign deterministic source IDs to the combined source catalog
- Validate citations Reject unsupported or malformed source references before publishing
- Evaluate the answer Apply structured review and deterministic retry policy
- Pass: finish with the validated answer
- Revise or research: bounded retry, then revalidate
- Human review: checkpointed approval or feedback
Retrieval engineering
Preserve document structure and evidence scope
Shared retrieval pipeline
PostgreSQL/PGVector supplies vector candidates. A BGE cross-encoder
(BAAI/bge-reranker-v2-m3) reranks them before
relevance-ordered, budgeted expansion to source sections or
paragraphs. The system tracks scope and completeness rather than
treating every chunk as a standalone answer.
Facet-aware selection
The optional P4-1 selector attributes candidates to facets in structured questions and chooses supporting parent sections from the existing candidate set. It is deterministic: it adds neither another semantic search nor an LLM call. It remains opt-in; the production default is unchanged.
The agent separates binding CFR requirements from nonbinding FDA guidance and keeps web material supplemental. Final answer citations are resolved against the merged evidence catalog, not accepted solely because a model produced citation-shaped text.
Measured outcome
Retrieval coverage improved without an answer-accuracy claim
The Phase 4 P4-1 report compares the baseline P4-0 with opt-in facet-aware parent selection on 23 retrieval cases; only three cases activated facet-aware selection.
1.000 → 1.000
Candidate Recall@15
0.928 → 1.000
Final evidence coverage
0.888 → 0.960
Complete-scope coverage
1,501 → 1,535 ms
Median retrieval pipeline latency
0
Citation integrity failures in the report
These are retrieval-evaluation results, not end-to-end answer accuracy. The benchmark measures candidate, evidence, and scope coverage; it does not establish how often generated answers are correct.
Application and next step
Persistent review workflow, local MVP
The Gradio interface uses PostgreSQL-backed conversation records and LangGraph checkpoints to restore conversations and human-review state. Its current configuration is local-only, with public sharing disabled; authentication and authorization are not implemented.
Future integration with the existing AI Clinical Trial Navigator is planned, but has not yet been implemented.