dsearch2 v7.2 Pipeline

Hybrid semantic disk search. 950K chunks from 128K files. Query flows left→right through alias expansion, 3 active retrieval arms, RRF fusion, then optional AI answer.

Active arms (3): BGE-M3 vector (W=1.0), TF-IDF lexical (W=1.0), Recoll path BM25 (W=1.5). These 3 cover orthogonal axes — semantic, exact-term, filesystem.
Disabled (2): Recoll content (W=0.0, noise) and FTS5 (W=0.0, lexical axis saturated — 0 gains in grid search).
Optional: HyDE via --hyde flag (helps abstract queries, hurts navigational — net −5.7pp, manual only).
Data stores: ChromaDB BGE (vectors), SQLite chunks.db (text + FTS5, 900× faster than ChromaDB text fetch).

38/70 golden hits (54.3%). Easy: 30/38 (79%). Hard: 8/32 (25%). Retrieval-side ceiling reached — further gains need data quality work.

Click a box or arrow for details · drag boxes · bend arrows with yellow dot · positions auto-saved per browser.

dsearch2 v7.2 — Pipeline Architecture

Drag boxes & labels · click an arrow then drag its blue ends to re-route or the yellow dot to bend · positions saved per-browser.
auto-saves every change (per-browser)
W = 1.0 W = 1.0 W = 1.5 W = 0.0 W = 0.0 vectors text (8.6ms) --hyde flag Query Input user search terms Greek + English Alias Expansion alias_map.json multi-word phrases BGE-M3 Vector BAAI/bge-m3 · 1024-dim primary arm · local · private TF-IDF Lexical term frequency scoring exact keyword matching Recoll Path BM25 filename + directory matching highest single-arm gain Recoll Content BM25 DISABLED — adds noise disabling gained +1.9pp SQLite FTS5 BM25 DISABLED — lexical saturated 0 gains, 1-2 regressions RRF Merge Reciprocal Rank Fusion k=60 · dedup_per_file vec_protect: top-5→top-10 score = Σ W/(k+rank) Top-20 Results 38/70 golden hits easy 30/38 · hard 8/32 dsearch2-answer Claude reads top results cited answer + evidence HyDE Query Rewrite Ollama qwen2.5:7b · opt-in net −5.7pp as default SQLite chunks.db 950K rows · text + metadata FTS5 index · 8.6ms lookup ChromaDB (BGE-M3) fulldisk-1k-bge · 1024-dim HNSW index · primary vectors ChromaDB (Cohere) fulldisk-1k · --cohere to enable disabled by default