Evaluating relationship traversal for multi-source retrieval

Result

In an internal evaluation run on July 26, 2026, relationship traversal increased ALL@10 from 25% to 63% within the tested retrieval configuration.

ALL@10 is the percentage of questions for which every required supporting passage appeared among the top ten retrieval results. It is a retrieval metric, not a measure of final-answer accuracy.

Evaluation setting

The evaluation used 100 questions sampled from MuSiQue-Ans development data with a fixed, stratified seed: 52 two-hop, 31 three-hop, and 17 four-hop questions. The supporting and distractor paragraphs for those questions were pooled into one corpus of 1,730 unique passages.

The corpus was processed through the Consilience application pipeline. The two conditions used the same questions, corpus, and retrieval pipeline. The comparison removed the relationship-traversal contribution in one condition and retained the shipped fusion configuration in the other.

R@10 measures the average fraction of each question’s supporting passages found in the top ten. ALL@10 is stricter: a question counts only when all of its supporting passages are present.

What this result supports

Within this sample, pooled corpus, and retrieval configuration, relationship traversal surfaced more of the complete evidence sets required by the multi-source questions. The result supports further investigation of connected retrieval for questions whose evidence is distributed across passages.

It does not show that graph retrieval is best for every query. In the same research program, keyword weight behaved differently on other corpora. Exact lookup and multi-source reasoning can favor different retrieval mixtures.

Limits

• This was an internal, unaudited evaluation.
• It measured retrieval only. No reader or answer-generation accuracy was measured.
• The sample contained 100 questions and 1,730 pooled passages. Published MuSiQue settings often use a larger pool, so the absolute scores must not be compared with external systems without matching the protocol.
• The comparison was an ablation within one Consilience retrieval configuration, not a comparison with external products.
• The result does not establish a universal causal claim about all graphs, corpora, models, or tasks.
• The retained report specifies a reproduction procedure, but this public note is not an independently reproduced or externally audited benchmark report.

Interpretation

The narrow conclusion is the useful one: for these 100 MuSiQue-Ans questions, in this pooled corpus and with this pipeline, the condition with relationship traversal retrieved all required evidence in the top ten for 63% of questions, compared with 25% when that contribution was removed.

The next work is to evaluate larger matched settings, measure answer quality separately, and test query-dependent retrieval mixtures across both multi-source benchmarks and representative user corpora.