LLM | Enjun Du（杜恩俊）

EviRank: Structured Relevance Evidence for Multimodal Image Re-ranking

Fri, 31 Jul 2026 00:00:00 +0000

From similarity to semantic verification

Real image-search requests are rarely one-dimensional. “Find this shirt in pink” asks the system to preserve the object and style, change one attribute, and ignore incidental factors such as the background. A single similarity score hides these separate requirements; free-form chain-of-thought may omit constraints or claim visual evidence that was never verified.

EviRank reframes multimodal image re-ranking as an explicit semantic constraint-satisfaction problem.

The method supports text-only, image-only, and composed queries through the same evidence interface. It can sit on top of any off-the-shelf retriever and re-rank its top candidates without retraining the retriever.

Structured Evidence Frames

An MLLM teacher parses each query into a typed Evidence Frame. The frame spans six semantic slots:

Entities
Attributes
Actions
Relations
Scene
Key details

Within each slot, a short criterion is marked as:

Required — what a relevant candidate must contain;
Forbidden — what would contradict the intended result;
Ignorable — what should not influence the decision.

This structure makes semantic coverage visible. It also unifies different query modalities: a reference image contributes what must be preserved, modification text contributes what must change, and incidental context becomes explicitly ignorable.

Evidence-conditioned re-ranking

EviRank first applies deterministic, slot-wise rubric scoring and then uses an evidence-grounded listwise comparison for global refinement. Both stages read the same Evidence Frame, so the final ranking can be traced back to which constraints each candidate satisfies or violates.

The paper studies four operating points:

EviRank-mini — rubric-only re-ranking with no MLLM at test time;
EviRank — a distilled lightweight student;
EviRank-plus — rubric scoring with Gemini-3-Flash refinement;
EviRank-pro — rubric scoring with Gemini-3-Pro refinement.

Evidence as supervision

The structured frame does more than explain the teacher. Per-slot scores, hard-pair distinctions, and typed constraints form a naturally decomposable supervision bundle for distillation. The student consumes the raw query and candidate images at inference time, while learning from the teacher’s explicit evidence during training.

Results

EviRank is evaluated on five benchmarks across text-to-image, image-to-image, and composed image retrieval. The full system achieves state-of-the-art results across the benchmark suite, while the distilled student retains more than 90% of the teacher’s capability at substantially lower cost.

The strongest gains appear in settings where fine-grained constraints matter most. On FashionIQ, for example, the method improves Recall@10 by up to 8.27 points over ImageScope on the Shirt category. Even EviRank-mini, which uses no MLLM at test time, matches or exceeds strong reasoning-based re-rankers—evidence that the structured representation itself is doing meaningful work.

LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger

Thu, 30 Jul 2026 00:00:00 +0000

Why trajectory faithfulness matters

A multimodal agent does more than emit an answer. It observes images, calls tools, retrieves evidence, forms intermediate claims, and sometimes repairs its own trajectory. Final-answer accuracy compresses all of this into one bit: correct or incorrect. It cannot reveal whether the answer was grounded, guessed from a language prior, or reached through errors that happened to cancel out.

LedgerMind treats the agent trajectory as an auditable state machine, not a free-form scratchpad.

This perspective targets four recurring failure modes: unsupported intermediate claims, citation-backed entity hallucination (Phantom Grounding), unnecessary deep reasoning on simple queries, and new unsupported content introduced during repair.

The LedgerMind runtime

LedgerMind places a Structured Evidence Ledger at the center of the agent. Each perception or retrieval result is normalized into a typed entry with its source, confidence, lifecycle state, and dependencies. Downstream claims may cite only active entries, turning provenance from a prompting preference into an execution constraint.

The runtime proceeds in two stages:

Evidence acquisition and ledger state. An adaptive dispatcher chooses either a direct path or the full tool pipeline. Tool outputs are recorded as active ledger entries rather than mixed into an unstructured text history.
Grounding-governed intervention. Every decision claim passes support coverage, Entity Consistency Check (ECC), and Numeric Coherence Check (NCC). A passing claim becomes the grounded final answer; a failing claim can trigger only predefined evidence-, action-, or trajectory-level repair.

Three-layer grounding

Support coverage checks whether the claim is substantively covered by its cited evidence.
ECC requires conclusion-level entities to appear in the cited evidence, with alias handling.
NCC requires numerical values to match cited values under type-aware tolerances.

Together, these checks catch claims that cite valid evidence IDs while quietly introducing a different entity, year, count, or measurement.

Adaptive execution and constrained repair

The Adaptive Dual-Path Dispatcher avoids routing every query through an expensive reasoning pipeline. Simple or knowledge-oriented queries can take the direct path, while questions requiring perception, retrieval, and multi-step reasoning use the full pipeline.

When verification fails, the Event-Triggered Verification-and-Repair engine applies typed state transitions. It may drop or refresh evidence, retry or switch an action, or stop and answer—but new evidence must still come from tools. This gives repair a provenance non-amplification guarantee.

Results

LedgerMind is evaluated on six public benchmarks—VTC-Bench, MMStar, MMMU, MMMU-Pro, EMMA, and MC-Search—plus the Hard-200 stress set, using six frontier MLLMs from four vendors.

On VTC-Bench, LedgerMind with Gemini-3-Flash reaches 58.9%, while the same runtime improves GPT-4o by 23.3 points.
On EMMA, it reaches 58.29% overall, a 9.58-point gain over the strongest thinking-mode baseline.
On Hard-200, every tested backbone improves by 11.2 to 19.7 points overall, with no negative sub-source result.
Trajectory-level audits show gains beyond answer accuracy, especially against unsupported reasoning and Phantom Grounding.

The ablations reinforce the design: removing the ledger causes the largest degradation, free-form repair is substantially weaker than typed repair, and disabling ECC/NCC most strongly harms the difficult split.

GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environments

Thu, 06 Mar 2025 00:00:00 +0000

Abstract

Graph Foundation Models need large, diverse graph corpora, but real-world graphs are often small, private, or expensive to annotate. GraphMaster turns graph synthesis into a coordinated agent workflow: four specialized LLM agents iteratively expand a text-attributed graph while preserving both its meaning and topology.

Motivation

The goal is not merely to make a graph larger. The new nodes must read naturally, connect plausibly, and remain useful for downstream learning.

Classical augmentation methods mainly manipulate topology and cannot generate meaningful textual attributes. Direct LLM generation introduces a different set of problems: whole graphs exceed context windows, locally plausible additions may violate global structure, and hallucinated nodes or edges can silently corrupt the training signal.

Method

GraphMaster decomposes synthesis into four accountable roles:

Manager Agent — Selects semantic or topological enhancement according to the current graph state and coordinates the workflow.
Perception Agent — Overcomes context-window limitations via semantic-aware community detection, mode-adaptive seed selection, and hierarchical PPR-based diffusion sampling to extract representative subgraphs.
Enhancement Agent — Generates new nodes and edges conditioned on extracted knowledge, with dual-mode generation for semantic coherence and structural fidelity.
Evaluation Agent — Scores semantic and structural quality, then decides whether another refinement round is needed.

Experimental Results

Evaluated on 6 data-limited benchmarks with 4 GNN architectures (GCN, JKNET, GraphSage, GAT) on 8x A100 GPUs using QwQ-32B as the base LLM.

GraphMaster consistently outperforms all baselines across all datasets. The bottom row (blue) shows GraphMaster achieving the highest accuracy and F1 scores on every benchmark.

Graph Feature Preservation

The synthesized graphs maintain high fidelity: KS statistic 0.357 (p=0.059) for degree distribution, 0.835 clustering coefficient similarity, and 0.988 label homogeneity — indicating near-perfect structural preservation.

Ablation Study

Removing the Evaluation Agent causes the largest performance drop, confirming the critical role of iterative quality control. Each agent contributes uniquely to the final synthesis quality.