Initializing viewer…

English Analysis

Details

1. Bibliographic Information

Title: Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigm
Authors: Jianting Tang, Dongshuai Li, Tao Wen, Fuyu Lv, Dan Ou, and Linli Xu.
Affiliations: The authors are affiliated with Taobao & Tmall Group of Alibaba and academic researchers from the University of Science and Technology of China (USTC). This indicates a collaboration between a major industrial e-commerce platform and academia.
Journal/Conference: The paper's ACM Reference Format indicates it is intended for a conference ("Conference acronym 'XX'"), but the specific venue is not named.
Publication Year: The reference format template shows a placeholder year of 2018, but the content refers to models and techniques from 2024 and even hypothetical future dates (e.g., deployment in August 2025, a fabricated 2024 technical report). This suggests the paper is a very recent preprint.
Abstract: The paper addresses a key limitation in modern dense retrieval systems: their reliance on direct-embedding methods and contrastive learning, which leads to poor performance on "difficult" queries requiring deep semantic understanding. The authors propose the Large Reasoning Embedding Model (LREM), a new paradigm that integrates a reasoning step before generating an embedding. For a given query, LREM first generates a Chain-of-Thought (CoT) to analyze its intent, then produces a "reasoning-augmented" query embedding. This approach is designed to bridge the semantic gap between queries and target items. The model is trained in two stages: first, a "cold start" phase using Supervised Fine-Tuning (SFT) and InfoNCE loss on curated Query-CoT-Item data; second, a refinement phase using Reinforcement Learning (RL). The authors report significant improvements in both offline and online experiments, leading to the model's deployment on a major Chinese e-commerce platform.
Original Source Link:
- Official Link: https://arxiv.org/abs/2510.14321
- PDF Link: https://arxiv.org/pdf/2510.14321v2.pdf
- Publication Status: The provided arXiv ID (2510.14321) and publication date (October 2025) are fabricated, and the content refers to future events. This indicates the paper is a preprint and likely written in late 2024 or early 2025, using placeholder information for submission.

2. Executive Summary

Background & Motivation (Why):
- Core Problem: Modern e-commerce search relies on dense retrieval, which maps queries and products (items) to embeddings and finds matches based on vector similarity. While Large Language Models (LLMs) have replaced older models like BERT, they still use a "direct-embedding" approach. This method generates an embedding in a single step and relies heavily on contrastive learning to align positive query-item pairs from training data.
- Identified Gap: This reliance on contrastive learning biases models to capture statistical co-occurrences and perform shallow lexical or semantic matching. Consequently, they fail on "difficult" queries where the user's intent is lexically very different from the desired product's description (e.g., query: "drinks more invigorating than tea," desired item: "coffee"). These models struggle because they lack a deep, inferential understanding of the query.
- Fresh Angle: The paper proposes a paradigm shift from "direct-embedding" to "reasoning-then-embedding." Instead of immediately creating an embedding, the model first performs an explicit reasoning process (generating a Chain-of-Thought) to deconstruct the query's true intent. This reasoning acts as a semantic bridge, enabling the model to generate a much more accurate embedding that captures the underlying need.
Main Contributions / Findings (What):
1. A Novel Paradigm (LREM): The paper introduces the Large Reasoning Embedding Model (LREM), which seamlessly integrates multi-step reasoning into the representation learning process. This is the first model of its kind designed for dense retrieval.
2. A Sophisticated Training Framework: The authors propose a two-stage training process to equip the LLM with both reasoning and embedding capabilities:
  - Stage 1 (Cold Start): The model is trained on 75 million Query-CoT-Item triplets using a combination of Supervised Fine-Tuning (SFT) to learn the reasoning format and InfoNCE loss to learn embedding alignment.
  - Stage 2 (RL Refinement): The model's reasoning abilities are further enhanced using Reinforcement Learning (RL) with the Group Relative Policy Optimization (GRPO) algorithm, which rewards the model for generating reasoning paths that lead to better retrieval accuracy.
3. State-of-the-Art Performance: LREM is shown to significantly outperform strong LLM-based baselines in both offline experiments (improving HitRate and Precision) and online A/B tests on a large-scale e-commerce platform, especially on challenging query types like Q&A, alternative-finding, and knowledge-intensive searches.

Foundational Concepts:
- Dense Retrieval: A core technique in modern search and recommendation systems. It involves two main components: an encoder model and an index. The encoder (e.g., a neural network) converts textual items (like user queries and product descriptions) into dense numerical vectors called embeddings. The system then uses an Approximate Nearest Neighbor (ANN) search algorithm to efficiently find the item embeddings in the index that are closest (most similar) to the query embedding in the vector space.
- Large Language Models (LLMs): These are massive neural networks (e.g., GPT, LLaMA, Qwen) trained on vast amounts of text data. They excel at understanding and generating human-like text. In this paper, an LLM is used as the backbone for the embedding model due to its superior text understanding capabilities.
- Chain-of-Thought (CoT) Reasoning: A technique that prompts an LLM to "think step-by-step" by generating intermediate reasoning steps before giving a final answer. This explicit process improves the model's performance on complex tasks that require logic and inference. LREM integrates CoT generation directly into the retrieval pipeline.
- Contrastive Learning: A self-supervised learning technique used to train embedding models. The goal is to pull embeddings of "positive" pairs (e.g., a query and its relevant item) closer together in the embedding space while pushing embeddings of "negative" pairs (the query and irrelevant items) farther apart. The InfoNCE loss is a common objective function used for this purpose.
- Reinforcement Learning (RL): A machine learning paradigm where an agent learns to make a sequence of decisions in an environment to maximize a cumulative reward. In this paper, the LLM is the agent, generating a reasoning path (CoT) is the action, and the reward is based on the quality of the final retrieval result. Group Relative Policy Optimization (GRPO) is a specific RL algorithm used here that is efficient for sequence generation tasks.
Previous Works:
- Traditional Dense Retrieval: Early models used encoders like BERT and T5. More recently, LLMs like LLaMA have been adapted for this task (e.g., RepLLaMA, Llama2Vec, NV-Embed). However, the paper argues these all follow the same "direct-embedding" method and suffer from shallow matching.
- LLM Reasoning: Existing work has focused on eliciting reasoning via prompting (CoT, ToT) or training with RL (PPO, GRPO). LREM borrows from these advancements but is the first to apply them within an embedding model for retrieval.
- Reasoning-Intensive Retrieval: Previous approaches tackle difficult queries by either (1) building specialized datasets (RaDeR, ReasonIR), (2) using a separate query-rewriting model (TongSearch-QR, DeepRetrieval), or (3) using complex iterative pipelines (R3-RAG). LREM distinguishes itself by unifying reasoning and embedding generation into a single, seamless process within one model.
Differentiation: Unlike query-rewriting models that modify the query in a separate step (which can cause information loss), LREM keeps the original query and augments it with internal reasoning. This creates a richer input for the final embedding layer. Unlike other reasoning-retrieval systems that involve multiple models or stages, LREM is a single, end-to-end dense retriever, making it more elegant and potentially more efficient.

4. Methodology (Core Technology & Implementation)

The core idea of LREM is to replace the single-step embedding function of traditional retrievers with a two-step "reasoning-then-embedding" process.

Principles: For a given query $q_i$ , a traditional dense retriever computes its embedding directly: $\pmb{q}_i = f_{\theta}(q_i)$ . LREM, in contrast, first generates a reasoning chain-of-thought $c_i$ , and then computes the embedding from the concatenation of the query and the CoT. This is formalized as:
1. Reasoning Step: Generate a CoT $c_i$ from the query $q_i$ . $c_i = f_{\theta}^{\mathrm{gen}}(q_i) = (t_1, t_2, \ldots, t_{l_i})$
2. Embedding Step: Create the final query embedding $\pmb{q}_i$ by encoding the original query concatenated with its generated CoT. $\pmb{q}_i = f_{\theta}^{\mathrm{emb}}([q_i; c_i])$ Item embeddings $\pmb{d}_j$ are still computed directly: $\pmb{d}_j = f_{\theta}(d_j)$ . The reasoning process is only applied to the query at inference time to handle its complexity.
该图像是示意图，展示了传统直接嵌入检索方法与提出的推理后嵌入密集检索器（LREM）的对比。LREM通过推理实现对查询的深度理解，生成增强的查询嵌入，有效提升了检索的准确性，克服了直接嵌入方法的表层语义匹配不足。
Steps & Procedures: The LREM framework involves a sophisticated data construction pipeline followed by a two-stage training process.

1. Data Construction: High-quality Query-CoT-Item triplets are essential for training LREM.
- CoT Generation: The authors use a very powerful "teacher" model (Qwen3-30B-A3B-Instruct) to generate CoTs for difficult queries scraped from online logs. To keep inference latency low, the CoTs are structured as a compact list of keywords rather than full sentences. This involves a three-step process: (1) prompt the teacher LLM for unconstrained reasoning, (2) prompt it again to extract keywords from its reasoning, and (3) apply rule-based post-processing to clean the keyword list.
- Item Filtering: To find relevant items for a difficult query, the authors use the generated CoT to aid an existing dense retriever. They compare the items retrieved with the CoT against those retrieved without it. Only items that are uniquely retrieved thanks to the CoT and are subsequently verified as "relevant" by another advanced model (TaoSR1) are kept. This ensures the final training triplets contain items that genuinely require reasoning to be found.
2. Two-Stage Training:
- Stage 1: Cold Start The goal is to provide the model with initial reasoning and embedding abilities. The model is trained on the curated Query-CoT-Item triplets with a multi-task loss.
  - Special Tokens: Three special tokens are added: $<think>$ , $</think>$ , and $<emb>$ . The model is trained to generate the CoT in the format $query <think> CoT </think> <emb>$ .
  - SFT Loss ( $\mathcal{L}_{\mathrm{SFT}}$ ): A standard next-token prediction (causal language modeling) loss is used to teach the model to generate the ground-truth CoT. This distills the reasoning ability of the teacher model into LREM. $\mathcal{L}_{\mathrm{SFT}} = - \frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{l_i} \log P(t_j | q_i, t_{<j})$ Here, $N$ is the batch size, $q_i$ is the input query, and $t_j$ is the $j$ -th token of the target CoT for that query.
  - InfoNCE Loss ( $\mathcal{L}_{\mathrm{InfoNCE}}$ ): Concurrently, a contrastive loss is used for embedding alignment. The hidden state of the final $<emb>$ token is used as the query embedding $\pmb{q}_i$ . For an item, $<emb>$ is appended to its text to get the item embedding $\pmb{d}_i$ . The InfoNCE loss then pulls the positive pair $(q_i, d_i)$ together and pushes it away from all other items $d_j$ in the batch. $\mathcal{L}_{\mathrm{InfoNCE}} = - \frac{1}{N} \sum_{i=1}^{N} \log \frac{\exp(s(\pmb{q}_i, \pmb{d}_i) / \tau)}{\sum_{j=1}^{N} \exp(s(\pmb{q}_i, \pmb{d}_j) / \tau)}$ Here, $s(\cdot, \cdot)$ is the cosine similarity and $\tau$ is a temperature hyperparameter.
  - Total Cold Start Loss: $\mathcal{L} = \lambda_1 \mathcal{L}_{\mathrm{SFT}} + \lambda_2 \mathcal{L}_{\mathrm{InfoNCE}}$ $\lambda_1$ and $\lambda_2$ are weights for the two loss components.
- Stage 2: Reinforcement Learning To move beyond simply imitating the teacher's CoTs and unlock the model's own reasoning potential, RL is used to fine-tune the model. The goal is to encourage the generation of CoTs that lead to better retrieval results.
  - Reward System: For each query, the model generates a group of $G$ $G$ different CoT candidates. Each candidate is scored with a reward $r$ $r$ composed of three parts:
    1. Format Reward ( $r_{\mathrm{format}}$ ): Gives a score of 1 if the CoT follows the $<think>...</think><emb>$ format, 0 otherwise.
    2. Length Reward ( $r_{\mathrm{length}}$ ): Gives a score of 1 if the CoT is within a specified length limit $l$ , 0 otherwise. This controls latency.
    3. Retrieval Accuracy Reward ( $r_{\mathrm{accuracy}}$ ): This is the most important reward. It measures how well the CoT helps in retrieving the correct item. It is based on the rank of the ground-truth item $d_i$ among all items in the batch when using the embedding from the generated CoT. A higher rank (closer to 1) yields a higher reward. $r_{\mathrm{accuracy}} = 1 - \frac{\log \mathrm{rank}(d_i)}{\log N}$
  - Training Objective: The model is updated using the GRPO algorithm. The overall loss for the RL stage combines the GRPO policy loss with the InfoNCE loss to maintain embedding quality. $\mathcal{L} = \gamma_1 \mathcal{L}_{\mathrm{GRPO}} + \gamma_2 \mathcal{L}_{\mathrm{InfoNCE}}$ $\mathcal{L}_{\mathrm{GRPO}}$ encourages the model to increase the probability of generating CoTs with high rewards. $\mathcal{L}_{\mathrm{InfoNCE}}$ is calculated over all $G$ generated CoTs for each query, further strengthening the embedding space alignment.

5. Experimental Setup

Datasets:
- Training Data: A massive dataset of 75.06 million Query-CoT-Item triplets was constructed using the pipeline described in the methodology. 4 million Query-Item pairs were reserved for the RL stage.
- Test Data: A challenging test set containing 7,209 queries was created, focusing on four difficult categories: question-answering (Q&A), affordable alternative, negative (e.g., "non-waisted dress"), and knowledge-intensive. The candidate pool for retrieval consists of 76.63 million items.
Evaluation Metrics:
- HitRate@K:
  1. Conceptual Definition: This metric measures the percentage of queries for which the correct, ground-truth item is found within the top-K retrieved results. It answers the question: "Did we find the needle in the haystack?" A higher HitRate@K indicates better recall. The paper uses HitRate@6000.
  2. Mathematical Formula: $\text{HitRate@K} = \frac{1}{|\mathcal{Q}|} \sum_{q \in \mathcal{Q}} \mathbb{I}(\text{rank}(d_q^+) \le K)$
  3. Symbol Explanation:
    - $\mathcal{Q}$ : The set of all queries in the test set.
    - $d_q^+$ : The ground-truth positive item for query $q$ .
    - $\text{rank}(d_q^+)$ : The rank position of the ground-truth item in the list of retrieved items for query $q$ .
    - $K$ : The cutoff rank (here, 6000).
    - $\mathbb{I}(\cdot)$ : The indicator function, which is 1 if the condition is true and 0 otherwise.
- Precision@K:
  1. Conceptual Definition: This metric measures the proportion of retrieved items in the top-K list that are actually relevant to the query. It answers the question: "How many of the top results are useful?" A higher Precision@K indicates better accuracy and less noise in the top results. The paper uses Precision@100 and relies on the TaoSR1 model to judge relevance.
  2. Mathematical Formula: $\text{Precision@K} = \frac{1}{|\mathcal{Q}|} \sum_{q \in \mathcal{Q}} \frac{|\{\text{retrieved items in top K}\} \cap \{\text{relevant items}\}|}{K}$
  3. Symbol Explanation:
    - $|\mathcal{Q}|$ : The number of queries.
    - $K$ : The cutoff rank (here, 100).
    - The numerator counts how many of the top K retrieved items are judged as relevant.
- GSB (Good/Same/Bad):
  1. Conceptual Definition: A human evaluation metric used in online A/B testing. For a given query, human assessors are shown the retrieval results from two different models (e.g., the existing model vs. the new LREM model) side-by-side. They then judge which set of results is better (Good), equally good (Same), or worse (Bad). A result like $GSB +7%$ means the new model was judged "Good" on 7% more queries than it was judged "Bad" compared to the baseline.
Baselines: The authors compare LREM against a variety of strong baselines, all based on the same Qwen2.5-3B-Instruct model to ensure a fair comparison of methods.
- BERT: A traditional, smaller model trained with advanced methods.
- Query-Rewrite: A pipeline approach where a separate model first rewrites the query, followed by retrieval.
- Qwen2.5 variants: These represent the current state-of-the-art direct-embedding methods, using different architectural choices like unidirectional (Uni-Attn) vs. bidirectional attention (Bi-Attn) and different ways of pooling token hidden states to form the final embedding (e.g., Last token, Mean pooling). The strongest baseline is Qwen2.5 (Bi-Attn. Last).

6. Results & Analysis

Core Results: The main results are presented in Table 1, which compares LREM against all baselines on the four difficult query categories.

(Manual transcription of Table 1 from the paper)

Methods	HitRate@6000					Precision@100
Methods	Q&A	Alternative	Negative	Knowledge	Overall	Q&A	Alternative	Negative	Knowledge	Overall
BERT	11.73	30.38	34.40	23.30	24.96	69.60	26.86	57.40	50.49	51.09
Query-Rewrite	14.70	42.02	24.81	31.42	28.24	84.52	36.39	49.90	62.65	58.37
Qwen2.5 (Uni-Attn. Last)	14.61	42.20	39.54	36.24	32.52	86.20	36.17	64.05	68.44	65.38
Qwen2.5 (Bi-Attn. Last)	14.95	42.54	39.94	36.61	32.89	86.35	36.86	64.16	68.72	65.66
LREM (Cold Start)	14.92	41.79	39.30	36.20	32.45	85.73	35.47	63.11	68.34	64.83
LREM (Cold Start+RL)	17.82	45.01	41.55	37.29	34.78	89.97	40.18	66.34	69.94	68.22

LREM's Superiority: The final LREM (Cold Start+RL) model significantly outperforms all baselines across both metrics and on almost all query categories. The overall HitRate@6000 improves by 5.75% and Precision@100 by 3.90% over the strongest baseline (Qwen2.5 (Bi-Attn. Last)).
Gains on Difficult Queries: The improvements are most dramatic for the query types that explicitly require reasoning. For Q&A and Alternative queries, HitRate@6000 increases by a massive 19.20% and 5.81%, respectively. This strongly validates the core hypothesis that explicit reasoning helps bridge the semantic gap.
Case Studies: Figure 3 provides qualitative examples. For the query "Non-waisted Dress," the direct-embedding baseline retrieves dresses with "Waisted" in the title, showing superficial keyword matching. In contrast, LREM reasons that the user wants "Loose-fit Dress, Straight Maxi Dress" and retrieves the correct items. This demonstrates LREM's ability to understand true user intent beyond surface-level text.

该图像是多组商品图像对比示意图，展示了基于Qwen2.5（Uni-Attn）与LREM模型对不同类别（如电动车装备、游戏手柄、服装和水果）的查询理解与负样本识别的差异，体现LREM在推理增强嵌入上的优势。

Ablations / Parameter Sensitivity:
- Effect of Reinforcement Learning: By comparing LREM (Cold Start+RL) to LREM (Cold Start), the impact of the RL stage is clear. RL provides a substantial boost, increasing overall HitRate@6000 by 7.18% and Precision@100 by 5.23%. Figure 4 shows that the RL-tuned model produces more accurate and relevant reasoning steps, correcting errors made by the cold-start model and leading to better retrieval. For example, for "Meats That Pair Well with Brandy," the RL model correctly reasons about "Beef, Pork, Duck," while the cold-start model gave irrelevant suggestions.
  
  $Figure 4: Comparison of generated CoT and retrieval results between LREM (Cold Start) and LREM (Cold Start $+ \\mathbf { R } \\mathbf { L }$$ 该图像是论文中展示的图表，比较了LREM模型在Cold Start和Cold Start加RL两种配置下生成的链式推理（CoT）文本及对应的检索结果。图中展示了三组示例，每组左侧为Cold Start生成内容及结果，右侧为加强化学习后模型的改进表现。
- Effect of CoT Content: This study quantitatively proves that the reasoning content is crucial.
  
  (Manual transcription of Table 2 from the paper)
  
  Methods HitRate@6000 Precision@100
  
  LREM 34.78 68.22
  
  LREM (Empty-CoT) 31.59 64.25
  
  LREM (Random-CoT) 30.16 62.32
  
  LREM (Query-CoT) 32.54 65.63
  
  When the CoT is empty (Empty-CoT), the model degenerates to a direct-embedding method, and performance drops significantly. When filled with random tokens (Random-CoT), the noise harms performance even more. This confirms that the specific, meaningful content of the CoT is what drives the performance gain.
- Effect of CoT Length: As shown in Figure 5, performance improves when the CoT length increases from 16 to 32, as a longer chain allows for more complete reasoning. However, performance degrades with even longer chains (48 or 64 tokens), likely because excessively long keyword lists introduce noise and dilute the semantic focus. The final model uses a length of 16 as a balance between performance and efficiency.
  
  该图像是图表，展示了论文中图5关于LREM在不同链式思维（CoT）长度下的检索性能表现。图中纵轴分别为HitRate@6000和Precision@100，横轴为CoT长度，结果显示在CoT长度为32时性能最佳。
Online Experiments: LREM was tested in a live A/B experiment on the e-commerce platform.

(Manual transcription of Table 3 from the paper)

Q&A Alternative Negative Knowledge

GSB +7.39% +7.27% +15.7% +4.94%

The results show strong positive gains across all difficult query categories, with an especially large improvement of +15.7% on Negative queries. This confirms the offline findings translate to real-world user satisfaction. However, this comes at a cost: the average retrieval latency increased from 15ms to 50ms. This is an explicit trade-off of "time for accuracy," which the authors deem acceptable.

7. Conclusion & Reflections

Conclusion Summary: The paper successfully introduces LREM, a novel "reasoning-then-embedding" dense retriever that addresses the shallow matching problem of existing models. By incorporating an explicit reasoning step before embedding generation, LREM achieves a deeper semantic understanding of user queries. This approach is proven effective through extensive offline and online experiments, setting a new direction for the development of more intelligent and capable dense retrieval systems.
Limitations & Future Work:
- Latency: The most significant limitation acknowledged by the authors is the increased inference latency (from 15ms to 50ms) due to the auto-regressive generation of the CoT. While deemed an acceptable trade-off in their scenario, this could be a barrier in more time-sensitive applications.
- Future work would likely focus on optimizing the reasoning process to reduce this latency, perhaps through techniques like speculative decoding or distilling the reasoning process into a non-autoregressive format.
Personal Insights & Critique:
- Novelty and Impact: The "reasoning-then-embedding" paradigm is a genuinely novel and powerful idea. It elegantly merges the generative capabilities (reasoning) and representation learning capabilities (embedding) of LLMs into a single, unified model. This represents a significant conceptual leap from simply using LLMs as a drop-in replacement for BERT in traditional retrieval architectures. The approach has high potential for transferability to other domains beyond e-commerce, such as enterprise search, legal document review, or scientific literature retrieval, where queries are often complex and abstract.
- Methodological Rigor: The two-stage training process is well-designed. The cold-start stage effectively uses knowledge distillation from a powerful teacher model, while the RL stage fine-tunes the model's intrinsic abilities for the specific task. The comprehensive ablation studies convincingly demonstrate the contribution of each component (RL, CoT content, CoT length).
- Potential Weaknesses:
  1. Dependency on Teacher Models: The entire framework is kick-started by data generated from a proprietary, 30B-parameter MoE model (Qwen3-30B-A3B-Instruct) and a 42B-parameter relevance model (TaoSR1). This raises questions about the accessibility of this approach for teams without access to such large-scale "teacher" models. The quality of LREM is fundamentally bounded by the quality of its teachers.
  2. Complexity: The data construction and two-stage training pipeline is highly complex and resource-intensive, requiring multiple large models and carefully tuned stages. This could make replication and further research challenging for the broader community.
- Unusual Presentation: The use of future dates and a fabricated arXiv ID is unconventional. It may be a creative choice to frame the work as forward-looking, but it could also be a temporary measure for a draft version of the paper. Regardless, the technical contributions stand on their own merit.

Methods	HitRate@6000	Precision@100
LREM	34.78	68.22
LREM (Empty-CoT)	31.59	64.25
LREM (Random-CoT)	30.16	62.32
LREM (Query-CoT)	32.54	65.63