1. Bibliographic Information

1.1. Title

Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender System

1.2. Authors

Sein Kim*, Hongseok Kang*, Seungyoon Choi, Donghyun Kim, Minchul Yang, Chanyoung Park†. The authors are primarily affiliated with KAIST (Korea Advanced Institute of Science and Technology), with some also associated with NAVER Corporation. Their research backgrounds appear to be in recommender systems and potentially large language models, given the paper's focus.

1.3. Journal/Conference

Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '24), August 25-29, 2024, Barcelona, Spain. KDD is a highly prestigious and influential conference in the fields of data mining, data science, and knowledge discovery. Publication at KDD signifies a high level of academic rigor and impact in these domains.

1.4. Publication Year

2024

1.5. Abstract

Collaborative filtering recommender systems (CF-RecSys) excel at enhancing user experience but struggle in cold scenarios due to sparse user-item interactions. Recent efforts leverage modality information (e.g., text, images) and Large Language Models (LLMs) to address cold-start, but they often underperform traditional CF models in warm scenarios because they lack collaborative knowledge. This paper introduces A-LLMRec, an efficient All-round LLM-based Recommender system designed to perform well in both cold and warm scenarios. Its core idea is to enable an LLM to directly access and utilize the collaborative knowledge embedded in a pre-trained state-of-the-art CF-RecSys. This is achieved by jointly exploiting the LLM's emergent abilities and the high-quality user/item embeddings from the CF-RecSys. A-LLMRec offers two key advantages: it is model-agnostic, allowing integration with various CF-RecSys, and efficient, as it eliminates the extensive fine-tuning typically required for LLM-based recommenders. Extensive experiments across diverse real-world datasets demonstrate A-LLMRec's superior performance in cold/warm, few-shot, cold user, and cross-domain scenarios. Beyond recommendations, A-LLMRec also shows potential in natural language generation tasks, such as favorite genre prediction, based on its understanding of collaborative knowledge.

1.6. Original Source Link

https://arxiv.org/abs/2404.11343 (Published as a preprint on arXiv, subsequently accepted to KDD '24)

2. Executive Summary

2.1. Background & Motivation

The core problem the paper addresses is the inherent trade-off in performance between different types of recommender systems, particularly under varying data sparsity conditions.

• Collaborative Filtering Recommender Systems (CF-RecSys): These systems are the cornerstone of recommendations, relying on past user-item interactions to find similar users or items. They are highly effective in warm scenarios (where there are abundant interactions) but severely struggle with the cold-start problem. The cold-start problem arises when there are new users or items with very few or no interactions, making it difficult to build collaborative knowledge for them.

• Modality-aware Recommender Systems: To counter the cold-start problem, recent research has leveraged modality information (e.g., text descriptions, images) of users and items. These systems often use pre-trained modality encoders (like BERT for text or Vision-Transformer for images) to generate rich embeddings that can inform recommendations even without extensive interaction data.

• Large Language Model (LLM)-based Recommender Systems: The advent of LLMs has further pushed this trend, using their vast pre-trained knowledge and advanced language understanding abilities to extract and integrate modality information. These systems have shown effectiveness in cold scenarios and cross-domain scenarios.

  The Gap/Challenge: The paper observes a critical limitation: while modality-aware and LLM-based recommenders excel in cold scenarios by leveraging rich content information, they underperform simple traditional collaborative filtering models under warm scenarios. This underperformance is attributed to their lack of collaborative knowledge, as their heavy reliance on textual information makes them less effective at capturing the intricate patterns of user preferences derived from extensive interactions that traditional CF-RecSys are optimized for. However, warm scenarios are crucial for real-world applications, generating the majority of user interactions and revenue.

Paper's Entry Point / Innovative Idea: The paper's innovative idea is to bridge this gap by creating an "all-round" system that combines the strengths of both approaches. It aims to enable an LLM to directly leverage the collaborative knowledge contained within a pre-trained state-of-the-art CF-RecSys. This allows for the joint exploitation of the LLM's emergent abilities (complex reasoning, language generation) and the high-quality user/item embeddings already learned by the CF-RecSys. The key is an alignment network that connects the CF-RecSys embeddings to the LLM's token space.

2.2. Main Contributions / Findings

The paper's primary contributions are:

1. Proposed A-LLMRec: An LLM-based recommender system that directly leverages collaborative knowledge from a pre-trained state-of-the-art CF-RecSys to achieve "all-round" performance across various scenarios.
2. Novel Alignment Mechanism: A-LLMRec introduces an alignment network that bridges the CF-RecSys and the LLM by mapping collaborative knowledge (item embeddings) to the LLM's token space. This alignment network is the only trainable neural network in the system.
3. Model-Agnosticism: The proposed framework is model-agnostic, meaning it can integrate with any existing CF-RecSys as its backbone. This makes it highly practical for services already using their own recommender models, allowing them to readily incorporate LLM capabilities.
4. Efficiency: A-LLMRec is highly efficient because it does not require fine-tuning either the CF-RecSys or the LLM. This significantly reduces training and inference time (e.g., 2.5-3 times faster training, 1.71 times faster inference than TALLRec), addressing a major bottleneck of many LLM-based recommenders.
5. Superior Performance Across Scenarios: Extensive experiments demonstrate A-LLMRec's superiority over existing CF-RecSys, modality-aware, and LLM-based recommenders in cold/warm, few-shot, cold user, and cross-domain scenarios. It outperforms traditional CF-RecSys in warm scenarios and other LLM-based recommenders in cold scenarios.
6. Potential for Language Generation: Beyond pure recommendation, A-LLMRec showcases its ability to generate natural language outputs (e.g., favorite genre prediction) based on its understanding of collaborative knowledge, highlighting the LLM's emergent abilities when properly integrated.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

• Recommender System (RecSys): A type of information filtering system that seeks to predict the "rating" or "preference" a user would give to an item. Recommender systems are ubiquitous in e-commerce, social media, and content platforms, helping users discover new items (products, movies, music, articles) they might like.

• Collaborative Filtering (CF): The most common and successful approach in recommender systems. The core idea is that users who agreed in the past will agree in the future, and similar items are often liked by similar users.

  • User-based CF: Recommends items to a user that similar users have liked.
  • Item-based CF: Recommends items that are similar to items the user has liked in the past.
  • Matrix Factorization: A prominent technique in CF where the user-item interaction matrix is decomposed into two lower-rank matrices: a user-latent factor matrix and an item-latent factor matrix. These latent factors represent underlying features or characteristics that influence user preferences and item properties.

• Cold-Start Problem: A significant challenge in recommender systems where it is difficult to make recommendations for new users or new items due to a lack of historical interaction data.

  • Cold User: A new user with very few or no past interactions, making it hard to determine their preferences.
  • Cold Item: A new item with very few or no past interactions from any user, making it hard to determine its appeal.
  • Warm Scenario: A situation where there is ample historical interaction data for both users and items.
  • Cold Scenario: A situation where there is sparse or limited historical interaction data, either for new users/items or generally sparse datasets.

• Modality Information: Refers to different types of data (or "modalities") associated with users or items, beyond just interaction IDs. Examples include:

  • Textual Modality: Item titles, descriptions, reviews, user profiles (e.g., demographics).
  • Visual Modality: Item images, video thumbnails.
  • Audio Modality: Music tracks. Leveraging modality information helps address cold-start by providing rich content-based signals when collaborative knowledge is scarce.

• Pre-trained Modality Encoders: Deep learning models that have been pre-trained on large datasets to understand specific data modalities.

  • BERT (Bidirectional Encoder Representations from Transformers): A pre-trained language model developed by Google, capable of understanding the context of words in text. It's often used to generate text embeddings (vector representations of text).
  • Vision-Transformer (ViT): A model that applies the Transformer architecture (originally for natural language processing) directly to image classification tasks, breaking images into patches and processing them like sequences of words. Used for image embeddings.
  • Sentence-BERT (SBERT): A modification of BERT that produces semantically meaningful sentence embeddings. It can be used to compare sentence similarity efficiently.

• Large Language Models (LLMs): Extremely large neural networks, often based on the Transformer architecture, pre-trained on massive amounts of text data. They exhibit advanced natural language understanding (NLU) and natural language generation (NLG) capabilities, including reasoning, summarization, translation, and code generation.

  • In-context Learning: The ability of LLMs to learn new tasks or adapt to new information by only using instructions or examples provided within the input prompt, without needing explicit fine-tuning.
  • Emergent Abilities: Unexpected capabilities observed in large neural networks (especially LLMs) that are not present in smaller models but appear at scale. These abilities often include complex reasoning and problem-solving.
  • Fine-tuning: The process of taking a pre-trained model (like an LLM) and further training it on a smaller, task-specific dataset to adapt it to a particular downstream task (e.g., recommendation).
  • Parameter-Efficient Fine-Tuning (PEFT) / LoRA (Low-Rank Adaptation): Techniques designed to fine-tune large models more efficiently by only updating a small subset of parameters (e.g., adding small, low-rank matrices to the existing weight matrices) rather than the entire model.

• Sequential Recommendation: A sub-field of recommender systems that aims to predict the next item a user will interact with, based on their ordered sequence of past interactions. It often uses models capable of capturing temporal dependencies and sequential patterns.

• Embeddings: Dense vector representations of discrete entities (like users, items, or words) in a continuous vector space. The idea is that semantically similar entities are mapped to points that are close to each other in this embedding space.

  • Item Embeddings: Vector representations of items, capturing their characteristics and properties.
  • User Representations: Vector representations of users, capturing their preferences and interaction history.
  • Token Space: The embedding space where individual words or sub-word units (tokens) are represented in LLMs.

• Multi-Layer Perceptron (MLP): A fundamental type of artificial neural network consisting of at least three layers of nodes: an input layer, one or more hidden layers, and an output layer. MLPs are used for various tasks, including classification and regression, and often serve as non-linear transformations in larger models.

• Mean Squared Error (MSE): A common loss function used in regression tasks. It measures the average of the squares of the errors (the differences between the predicted values and the actual values). A lower MSE indicates a better fit of the model to the data.

• Sigmoid Function ($\sigma$): An S-shaped activation function used in neural networks, particularly in the output layer for binary classification. It squashes any real-valued number into a range between 0 and 1, which can be interpreted as a probability. The formula is $\sigma(x) = \frac{1}{1 + e^{-x}}$.

• Soft Prompt / Prompt Tuning: Techniques to adapt LLMs by adding a small, trainable sequence of continuous vectors (the "soft prompt") to the input, rather than modifying the LLM's weights directly. This allows steering the LLM's behavior for specific tasks while keeping the base model frozen.

3.2. Previous Works

The paper discusses three main categories of previous work: Collaborative Filtering, Modality-aware Recommender Systems, and LLM-based Recommender Systems.

3.2.1. Collaborative Filtering (CF)

• Foundation: CF is built on the premise of leveraging historical preferences.
• Matrix Factorization (MF): A significant advancement, modeling latent factors.
  • Examples: Probabilistic Matrix Factorization (PMF) [5, 33], Singular Value Decomposition (SVD) [30, 53].
  • Core Idea of MF: Decomposes the user-item interaction matrix $R \in \mathbb{R}^{M \times N}$ (where $M$ is number of users, $N$ is number of items) into two lower-rank matrices, $U \in \mathbb{R}^{M \times K}$ and $V \in \mathbb{R}^{N \times K}$, such that $R \approx UV^T$. Each row of $U$ represents a user's latent factors, and each row of $V$ represents an item's latent factors. The predicted rating for user $u$ on item $i$ is then $\hat{r}_{ui} = u_u \cdot v_i$.
• Deep Learning for CF:
  • AutoRec [39]: Uses autoencoders for CF.
  • Neural Matrix Factorization (NMF) [15]: Uses MLP to model user-item interactions.
• Sequential CF: Models user preferences based on sequential interaction history.
  • Caser [41], NextItNet [50]: Utilize Convolutional Neural Networks (CNNs) to capture local sequence information.
  • GRU4Rec [17]: Employs Recurrent Neural Networks (RNNs) to model user sessions.
  • SASRec [20]: A state-of-the-art sequential recommender that uses a self-attention mechanism to capture long-range dependencies in user behavior sequences. It models the sequence of items a user has interacted with, where the prediction for the next item is based on the representations learned from this sequence.

3.2.2. Modality-aware Recommender Systems

These systems use modality information (text, images) to enhance recommendations, particularly in cold scenarios.

• Early Approaches: CNNs for visual features, Mahalanobis distance [31].
• Modern Approaches with Pre-trained Encoders:
  • NOVA [27], DMRL [28]: Integrate pure item embeddings and text-integrated item embeddings using attention mechanisms.
  • MoRec [51]: Uses pre-trained modality encoders (like BERT, Vision-Transformer) to project raw modality features (e.g., item texts, images), replacing standard item IDs with these richer modality embeddings in CF models.
  • CTRL [25]: Pre-trains CF models using contrastive learning on paired tabular data and textual data, then fine-tunes for specific tasks.
  • RECFORMER [24]: Formulates sequential recommendation as a next item sentence prediction task, modeling user preferences and item features as language representations using the Transformer architecture.

3.2.3. LLM-based Recommender Systems

Leverage the pre-trained knowledge and reasoning power of LLMs.

• In-context Learning (ICL) Approaches:
  • OpenAI-GPT with ICL [12, 16, 44]: Adapts to new tasks based on input prompt context.
  • Sanner et al. [37]: Explores various prompting styles (completion, instructions, few-shot) for recommendations using item texts and user descriptions.
  • Gao et al. [12]: Assigns LLMs the role of a recommender expert for zero-shot recommendations.
  • Limitation of ICL: These ICL approaches often underperform traditional recommendation models due to the gap between LLM training tasks and recommendation tasks.
• Fine-tuning Approaches:
  • TALLRec [2]: Addresses the gap by fine-tuning LLMs with recommendation data using LoRA [18]. It converts the recommendation task into an instruction text. TALLRec demonstrates enhanced efficacy in cold-start and cross-domain scenarios compared to traditional CF models.
  • Limitation of TALLRec: Although it fine-tunes LLMs, TALLRec primarily relies on textual information and still fails to explicitly capture the collaborative knowledge crucial in warm scenarios.

3.3. Technological Evolution

The evolution of recommender systems has progressed from purely ID-based collaborative filtering (e.g., Matrix Factorization, SASRec) which excels in warm scenarios but fails in cold-start, to modality-aware systems (e.g., MoRec, CTRL) that leverage item/user content (text, images) using pre-trained encoders to mitigate cold-start. The latest frontier is the integration of Large Language Models (e.g., TALLRec), which bring powerful natural language understanding and generation capabilities to the table, further improving cold-start and cross-domain performance. However, this progression often came with a trade-off: modality-aware and LLM-based systems, while strong in cold scenarios, typically underperformed traditional CF in warm scenarios due to a diluted focus on collaborative knowledge. This paper's A-LLMRec work fits into this timeline as an attempt to harmonize these advancements, aiming for an "all-round" solution that retains the strengths of CF in warm scenarios while fully exploiting LLM capabilities for cold scenarios and natural language generation.

3.4. Differentiation Analysis

Compared to the main methods in related work, A-LLMRec introduces several core differences and innovations:

• Versus Traditional CF (e.g., SASRec):

  • CF Core: Relies solely on user-item interaction IDs to build collaborative knowledge.
  • CF Strengths: Excellent in warm scenarios with dense interaction data.
  • CF Weaknesses: Fails in cold scenarios due to lack of modality information.
  • A-LLMRec Innovation: A-LLMRec explicitly incorporates collaborative knowledge from a pre-trained CF-RecSys and combines it with modality information through an LLM. This allows A-LLMRec to leverage CF's warm scenario strength while also addressing cold scenarios effectively, where CF alone would fail.

• Versus Modality-aware RecSys (e.g., MoRec, CTRL, RECFORMER):

  • Modality-aware Core: Uses modality encoders to get rich content features, often integrating them into CF models or replacing ID embeddings.
  • Modality-aware Strengths: Good in cold scenarios by using content.
  • Modality-aware Weaknesses: Can sometimes underperform traditional CF in warm scenarios because the emphasis on content might dilute collaborative knowledge or create over-smoothed representations. Some models like RECFORMER might struggle if the "language representation" of items doesn't fully capture collaborative patterns.
  • A-LLMRec Innovation: A-LLMRec directly leverages pre-trained collaborative knowledge from a CF-RecSys rather than trying to infer it from modality information or combine it in a less explicit way. The alignment ensures that the collaborative knowledge is preserved and injected into the LLM's understanding.

• Versus LLM-based RecSys (e.g., LLM-Only, TALLRec):

  • LLM-Only Core: Uses LLMs directly with prompts for recommendations, relying on LLM's in-context learning and knowledge.
  • LLM-Only Weaknesses: Severely underperforms because LLMs are not inherently designed for recommendation tasks and lack direct access to collaborative knowledge.
  • TALLRec Core: Fine-tunes LLMs (e.g., using LoRA) on recommendation data converted into instruction text.
  • TALLRec Strengths: Improves LLM performance in cold and cross-domain scenarios compared to LLM-Only.
  • TALLRec Weaknesses: Still underperforms traditional CF in warm scenarios due to a lack of explicit collaborative knowledge capture. Requires extensive fine-tuning of the LLM which is computationally expensive and slow.
  • A-LLMRec Innovation:
    1. Explicit Collaborative Knowledge: Unlike TALLRec which implicitly tries to learn collaborative knowledge from instruction text, A-LLMRec explicitly integrates high-quality collaborative knowledge (user and item embeddings) from a pre-trained CF-RecSys into the LLM's token space.
    2. Efficiency: A-LLMRec does not fine-tune the LLM. Only a small alignment network is trained. This makes it significantly faster in both training and inference compared to TALLRec.
    3. Model-Agnosticism: A-LLMRec can integrate any pre-trained CF-RecSys, making it highly flexible and adaptable to different domains or existing infrastructure, a feature not highlighted in TALLRec.
    4. All-round Performance: By combining the explicit collaborative knowledge with LLM's content understanding, A-LLMRec achieves superior performance in both cold and warm scenarios, overcoming the trade-off seen in prior LLM-based and modality-aware systems.

4. Methodology

The paper proposes A-LLMRec, an All-round LLM-based Recommender system, designed to excel in both cold and warm scenarios. Its core idea is to enable a Large Language Model (LLM) to directly leverage the collaborative knowledge embedded within a pre-trained state-of-the-art Collaborative Filtering Recommender System (CF-RecSys). This is achieved by creating an alignment network that maps the CF-RecSys's user and item embeddings into the LLM's token space, allowing the LLM to understand and utilize this collaborative knowledge for recommendation tasks. The methodology is structured into two main stages.

4.1. Principles

The fundamental principle behind A-LLMRec is to combine the strengths of collaborative filtering and large language models without inheriting their individual weaknesses.

1. Exploiting Collaborative Knowledge: Traditional CF-RecSys are highly effective at capturing intricate user-item preference patterns from dense interaction data (i.e., warm scenarios). The idea is to preserve and directly transfer this high-quality collaborative knowledge (represented as user and item embeddings) into the LLM's operational context.
2. Leveraging LLM's Emergent Abilities: LLMs possess powerful natural language understanding and reasoning capabilities, which are crucial for handling modality information (like text descriptions) and generalizing to cold scenarios or cross-domain tasks. The goal is to integrate LLMs without the prohibitive cost of fine-tuning their vast parameters.
3. Bridging Modality Gaps: The challenge lies in translating the numerical, ID-based collaborative knowledge from CF-RecSys into a format that an LLM (which primarily operates on text tokens) can understand and utilize. An alignment network serves as this bridge, projecting embeddings into the LLM's token space.
4. Efficiency and Model-Agnosticism: By keeping both the CF-RecSys and the LLM frozen and only training a small alignment network, the system remains efficient and flexible, allowing any CF-RecSys to be swapped in.

4.2. Core Methodology In-depth (Layer by Layer)

4.2.1. Problem Formulation

The paper focuses on the sequential recommendation task, where the objective is to predict the next item a user will interact with based on their historical interaction sequence.

• Notations:

  • $\mathcal{D}$: The historical user-item interaction dataset.
  • $(\mathcal{U}, \mathcal{I}, \mathcal{T}, S) \in \mathcal{D}$: Denotes the set of users, items, item titles/descriptions, and item sequences, respectively.
  • $S^u = (i_1^u, i_2^u, \cdot \cdot \cdot, i_k^u, \cdot \cdot \cdot i_{|S^u|}^u) \in S$: Represents the sequence of interactions for a user $u \in \mathcal{U}$. Here, $i_k^u$ is the $k$-th item interacted with by user $u$.
  • $(t^i, d^i) \in \mathcal{T}$: The title and description text associated with each item $i \in \mathcal{I}$.
  • $\mathbf{E} \in \mathbb{R}^{|\mathcal{I}| \times d}$: The embedding matrix of items, where $d$ is the embedding dimension.
  • $\mathbf{E}_{1:k}^u = (\mathbf{E}_{i_1^u}, \mathbf{E}_{i_2^u}, \cdot \cdot \cdot, \mathbf{E}_{i_k^u}) \in \mathbb{R}^{k \times d}$: The embedding matrix for the sequence $S_{1:k}^u$, where $\mathbf{E}_{i_j^u}$ is the row from $\mathbf{E}$ corresponding to item $i_j^u$.

• Task: Sequential Recommendation: The goal is to predict $i_{k+1}^u$ (the next item) given the historical sequence $S_{1:k}^u$. A CF-RecSys (e.g., SASRec) is trained to maximize the probability of observing the next item in the sequence: $$\operatorname*{max}_{\Theta} \prod_{u \in \mathcal{U}} \prod_{k=1}^{|S^u|-1} p(i_{k+1}^u | S_{1:k}^u ; \Theta)$$

  • $\operatorname*{max}_{\Theta}$: Optimize the parameters $\Theta$ to maximize the product of probabilities.
  • $\mathcal{U}$: Set of all users.
  • $S^u$: Interaction sequence for user $u$.
  • $i_{k+1}^u$: The $(k+1)$-th item in user $u$'s sequence (the target item to predict).
  • $S_{1:k}^u$: The historical interaction sequence for user $u$ up to the $k$-th item.
  • $p(i_{k+1}^u | S_{1:k}^u ; \Theta)$: The probability of item $i_{k+1}^u$ being the next item, conditioned on the sequence $S_{1:k}^u$ and the model parameters $\Theta$.
  • $\Theta$: The set of learnable parameters of the CF-RecSys. By optimizing this objective, the model learns user representations and item embeddings that can predict future interactions.

4.2.2. A-LLMRec Architecture Overview

A-LLMRec operates in two pre-training stages:

1. Stage-1: Alignment between Collaborative and Textual Knowledge: This stage focuses on aligning item embeddings from a frozen CF-RecSys with their associated text embeddings using autoencoders. This creates "joint collaborative-text embeddings" that capture both collaborative and modality knowledge.

2. Stage-2: Alignment between Joint Collaborative-Text Embedding and LLM: This stage projects the user representations and joint collaborative-text embeddings from Stage-1 into the token space of a frozen LLM. A specially designed prompt then enables the LLM to perform recommendations by leveraging this integrated knowledge.

   The overall architecture for Stage-2 (which builds upon Stage-1) is depicted in Figure 2 of the original paper, showing how CF-RecSys and SBERT interact with A-LLMRec components to feed into the LLM.

   该图像是示意图，展示了A-LLMRec推荐系统的框架及其工作流程。图中包括三个主要部分：CF-RecSys、A-LLMRec以及大型语言模型（LLM），并显示了用户-项目交互历史和输入提示如何流入不同组件以生成推荐。每个部分的功能通过箭头和标签进行了标注，清晰地展示了系统在冷场景和暖场景中的应用。

4.2.3. Stage-1: Alignment between Collaborative and Textual Knowledge

The objective of Stage-1 is to create a unified representation that combines collaborative knowledge (from CF-RecSys) with textual knowledge (from item descriptions). This is achieved by training an alignment network that consists of encoders and decoders, designed to map the CF-RecSys item embeddings and SBERT text embeddings into a common latent space.

• Components:

  • Frozen CF-RecSys: A pre-trained CF-RecSys (e.g., SASRec) provides fixed item embeddings $\mathbf{E}_i \in \mathbb{R}^d$ for each item $i$.
  • Sentence-BERT (SBERT): A pre-trained SBERT model generates text embeddings $\mathbf{Q}_i \in \mathbb{R}^{768}$ for item texts. The SBERT model is fine-tuned during the training of Stage-1 to better adapt to the recommendation task's textual context. The input to SBERT is a structured string: \mathbf{Q}_i = SBERT(^{\propto}Title: t^i, Description: d^{i\cdot p}).
  • Item Encoder ($f_I^{enc}$): A 1-layer Multi-Layer Perceptron (MLP) that takes an item embedding $\mathbf{E}_i \in \mathbb{R}^d$ from the CF-RecSys and transforms it into a latent item embedding $\mathbf{e}_i \in \mathbb{R}^{d'}$. That is, $\mathbf{e}_i = f_I^{enc}(\mathbf{E}_i)$.
  • Text Encoder ($f_T^{enc}$): A 1-layer MLP that takes a text embedding $\mathbf{Q}_i \in \mathbb{R}^{768}$ from SBERT and transforms it into a latent text embedding $\mathbf{q}_i \in \mathbb{R}^{d'}$. That is, $\mathbf{q}_i = f_T^{enc}(\mathbf{Q}_i)$.
  • Item Decoder ($f_I^{dec}$): A decoder corresponding to $f_I^{enc}$, used for reconstruction.
  • Text Decoder ($f_T^{dec}$): A decoder corresponding to $f_T^{enc}$, used for reconstruction.

• Latent Space Matching (Matching Loss): This loss encourages the latent representations generated by the item encoder and text encoder to be similar for the same item, thereby aligning the collaborative knowledge and textual knowledge. $$\begin{array}{rcl} \mathcal{L}_{\mathrm{matching}} & = & \mathbb{E}_{S^u \in S} \left[ \mathbb{E}_{i \in S^u} \left[ MSE(\mathbf{e}_i, \mathbf{q}_i) \right] \right] \\ & = & \mathbb{E}_{S^u \in S} \left[ \mathbb{E}_{i \in S^u} \left[ MSE(f_I^{enc}(\mathbf{E}_i), f_T^{enc}(\mathbf{Q}_i)) \right] \right] \end{array}$$

  • $\mathcal{L}_{\mathrm{matching}}$: The matching loss.
  • $\mathbb{E}_{S^u \in S}$: Expectation over all user sequences $S^u$ in the dataset $S$.
  • $\mathbb{E}_{i \in S^u}$: Expectation over all items $i$ within a user sequence $S^u$.
  • $MSE(\cdot, \cdot)$: The Mean Squared Error loss, which quantifies the difference between the two input vectors.
  • $\mathbf{e}_i$: The latent item embedding for item $i$, output of $f_I^{enc}(\mathbf{E}_i)$.
  • $\mathbf{q}_i$: The latent text embedding for item $i$, output of $f_T^{enc}(\mathbf{Q}_i)$.
  • $f_I^{enc}(\cdot)$: The item encoder function.
  • $f_T^{enc}(\cdot)$: The text encoder function.
  • $\mathbf{E}_i$: The item embedding from the frozen CF-RecSys.
  • $\mathbf{Q}_i$: The text embedding from SBERT.

• Avoiding Over-smoothed Representation (Reconstruction Losses): To prevent the encoders from collapsing to trivial solutions (e.g., producing identical or zero outputs to minimize matching loss), reconstruction losses are introduced. These ensure that the original information of the item embeddings and text embeddings can be reconstructed from their latent representations.

  • Item Reconstruction Loss: $$\mathcal{L}_{\mathrm{item-recon}} = \mathbb{E}_{S^u \in S} \left[ \mathbb{E}_{i \in S^u} \left[ MSE(\mathbf{E}_i, f_I^{dec}(f_I^{enc}(\mathbf{E}_i))) \right] \right]$$
    • $\mathcal{L}_{\mathrm{item-recon}}$: The item reconstruction loss.
    • $f_I^{dec}(\cdot)$: The item decoder function, which attempts to reconstruct $\mathbf{E}_i$ from $\mathbf{e}_i = f_I^{enc}(\mathbf{E}_i)$.
  • Text Reconstruction Loss: $$\mathcal{L}_{\mathrm{text-recon}} = \mathbb{E}_{S^u \in S} \left[ \mathbb{E}_{i \in S^u} \left[ MSE(\mathbf{Q}_i, f_T^{dec}(f_T^{enc}(\mathbf{Q}_i))) \right] \right]$$
    • $\mathcal{L}_{\mathrm{text-recon}}$: The text reconstruction loss.
    • $f_T^{dec}(\cdot)$: The text decoder function, which attempts to reconstruct $\mathbf{Q}_i$ from $\mathbf{q}_i = f_T^{enc}(\mathbf{Q}_i)$.

• Recommendation Loss: This loss explicitly incorporates collaborative knowledge and guides the model towards the recommendation task. It is based on a binary cross-entropy like formulation, discriminating between a positive next item and a negative sampled item. $$\begin{array}{r} \mathcal{L}_{\mathrm{rec}} = - \displaystyle \sum_{S^u \in S} \left[ \log(\sigma(s(\mathbf{x}_{|S^u|-1}^u, f_I^{dec}(f_I^{enc}(\mathbf{E}_{i_{|S^u|}^u}))))) \right. \\ \left. + \log(1 - \sigma(s(\mathbf{x}_{|S^u|-1}^u, f_I^{dec}(f_I^{enc}(\mathbf{E}_{i_{|S^u|}^u, -})))))) \right] \end{array}$$

  • $\mathcal{L}_{\mathrm{rec}}$: The recommendation loss.
  • $\mathbf{x}_{|S^u|-1}^u = \mathrm{CF-RecSys}(S_{1:|S^u|-1}^u) \in \mathbb{R}^d$: The user representation extracted from the CF-RecSys after user $u$ has interacted with the last item in the sequence $S_{1:|S^u|-1}^u$.
  • $\mathbf{E}_{i_{|S^u|}^u} \in \mathbb{R}^d$: The original item embedding from CF-RecSys for the actual next item (positive sample).
  • $\mathbf{E}_{i_{|S^u|}^u, -} \in \mathbb{R}^d$: The original item embedding from CF-RecSys for a randomly sampled negative item (not interacted by the user).
  • $s(\mathbf{a}, \mathbf{b})$: A dot product similarity function between two vectors $\mathbf{a}$ and $\mathbf{b}$.
  • $\sigma(\cdot)$: The sigmoid function, which converts the similarity score into a probability between 0 and 1. The loss aims to maximize the probability score for the true next item and minimize it for negative items. Note: For efficiency, this loss is computed only for the last item in each user sequence.

• Final Loss of Stage-1: The total objective for Stage-1 is a weighted sum of the matching loss, reconstruction losses, and recommendation loss: $$\mathcal{L}_{\mathrm{stage-1}} = \mathcal{L}_{\mathrm{matching}} + \alpha \mathcal{L}_{\mathrm{item-recon}} + \beta \mathcal{L}_{\mathrm{text-recon}} + \mathcal{L}_{\mathrm{rec}}$$

  • $\alpha, \beta$: Hyperparameters that control the importance of the item reconstruction and text reconstruction terms, respectively.

• Joint Collaborative-Text Embedding: After Stage-1 training, the output of the item encoder, $\mathbf{e}_i = f_I^{enc}(\mathbf{E}_i)$, is considered the joint collaborative-text embedding for item $i$. This embedding carries both collaborative knowledge (from $\mathbf{E}_i$) and textual knowledge (aligned with $\mathbf{Q}_i$).

  • Handling Cold Items: For new items that were not seen during the CF-RecSys training (and thus lack $\mathbf{E}_i$), the text encoder is used: $\mathbf{q}_i = f_T^{enc}(\mathbf{Q}_i)$. Since $f_I^{enc}$ and $f_T^{enc}$ are trained to align their latent spaces, $\mathbf{q}_i$ is expected to implicitly capture some collaborative knowledge alongside textual knowledge. This is crucial for cold-start, few-shot, and cross-domain scenarios.

4.2.4. Stage-2: Alignment between Joint Collaborative-Text Embedding and LLM

Stage-2 focuses on integrating the joint collaborative-text embeddings and user representations (from the CF-RecSys via Stage-1) into the token space of a frozen LLM, and then designing a prompt to enable the LLM to perform recommendations.

• Projecting Collaborative Knowledge onto the LLM's Token Space: The numerical user representations $\mathbf{x}^u$ (from the CF-RecSys) and the joint collaborative-text embeddings $\mathbf{e}_i$ (from Stage-1) need to be converted into a format suitable for the LLM's input. This is done by projecting them into the LLM's token embedding space.

  • User Projection Network ($F_U$): A 2-layer MLP that maps the user representation $\mathbf{x}^u \in \mathbb{R}^d$ to a user embedding in the LLM token space $\mathbf{O}_u \in \mathbb{R}^{d^{\mathrm{token}}}$.
  • Item Projection Network ($F_I$): A 2-layer MLP that maps the joint collaborative-text embedding $\mathbf{e}_i \in \mathbb{R}^{d'}$ to an item embedding in the LLM token space $\mathbf{O}_i \in \mathbb{R}^{d^{\mathrm{token}}}$. The projection equations are: $$\mathbf{O}_u = F_U(\mathbf{x}^u), \quad \mathbf{O}_i = F_I(\mathbf{e}_i)$$
  • $\mathbf{O}_u \in \mathbb{R}^{d^{\mathrm{token}}}$: The projected embedding of user $u$ in the LLM token space.
  • $\mathbf{O}_i \in \mathbb{R}^{d^{\mathrm{token}}}$: The projected joint collaborative-text embedding of item $i$ in the LLM token space.
  • $d^{\mathrm{token}}$: The dimension of the LLM's token embedding space. These projected embeddings $\mathbf{O}_u$ and $\mathbf{O}_i$ can now be treated as ordinary tokens by the LLM.

• Prompt Design for Integrating Collaborative Knowledge: A novel prompt design is crucial to guide the LLM to utilize the injected collaborative knowledge.

  • The projected user representation $\mathbf{O}_u$ is placed at the beginning of the prompt. This provides the LLM with user-specific collaborative knowledge as a "soft prompt" [26], crucial for personalized recommendation.
  • The projected joint embedding $\mathbf{O}_i$ for each item is placed directly next to its title within the prompt. This links the LLM's textual understanding of the item with its collaborative features. The LLM (which remains frozen) then processes this structured prompt, and the goal is to generate the recommended item title.

  The following figure (Figure 3 from the original paper) shows an example prompt for the Amazon Movies dataset:

  该图像是一个示意图，展示了 A-LLMRec 在推荐电影中的输入输出结构。用户输入包含过去观看的电影历史和候选电影集合，LLM 输出为下一个推荐的电影标题，其中历史和候选电影包含其标题与嵌入信息。

• Learning Objective of Stage-2: The networks $F_U$ and $F_I$ are trained to maximize the likelihood of the LLM generating the correct next item title, given the prompt with injected collaborative knowledge. $$\operatorname*{max}_{\theta} \sum_{S^u \in S} \sum_{k=1}^{|y^u|} \log(P_{\theta, \Theta}(y_k^u | \mathcal{p}^u, y_{<k}^u))$$

  • $\operatorname*{max}_{\theta}$: Optimize the learnable parameters $\theta$ of $F_U$ and $F_I$.
  • $\Theta$: The frozen parameters of the LLM.
  • $\sum_{S^u \in S}$: Sum over all user sequences.
  • $\sum_{k=1}^{|y^u|}$: Sum over all tokens in the target item title.
  • $P_{\theta, \Theta}(y_k^u | \mathcal{p}^u, y_{<k}^u)$: The probability of generating the $k$-th token $y_k^u$ of the next item title, conditioned on the input prompt $\mathcal{p}^u$ and the previously generated tokens $y_{<k}^u$.
  • $\mathcal{p}^u$: The input prompt for user $u$, containing $\mathbf{O}_u$, and $\mathbf{O}_i$ for candidate items.
  • $y^u$: The true next item title for user $u$.
  • $y_k^u$: The $k$-th token of the target item title $y^u$.
  • $y_{<k}^u$: The sequence of tokens generated before $y_k^u$. For efficiency, similar to Stage-1, training in Stage-2 also focuses on optimizing for the next item of each user sequence.

5. Experimental Setup

5.1. Datasets

The experiments utilize four real-world datasets from Amazon [13, 32], all containing rich textual information (title and description). These datasets were selected to represent varying scales and characteristics, enabling a comprehensive analysis.

• Movies and TV:

  • Scale: Large scale (approx. 300K users, 60K items).
  • Statistics (after preprocessing): #Users = 297,498, #Items = 59,944, #Interactions = 3,409,147, Avg. Len = 11.46.
  • Preprocessing: Removed users and items with fewer than 5 interactions.

• Video Games:

  • Scale: Moderate scale (approx. 64K users, 33K items).
  • Statistics (after preprocessing): #Users = 64,073, #Items = 33,614, #Interactions = 598,509, Avg. Len = 8.88.
  • Preprocessing: Removed users and items with fewer than 5 interactions.

• Beauty:

  • Scale: Small and cold dataset (approx. 9K users, 6K items).
  • Statistics (after preprocessing): #Users = 9,930, #Items = 6,141, #Interactions = 63,953, Avg. Len = 6.44.
  • Preprocessing: Removed users and items with fewer than 4 interactions. User ratings above 3 were treated as positive, others as negative.

• Toys:

  • Scale: Dataset where the number of items is larger than the number of users (approx. 3K users, 6K items).

  • Statistics (after preprocessing): #Users = 30,831, #Items = 61,081, #Interactions = 282,213, Avg. Len = 9.15.

  • Preprocessing: Removed users and items with fewer than 4 interactions. User ratings above 3 were treated as positive, others as negative.

    Example of a data sample: For items, the data would include an item ID, a title (e.g., "The Matrix"), and a description (e.g., "A computer hacker learns from mysterious rebels about the true nature of his reality and his role in the war against its controllers."). For users, it would be a sequence of item IDs they have interacted with.

The diverse characteristics of these datasets (scale, user-item ratio, sparsity) make them effective for validating the method's performance across various real-world recommendation challenges, including cold-start and warm scenarios.

5.2. Evaluation Metrics

For quantitative comparison, the paper employs a widely used metric for sequential recommendation tasks: Hit Ratio at 1 (Hit@1).

5.2.1. Hit Ratio at 1 (Hit@1)

• Conceptual Definition: Hit@1 is a straightforward metric that evaluates the accuracy of a recommender system by checking if the ground truth (the actual next item a user interacted with) is present in the top 1 recommendation generated by the model. It's a binary measure: either the model "hits" the correct item at the very first position, or it doesn't. A higher Hit@1 value indicates better performance, meaning the model is very precise in predicting the exact next item.

• Mathematical Formula: The general formula for Hit Ratio at K is: $ \mathrm{Hit@K} = \frac{\sum_{u \in \mathcal{U}} \mathbb{I}(\text{ground truth item for user } u \text{ is in top K recommendations for user } u)}{|\mathcal{U}|} $ For Hit@1, K is simply 1: $ \mathrm{Hit@1} = \frac{\sum_{u \in \mathcal{U}} \mathbb{I}(\text{ground truth item for user } u \text{ is in top 1 recommendation for user } u)}{|\mathcal{U}|} $

• Symbol Explanation:

  • $\mathrm{Hit@1}$: The Hit Ratio at 1 score.

  • $\mathcal{U}$: The set of all users in the test set.

  • $|\mathcal{U}|$: The total number of users in the test set.

  • $\mathbb{I}(\cdot)$: The indicator function. It returns 1 if the condition inside the parentheses is true, and 0 otherwise.

  • ground truth item for user u: The actual item that user $u$ interacted with next in the test set.

  • top 1 recommendation for user u: The single item that the recommender system predicts as most likely for user $u$.

    Evaluation Setting Details: User sequences are split into training, validation, and test sets. The most recently interacted item $i_{|S^u|}^u$ serves as the test item, and $i_{|S^u|-1}^u$ as the validation item. For evaluation, 19 randomly selected non-interacted items are added to the test set for each user, creating a test set with 1 positive item and 19 negative items. This simulates a ranking task, and Hit@1 specifically checks if the positive item is ranked first among these 20 items.

5.3. Baselines

The performance of A-LLMRec is compared against a comprehensive set of baselines, categorized into three types:

1. Collaborative Filtering (CF) Recommender Systems: These models primarily rely on user-item interaction data.

   • NCF [15]: Neural Collaborative Filtering. Combines MLPs to capture collaborative information. It's a two-tower model with separate components for user and item embeddings.
   • NextItNet [50]: Employs a temporal convolutional network with 1D-dilated convolutional layers and residual connections to capture long-term dependencies in interaction sequences.
   • GRU4Rec [17]: Uses Recurrent Neural Networks (RNNs) to model user behavior sequences, particularly for session-based recommendations.
   • SASRec [20]: Self-Attentive Sequential Recommendation. A state-of-the-art sequential recommender that uses a self-attention encoding method to model user preferences from interaction sequences. It is used as the backbone CF-RecSys for A-LLMRec and some modality-aware models.

2. Modality-aware Recommender Systems: These models leverage modality information (e.g., text) in addition to interaction data.

   • MoRec [51]: Utilizes a pre-trained Sentence-BERT (SBERT) to generate initial embeddings for items from their text information, which are then used within collaborative filtering models. SASRec is used as its backbone model.
   • CTRL [25]: Connect Tabular and Language Model. Involves a two-stage learning process: contrastive learning on textual information for initialization, followed by fine-tuning on recommendation tasks. SASRec is used as its backbone model.
   • RECFORMER [24]: Models user preferences and item features as language representations using the Transformer architecture. It formulates sequential recommendation as predicting the next item sentence, by converting item attributes into a sentence format (using Longformer as backbone).

3. LLM-based Recommender Systems: These models directly or indirectly use Large Language Models.

   • LLM-Only: Uses an open-source LLM model, OPT-6.7B [52], with prompts for recommendation tasks. It relies on the LLM's in-context learning without specific fine-tuning for recommendation. The prompt (Figure 6) is similar to TALLRec but lacks injected embeddings. The following figure (Figure 6 from the original paper) shows an example prompt designed for the Amazon Movies dataset used by LLM-based models, i.e., TALLRec and LLM-Only models.

     该图像是一个示意图，展示了如何使用LLM为用户推荐电影。图中包括用户观看历史和候选电影标题的输入形式，以及LLM生成推荐项的输出示例。

   • TALLRec [2]: Transformer-based All-round Language Model for Recommendation. A main baseline that fine-tunes LLMs (using LoRA [18]) based on prompts consisting solely of text. It converts the recommendation task into an instruction, where the model determines if a user will prefer a target item given their history. Uses OPT-6.7B as backbone.

   • MLP-LLM: An additionally designed LLM-based recommendation model for ablation analysis. It replaces A-LLMRec's two-stage alignment module with simple MLP layers to directly connect user and item embeddings from a frozen CF-RecSys to the LLM. It uses the same prompt structure as A-LLMRec (Figure 3).

5.4. Implementation Details

• Backbone Models:

  • LLM: OPT-6.7B [52] is used as the backbone LLM for A-LLMRec, LLM-Only, TALLRec, and MLP-LLM.
  • CF-RecSys: SASRec [20] is adopted as the backbone CF-RecSys for A-LLMRec, MoRec, and CTRL.
  • RECFORMER: Employs Longformer [3] as its backbone network.

• Embedding Dimension: Fixed to 50 for all CF-RecSys and model embeddings across all methods and datasets.

• Training Details:

  • Batch Size: 128 for collaborative filtering-based and modality-aware models.
  • Batch Size for LLM-based: 32 for Stage-1 of A-LLMRec; 4 for MLP-LLM, TALLRec, and Stage-2 of A-LLMRec.
  • Epochs: Stage-1 of A-LLMRec trained for 10 epochs. Stage-2 of A-LLMRec trained for 5 epochs. TALLRec trained for a maximum of 5 epochs.
  • Optimizer: Adam optimizer used for all models and datasets.

• Hyperparameter Tuning: Learning rates ($\eta_1, \eta_2$) tuned in $\{0.01, 0.001, 0.0005, 0.0001\}$. Coefficients $\alpha, \beta$ for Stage-1 loss tuned in $\{0.1, 0.2, 0.5, 0.75, 1.0\}$. The best-performing hyperparameters for each dataset are reported in Table 3. The following are the results from Table 3 of the original paper:

  	Learning ratestage 1	Learning ratestage 2	embedding dim(CF-RecSys) d	embedding dim(f enc, enc) d"	alpha	beta
  Movies and TV	0.0001	0.0001	50	128	0.5	0.5
  Video Games	0.0001	0.0001	50	128	0.5	0.5
  Beauty	0.0001	0.0001	50	128	0.5	0.2
  0.5	0.2

• Hardware: Four NVIDIA GeForce A6000 48GB GPUs for LLM-based models on the Movies and TV dataset, and one NVIDIA GeForce A6000 48GB for other datasets and models.

6. Results & Analysis

The experimental results demonstrate the superiority of A-LLMRec across a wide range of scenarios, validating its "all-round" capability.

6.1. Core Results Analysis

6.1.1. Overall Performance

The following are the results from Table 1 of the original paper:

Analysis:

• A-LLMRec consistently achieves the best Hit@1 scores across all four datasets, demonstrating its superior "all-round" performance.
• Comparing A-LLMRec with other LLM-based recommenders (LLM-Only, TALLRec, MLP-LLM), A-LLMRec significantly outperforms them. This highlights the importance of its approach to explicitly integrate collaborative knowledge.
• MLP-LLM, which uses a simpler MLP for alignment instead of A-LLMRec's two-stage autoencoder, performs worse than A-LLMRec but generally better than LLM-Only and TALLRec (except for Beauty where TALLRec is slightly better). This suggests that MLP-LLM attempts to bridge the gap but A-LLMRec's more sophisticated alignment is more effective.
• LLM-Only performs the worst by a large margin, underscoring that LLMs alone, without specific adaptation or collaborative knowledge, are not effective recommenders.
• TALLRec, despite fine-tuning the LLM, often underperforms even SASRec (e.g., on Movies and TV, Video Games, Toys). This confirms the paper's hypothesis that text information alone, even with fine-tuning, is insufficient to capture crucial collaborative knowledge for general recommendation.
• Modality-aware models (MoRec, CTRL, RECFORMER) generally underperform SASRec. This supports the argument that focusing heavily on modality knowledge can sometimes hinder the learning of collaborative knowledge, leading to performance degradation in general scenarios.

6.1.2. Cold/Warm Item Scenarios

The following are the results from Table 4 of the original paper:

Analysis:

• A-LLMRec demonstrates superior performance in both cold and warm item scenarios across all datasets. This is a critical validation of its "all-round" claim.
• SASRec performs well in warm scenarios but poorly in cold scenarios, as expected, due to its reliance on interaction history.
• TALLRec outperforms SASRec in cold scenarios but struggles in warm scenarios (e.g., Movies and TV), confirming its cold-start strength but warm scenario weakness.
• A-LLMRec (SBERT), a variant where text encoder output $\mathbf{q}_i$ is used instead of item encoder output $\mathbf{e}_i$ for inference, performs slightly better than standard A-LLMRec in cold item scenarios (e.g., Movies and TV, Video Games). This supports the idea from Section 4.1.4 that for cold items (which lack interaction history for $\mathbf{E}_i$), relying more directly on the text encoder's output (which inherently captures textual knowledge) is beneficial.
• Conversely, standard A-LLMRec generally outperforms A-LLMRec (SBERT) in warm item scenarios, implying that when collaborative knowledge is abundant, the item encoder's output $\mathbf{e}_i$ (which is more directly aligned with the original CF-RecSys embeddings) is more robust.

6.1.3. Cold User Scenarios

The following are the results from Table 5 of the original paper:

Analysis:

• A-LLMRec consistently outperforms other models in the cold user scenario, especially on larger datasets like Movies and TV. This indicates its ability to handle new users with limited interaction history by effectively leveraging both collaborative and textual knowledge.
• SASRec performs poorly, particularly on Movies and TV, due to the inherent difficulty of building collaborative knowledge for cold users.
• LLM-based models (MLP-LLM, TALLRec) generally perform better than SASRec in this scenario, as text information from items can compensate for the lack of user interaction data.

6.1.4. Few-shot Training Scenario

The following are the results from Table 6 of the original paper:

Analysis:

• A-LLMRec and especially A-LLMRec (SBERT) significantly outperform all other baselines in few-shot training scenarios (where $K$ is the number of users in the training set, very limited). This demonstrates A-LLMRec's robustness when training data is extremely scarce.
• A-LLMRec (SBERT) consistently performs best. This reinforces the finding from cold item scenarios that when items lack sufficient interaction data, relying on the text encoder (which directly uses modality information) is more effective for deriving joint collaborative-text embeddings.
• LLM-based models (TALLRec) generally outperform CF-RecSys (SASRec) under few-shot conditions. This is because LLMs can leverage textual understanding from item descriptions, which is vital when collaborative knowledge from interactions is insufficient for CF models.

6.1.5. Cross-domain Scenario

The following are the results from Table 7 of the original paper:

Analysis:

• A-LLMRec (SBERT) achieves the best performance in the cross-domain scenario (trained on Movies and TV, evaluated on Video Games), significantly outperforming all other models. This indicates strong generalization ability.
• Again, the superior performance of A-LLMRec (SBERT) underscores the importance of the text encoder's output $\mathbf{q}_i$ when collaborative information is completely lacking or cannot be directly transferred across domains. Modality information extracted from text becomes the primary signal.
• SASRec performs very poorly, highlighting its inability to generalize to new domains without overlapping interaction histories.
• Modality-aware and LLM-based models (MoRec, RECFORMER, TALLRec) perform better than SASRec, confirming that textual knowledge is crucial for cross-domain recommendation.

6.2. Ablation Studies

6.2.1. Effect of Components in Stage-1

The following are the results from Table 8 of the original paper:

Analysis:

• Removing $\mathcal{L}_{\mathrm{matching}}$: A significant performance drop is observed across all datasets (e.g., from 0.6237 to 0.5838 on Movies and TV). This confirms the critical role of the matching loss in aligning item embeddings and text embeddings. This alignment is essential for the LLM to understand the textual information in the context of collaborative knowledge.
• Removing $\mathcal{L}_{\mathrm{item-recon}}$ and $\mathcal{L}_{\mathrm{text-recon}}$: Performance decreases (e.g., from 0.6237 to 0.5482 on Movies and TV). This validates the need for reconstruction losses to prevent over-smoothed representations and preserve the rich information content of the original embeddings.
• Removing $\mathcal{L}_{\mathrm{rec}}$: This leads to a substantial performance drop, especially on the Toys dataset (from 0.3336 to 0.1541). This highlights the importance of the recommendation loss in explicitly incorporating collaborative knowledge and guiding the model towards the recommendation task. Without it, the model loses crucial task-specific signal.
• Freezing SBERT: Freezing SBERT (i.e., not fine-tuning it) results in lower performance across all datasets. This indicates that fine-tuning SBERT allows its text embeddings to better adapt to the specific nuances and semantics relevant for the recommendation task in each dataset, thus providing higher quality textual input for the alignment network.

6.2.2. Effect of the Alignment method in Stage-2

The following are the results from Table 9 of the original paper:

Analysis:

• A-LLMRec w/o user representation (Row 2): Removing the projected user representation $\mathbf{O}_u$ from the LLM prompt leads to a performance decrease across all datasets. This shows that providing the LLM with explicit information about the user, derived from CF-RecSys, is valuable for personalization.
• A-LLMRec w/o joint embedding (Row 3): Excluding the projected joint collaborative-text embedding $\mathbf{O}_i$ from the prompt results in a much more substantial performance drop (e.g., from 0.6237 to 0.1224 on Movies and TV). This highlights the critical role of these joint embeddings, as they carry both the collaborative and textual knowledge of items, which is essential for the LLM to make informed recommendations.
• A-LLMRec with random joint embedding (Row 4): Replacing the learned joint embeddings with randomly initialized ones causes a severe performance degradation, especially on Toys (from 0.3336 to 0.0776). This unequivocally demonstrates that the quality and informational content of the joint collaborative-text embeddings are crucial, and simply having a placeholder embedding is not enough; it must contain meaningful collaborative knowledge.

6.3. Model Analysis

6.3.1. Train/Inference Speed

The following are the results from Table 10 of the original paper:

Analysis:

• Efficiency Advantage: A-LLMRec is significantly more efficient than TALLRec. It trains approximately 2.5 times faster (232.5 min vs. 588.58 min) and infers 1.7 times faster per batch (1.98 sec/batch vs. 3.36 sec/batch). This is a direct benefit of A-LLMRec not fine-tuning the LLM and only training a smaller alignment network. This efficiency is crucial for real-world applicability and handling large-scale datasets.
• A-LLMRec_all: This variant trains A-LLMRec using all items in each user sequence for optimization, instead of just the last one.
  • Performance: A-LLMRec_all shows a marginal improvement in Hit@1 (0.6002 vs. 0.5809), indicating that considering more sequential data can slightly enhance recommendations.
  • Efficiency Trade-off: However, its training time increases significantly (643.33 min), becoming even longer than TALLRec. The inference time remains the same because the final model architecture is identical. This highlights a practical trade-off between marginal performance gains and significantly increased training cost when processing full sequences. The comparable performance of vanilla A-LLMRec with A-LLMRec_all also suggests good generalization even with less training data.

6.3.2. A-LLMRec is Model-Agnostic

The following are the results from Table 11 of the original paper:

Analysis:

• Model-Agnostic Property Confirmed: The results clearly show that A-LLMRec can successfully integrate with various CF-RecSys backbones (SASRec, NextItNet, GRU4Rec, NCF) and consistently improve their performance. For every CF-RecSys listed, the A-LLMRec variant (e.g., A-LLMRec (NextItNet)) achieves a higher Hit@1 than its standalone CF-RecSys counterpart. This is a crucial practical advantage, demonstrating flexibility and future-proofing.
• Impact of Backbone Quality: A-LLMRec (SASRec) performs the best among all variants, which is expected since SASRec itself is the strongest standalone CF-RecSys among the baselines. This suggests that the quality of the collaborative knowledge from the chosen backbone CF-RecSys directly influences the overall performance of A-LLMRec.
• Reducing Performance Gaps: Interestingly, while SASRec and NCF have a large performance difference as standalone models, integrating them into A-LLMRec significantly narrows this gap. For instance, on Beauty, NCF (0.2957) is far below SASRec (0.5298), but A-LLMRec (NCF) (0.5431) gets much closer to A-LLMRec (SASRec) (0.5809). This indicates that A-LLMRec's modality information and LLM capabilities can largely compensate for the weaknesses of a less performant CF-RecSys backbone, making it a powerful enhancer.

6.3.3. Beyond Recommendation: Language Generation Task (Favorite Genre Prediction)

The following figure (Figure 4 from the original paper) shows a comparison between A-LLMRec and LLM-Only on the favorite genre prediction task.

该图像是一个对比图，展示了 A-LLMRec 与 LLM-Only 在电影和电视数据集上的用户推荐表现。左侧展示了 A-LLMRec 如何利用用户观看历史生成个性化推荐，而右侧则是 LLM-Only 的较为简化的推荐过程。

The following figure (Figure 5 from the original paper) shows A-LLMRec, LLM-Only, and TALLRec on the favorite genre prediction task.

该图像是一个示意图，展示了A-LLMRec、LLM-Only和TALLRec在电影和电视剧数据集上的喜好类型预测任务的用户表现。图中详细列出用户观看的影片及其推荐的类型，突出不同推荐系统的用户建模能力。 Analysis:

• A-LLMRec's Understanding of Collaborative Knowledge: A-LLMRec successfully generates natural language outputs for favorite genre prediction, providing proper and relevant answers based on the user's movie watching history. This demonstrates that A-LLMRec's alignment mechanism effectively enables the LLM to understand and utilize the collaborative knowledge (user preferences and item characteristics) for tasks beyond just numerical ranking.
• LLM-Only's Failure: LLM-Only fails to generate meaningful responses for this task. This further emphasizes that LLMs require structured injection of collaborative knowledge to perform well in recommendation-related language tasks. Simply providing raw textual data or basic prompts is insufficient.
• TALLRec's Limitation: The paper notes that TALLRec was unable to produce valid outputs for this natural language generation task. This is attributed to TALLRec's fine-tuning process, which primarily adapts the LLM for a specific instruction-tuning format for binary recommendation decisions. This specialized fine-tuning might inadvertently restrict its broader natural language generation capabilities or require a very specific prompt format to elicit responses. This highlights a potential trade-off: while fine-tuning can boost recommendation accuracy for specific instruction formats, it might reduce the LLM's general emergent abilities for free-form natural language generation based on collaborative knowledge. A-LLMRec, by keeping the LLM frozen, retains its full language generation capabilities while still benefiting from injected collaborative knowledge.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper introduces A-LLMRec, an innovative All-round LLM-based Recommender system that successfully bridges the gap between collaborative filtering and large language models. The core contribution lies in its ability to enable a frozen LLM to directly leverage the collaborative knowledge embedded in a pre-trained CF-RecSys through a novel, lightweight alignment network. This approach yields two crucial advantages: model-agnosticism, allowing integration with any existing CF-RecSys, and efficiency, as it avoids the computationally expensive fine-tuning of the LLM typically required by other LLM-based recommenders.

Extensive experiments on real-world datasets demonstrate A-LLMRec's superior performance across a comprehensive set of challenging scenarios, including cold/warm item, cold user, few-shot training, and cross-domain recommendation. Critically, it outperforms traditional CF-RecSys in warm scenarios and other LLM-based recommenders in cold scenarios, achieving true "all-round" efficacy. Beyond traditional recommendation metrics, A-LLMRec also showcases its potential in natural language generation tasks, such as favorite genre prediction, indicating that the LLM can interpret and utilize the injected collaborative knowledge for more complex, qualitative outputs.

7.2. Limitations & Future Work

The authors suggest that for future work, they plan to further enhance the ability of the LLM in A-LLMRec based on advanced prompt engineering such as chain-of-thought prompting [46]. This implies that while the current prompt design effectively integrates collaborative knowledge, more sophisticated prompting techniques could potentially unlock even deeper reasoning and understanding from the LLM, leading to further performance improvements or more nuanced natural language outputs.

7.3. Personal Insights & Critique

This paper presents a highly practical and impactful solution for LLM-based recommender systems. The core idea of not fine-tuning the LLM and instead focusing on an efficient alignment mechanism is a significant leap forward, directly addressing the computational and practical challenges associated with deploying LLMs in real-world recommendation scenarios.

Inspirations and Applications:

• Efficiency as a Key Driver: The emphasis on efficiency is a strong takeaway. For many industrial applications, the sheer scale and update frequency of recommender systems make full LLM fine-tuning prohibitive. A-LLMRec offers a viable path to integrate LLM benefits without incurring excessive costs.

• Model-Agnosticism for Practical Adoption: The model-agnostic nature is another powerful feature. Companies often have highly optimized, proprietary CF-RecSys in production. A-LLMRec allows them to augment these existing systems with LLM capabilities incrementally, rather than undertaking a complete overhaul.

• Bridging ID-based and Content-based Recommendations: The paper effectively demonstrates a robust way to combine the strengths of ID-based collaborative filtering (for warm scenarios and deep collaborative patterns) with content-based recommendations (for cold scenarios and modality information) via the LLM's textual understanding. This hybrid approach is often sought after but rarely achieved with such elegance and efficiency.

• Beyond Ranking: The favorite genre prediction task is a compelling demonstration. It highlights the potential for LLMs in recommender systems to go beyond simply ranking items. They could provide explanations for recommendations, generate personalized summaries, or even engage in conversational recommendation, transforming the user experience.

  Potential Issues, Unverified Assumptions, and Areas for Improvement:

• Quality of Pre-trained CF-RecSys: A-LLMRec's performance is inherently tied to the quality of the pre-trained CF-RecSys. While it can somewhat mitigate the weaknesses of a weaker CF-RecSys (as shown with NCF), a high-quality backbone CF-RecSys remains crucial for optimal performance. The initial collaborative knowledge must be robust.

• Dependence on Item Text Modality: While demonstrating "all-round" performance, the SBERT component and text encoder are central to handling cold scenarios and cross-domain tasks. The performance might be sensitive to the availability and quality of item textual descriptions. If items lack rich text or have noisy descriptions, the modality-aware components might struggle.

• Scalability of SBERT Fine-tuning: While the LLM is frozen, SBERT is fine-tuned in Stage-1. For extremely large item catalogs or dynamically changing item descriptions, the cost of fine-tuning SBERT might still be a consideration, though significantly less than an LLM.

• Generalization to Other Modalities: The current work primarily focuses on textual modality information. Extending A-LLMRec to incorporate other modalities like images or audio (e.g., using Vision-Transformer with an image encoder) would be a natural next step and would further enhance its all-round capabilities for multimodal recommendation.

• Interpretability of Alignment: While the ablation studies show the necessity of the alignment network components, the exact mechanisms by which the LLM "understands" and synthesizes the injected numerical embeddings for specific recommendation decisions could be further explored for interpretability.

• Cold-User Sequence Length: The paper samples cold users as those with exactly three interactions. While a valid setup, understanding how A-LLMRec performs with even sparser user histories (e.g., one or two interactions) or completely new users (zero interactions, requiring only item modality information) could provide further insights.

  Overall, A-LLMRec presents a highly promising and practical paradigm for LLM-based recommendation, effectively leveraging existing strengths while efficiently integrating new capabilities.