1. Bibliographic Information

1.1. Title

The central topic of the paper is LLM4WM: Adapting LLM for Wireless Multi-Tasking. It focuses on developing a framework that leverages Large Language Models (LLMs) for handling multiple channel-associated tasks in wireless communication systems.

1.2. Authors

The authors of this paper are:

• Xuanyu Liu (State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, China)

• Shijian Gao (Internet of Things Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China)

• Boxun Liu (State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, China)

• Xiang Cheng (State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, China)

• Liuqing Yang (Internet of Things Thrust and Intelligent Transportation Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China, and Department of Electronic and Computer Engineering and the Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Hong Kong, SAR, China)

  The authors have backgrounds in electrical engineering, communications, and computer science, with affiliations to prominent universities and research institutions known for work in wireless communication, signal processing, and AI.

1.3. Journal/Conference

This paper was published as a preprint on arXiv. While not a peer-reviewed journal or conference yet, arXiv is a highly reputable platform for disseminating cutting-edge research in physics, mathematics, computer science, and related fields. Papers on arXiv often precede formal publication and allow for early sharing and feedback within the academic community.

1.4. Publication Year

The paper was published at (UTC): 2025-01-22T16:12:38.000Z.

1.5. Abstract

The abstract introduces the concept of channel-associated tasks in wireless communication, highlighting their potential for joint learning to enhance system design. To leverage this, the paper proposes LLM4WM, a Large Language Model (LLM) multi-task fine-tuning framework specifically designed for these tasks. The framework employs a Mixture of Experts with Low-Rank Adaptation (MoE-LoRA) approach for multi-task fine-tuning, which transfers the pre-trained LLM's general knowledge to the wireless domain. It also incorporates preprocessing modules, adapter modules, and multi-task output layers to align the unique characteristics of wireless channel data with the LLM's semantic feature space. Experimental results on a channel-associated multi-task dataset demonstrate that LLM4WM surpasses existing methodologies in both full-sample and few-shot scenarios, owing to its robust multi-task joint modeling and transfer learning capabilities.

1.6. Original Source Link

• Original Source Link: https://arxiv.org/abs/2501.12983
• PDF Link: https://arxiv.org/pdf/2501.12983v2.pdf
• Publication Status: Preprint on arXiv.

2. Executive Summary

2.1. Background & Motivation

The core problem the paper aims to solve is the efficient and robust handling of multiple interconnected tasks in wireless communication systems, often referred to as channel-associated tasks. These tasks, such as channel estimation, prediction, and beam management, are fundamental to communication quality and system design.

This problem is important because:

1. Complexity of Wireless Channels: Modern wireless systems, especially those using millimeter-wave (mmWave) and Multiple-Input Multiple-Output (MIMO) technologies, face increasing complexity due to a growing number of antennas and dynamic environments. Accurate understanding and management of the wireless channel are crucial for optimizing performance.

2. Limitations of Existing AI Approaches: While Artificial Intelligence (AI) has significantly improved various communication tasks, current AI-powered methods often require large amounts of high-quality data, incur substantial communication overhead for data collection and model retraining due to environmental changes, and struggle in complex, highly dynamic scenarios due to limited model scale and generalization issues.

3. Inefficiency of Single-Task Learning: Many communication tasks are deeply rooted in the characteristics of the wireless channel. Jointly learning these channel-associated tasks could enable shared representations and significant training benefits, which single-task approaches fail to capitalize on. Existing multi-task learning methods, however, often face issues with data imbalance, the seesaw effect, and limited capacity to scale with the number and diversity of tasks.

   The paper's entry point and innovative idea revolve around leveraging Large Language Models (LLMs) for wireless multi-tasking. Inspired by the remarkable multi-task handling, reasoning, and generalization capabilities of LLMs in other domains (NLP, healthcare, finance), the authors propose adapting these powerful pre-trained models to wireless communication tasks. The innovation lies in developing a specialized framework, LLM4WM, that effectively transfers the general knowledge of LLMs to the unique context of wireless channel data, while addressing challenges like cross-domain knowledge transfer and task diversity.

2.2. Main Contributions / Findings

The primary contributions of the paper are:

1. Introduction of LLM4WM: The paper proposes LLM4WM, a novel method that pioneers the use of LLMs for wireless multi-tasking. It is the first to fine-tune an LLM using Mixture of Experts with Low-Rank Adaptation (MoE-LoRA) to extract a joint representation specifically for wireless multi-task scenarios, setting a new standard in this research area.

2. Customized Cross-Domain Alignment Mechanisms: To bridge the gap between the LLM's semantic feature space and the specific feature space of wireless tasks, the authors designed:

   • A customized pre-processing method for each task.
   • Corresponding output headers for each task.
   • Multi-task adapters at both the input and output layers of the LLM. These ensure effective alignment and enhance adaptability and performance.

3. Demonstrated Robust Performance and Generalization: The LLM4WM model exhibits excellent performance across a range of wireless communication tasks, including channel estimation, channel prediction, localization enhancement, and beam management. Furthermore, it demonstrates impressive generalization capabilities, highlighting its robustness and versatility for diverse applications in the wireless domain, even in few-shot and transfer learning scenarios.

   Key conclusions and findings reached by the paper include:

• LLM4WM significantly outperforms existing methodologies (traditional, small single-task, small multi-task, and single-task large models) in various channel-associated tasks.
• The MoE-LoRA fine-tuning approach effectively enables tasks to share common knowledge through shared expert weights while maintaining differentiation for task-specific features via independent experts and a gating mechanism.
• Large models, when properly adapted, are better suited for handling multiple downstream tasks in wireless communication compared to small models, which struggle with conflicting task knowledge and limited capacity for joint representations.
• The multi-task adapter modules and preprocessing/output layer designs are crucial for aligning the unique characteristics of wireless channel data with the LLM's semantic space, thereby enhancing performance.
• Ablation studies confirm the critical contribution of each proposed module (preprocessor, multi-task adapters, and backbone LLM) to the overall system performance.
• Hyper-parameter analysis provides insights into optimizing LoRA rank and the number of experts for balanced performance and computational efficiency.
• Efficiency evaluations demonstrate that MoE-LoRA makes LLM4WM comparable in trainable parameters to smaller models, indicating practical deployability.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To understand this paper, a reader should be familiar with the following fundamental concepts:

• Wireless Channel: The medium through which electromagnetic waves travel to transmit information. It's not a perfect conduit; signals are affected by phenomena like fading (variation in signal strength), interference (unwanted signals), and multipath propagation (signals arriving at the receiver via multiple paths due to reflections, diffractions, or scattering). Understanding these characteristics is crucial for designing robust communication systems.

• Millimeter-Wave (mmWave): A band of radio frequencies in the electromagnetic spectrum between 30 GHz and 300 GHz. mmWave offers very high bandwidth, enabling extremely fast data rates. However, it is highly susceptible to blockages and experiences significant path loss, necessitating directional antennas and sophisticated beamforming techniques.

• Multiple-Input Multiple-Output (MIMO): A wireless technology that uses multiple antennas at both the transmitter and receiver to improve communication performance. By exploiting spatial diversity, MIMO can increase data throughput and link reliability without requiring additional bandwidth or transmit power.

• Orthogonal Frequency Division Multiplexing (OFDM): A digital modulation scheme that divides a single high-rate data stream into several lower-rate data streams, which are transmitted simultaneously over multiple orthogonal subcarriers. This makes OFDM robust against frequency-selective fading and inter-symbol interference (ISI).

• Channel State Information (CSI): The known channel properties of a communication link. This information describes how a signal propagates from the transmitter to the receiver, including effects like scattering, fading, and power decay. Accurate CSI is vital for tasks like channel estimation, beamforming, and channel prediction.

• Channel Estimation: The process of acquiring an estimate of the CSI. In wireless systems, the channel characteristics can change rapidly, so accurate and timely estimation is critical for coherent detection and optimizing transmission parameters. Methods often involve sending known pilot signals and inferring the channel from their received versions.

• Channel Prediction: The task of forecasting future channel conditions based on past and current CSI. This is particularly important in dynamic environments with mobile users, where predicting future channel states can enable proactive resource allocation and beamforming.

• Beamforming: A signal processing technique used in smart antennas for directional signal transmission or reception. By combining signals from multiple antennas with appropriate phase and amplitude weights, a beam can be steered towards a desired user, enhancing signal strength and reducing interference. In mmWave, analog beamforming uses phase shifters, while digital beamforming uses independent digital signal processing for each antenna element.

• Artificial Intelligence (AI) in Wireless Communication: The application of machine learning techniques (e.g., neural networks) to solve problems in wireless systems. This includes optimizing network performance, improving signal processing tasks, and enabling intelligent decision-making, often by learning complex patterns from data.

• Multi-Task Learning (MTL): A machine learning paradigm where a single model is trained to perform multiple related tasks simultaneously. The goal is to improve the learning efficiency and prediction accuracy for all tasks by sharing representations that capture common underlying structures. Benefits include improved generalization, reduced overfitting, and the ability to learn from small datasets for individual tasks. Challenges include data imbalance (some tasks have more data than others) and the seesaw effect (improving one task's performance degrades another's).

• Large Language Models (LLMs): A type of artificial intelligence model characterized by a vast number of parameters (billions to trillions) and trained on massive amounts of text data. LLMs exhibit powerful capabilities in natural language understanding, generation, reasoning, and generalization across various domains. They are pre-trained on a general corpus and then often fine-tuned for specific downstream tasks.

• Mixture of Experts (MoE): A neural network architecture that consists of multiple "expert" sub-networks and a "gating network." For a given input, the gating network learns to activate (or weight the outputs of) a subset of experts. This allows the model to selectively use specialized experts for different parts of the input space or different tasks, increasing model capacity without a proportional increase in computational cost for inference.

• Low-Rank Adaptation (LoRA): A parameter-efficient fine-tuning technique for large pre-trained models. Instead of fine-tuning all parameters of a large model, LoRA freezes the pre-trained model weights and injects small, trainable low-rank matrices into each layer (typically the attention layers). These low-rank matrices capture the task-specific adaptations, significantly reducing the number of trainable parameters and computational cost during fine-tuning, while preserving the general knowledge of the pre-trained model.

• Adapter Modules: Small neural network modules inserted into a pre-trained model (e.g., within transformer layers) to adapt it to new tasks or domains. Like LoRA, they aim to reduce the number of trainable parameters during fine-tuning. Adapters typically consist of a down-projection, a non-linearity, and an up-projection, acting as a bottleneck. They allow the pre-trained backbone to remain mostly frozen, preserving its knowledge while the adapters learn task-specific features.

3.2. Previous Works

The paper contextualizes its approach by discussing several prior studies and existing methodologies:

• AI for Channel Estimation and Communication Tasks:

  • The paper acknowledges the significant improvements AI has brought to channel estimation accuracy [6]-[8] and its enhancements to tasks like channel prediction, beamforming, and positioning.
  • Limitation: These methods often require vast amounts of high-quality data, incur substantial communication overhead for retraining in dynamic environments, and frequently struggle in complex scenarios due to limited model scale and generalization issues.

• Multi-modal Sensing and Synesthesia of Machines (SoM):

  • [9] proposed multi-modal sensing to capture wireless channel propagation characteristics, a concept summarized as Synesthesia of Machines (SoM).
  • [10], [11] further explored this for optimizing communication systems.
  • Relevance: This highlights the idea of extracting richer information from the wireless environment, which aligns with the goal of LLM4WM to learn comprehensive channel representations.

• Traditional Multi-Task Learning in Wireless:

  • [12] jointly trained signal classification and modulation recognition.
  • [13] proposed joint training and inference for direct channel and cascaded channel estimation in reconfigurable intelligent surface (RIS) systems to reduce pilot overhead.
  • Limitation: These methods often face issues with data imbalance, the seesaw effect (where optimizing one task degrades another), and struggle to scale with an increasing number and diversity of tasks due to limited model capacity, typically combining only two closely related tasks.

• LLMs in Wireless Communication:

  • LLM4CP [19] was an early work that applied LLMs for channel prediction, demonstrating improved few-shot generalization.
  • WiFo [20] introduced a foundational channel model trained on diverse channel datasets for tasks like time-domain and frequency-domain prediction, focusing on channel reconstruction with zero-shot learning.
  • Limitation: These LLM applications primarily focus on single channel reconstruction tasks or specific predictive capabilities, and do not sufficiently consider the complex relationships and joint learning across multiple diverse channel-associated tasks, which limits their ability to acquire truly general representations across a wider range of wireless objectives.

3.3. Technological Evolution

The evolution of technologies addressing wireless communication challenges can be broadly categorized:

1. Traditional Model-Based Approaches: Early methods relied on mathematical models of the wireless channel and signal processing techniques (e.g., Least Squares for channel estimation, classical codebooks for beamforming). These methods are deterministic but struggle with the complexity and dynamism of real-world channels.
2. Small-Scale AI Models: With the rise of deep learning, specialized neural networks like CNNs, LSTMs, and MLPs were applied to specific wireless tasks. These models learned patterns from data, outperforming traditional methods.
   • Examples: CNN for CSI data as image processing [24], LSTM for fading channel prediction [39], MLP for mapping complex relationships [37], [38], Transformer for channel prediction [23].
   • Multi-task extensions: Approaches like Cross-stitch networks [40] attempted to enable multi-task learning by combining activations from multiple networks.
   • Limitations: These models are often task-specific, require significant data, and struggle with generalization across highly dynamic environments or diverse tasks, necessitating retraining and potentially facing capacity limitations for complex multi-task learning scenarios.
3. Large Pre-trained Models (Foundational Models / LLMs): The recent paradigm shift in AI, especially with Transformers and LLMs, has shown that models pre-trained on vast, diverse datasets can acquire broad general knowledge and remarkable few-shot and zero-shot learning capabilities.
   • Examples: GPT-4 in NLP [14], TTM for time series [18], BloombergGPT in finance [17].
   • Early wireless adaptations: LLM4CP [19] and WiFo [20] demonstrated the potential of LLMs for single wireless tasks.
   • Contribution of This Paper: LLM4WM fits into this timeline by pushing the boundaries of LLM application in wireless. It moves beyond single-task adaptation to a holistic multi-task framework, specifically addressing the unique challenges of learning joint representations for diverse channel-associated tasks by proposing MoE-LoRA and customized adapters.

3.4. Differentiation Analysis

Compared to the main methods in related work, LLM4WM offers several core differences and innovations:

• From Single-Task to Multi-Task LLM Adaptation:

  • Prior LLM works (e.g., LLM4CP, WiFo): Primarily focused on adapting LLMs for single wireless tasks, such as channel prediction or reconstruction. While demonstrating strong performance for their specific tasks, they don't fully exploit the LLM's potential for joint learning across diverse communication objectives.
  • LLM4WM's Innovation: It's a pioneering effort in fine-tuning LLMs for multiple, concurrent channel-associated tasks. This allows the model to learn shared, generalized representations of the wireless channel that benefit all tasks simultaneously.

• Advanced Multi-Task Fine-tuning with MoE-LoRA:

  • Traditional Multi-Task Learning (e.g., Cross-stitch): Often relies on shared bottom layers or explicit feature fusion mechanisms. These approaches can suffer from seesaw effects (conflicting gradients) and limited scalability when the number and diversity of tasks increase. Small models typically lack the capacity to effectively capture diverse task knowledge without conflict.
  • LLM4WM's Innovation: Employs Mixture of Experts (MoE) combined with Low-Rank Adaptation (LoRA). MoE-LoRA allows different tasks to activate specific "expert" sub-networks, enabling both shared knowledge acquisition (through common components) and task-specific specialization (through distinct experts). This dynamically balances commonality and specificity, mitigating the seesaw effect and enhancing scalability. The LoRA component ensures parameter efficiency, making fine-tuning a large LLM feasible.

• Comprehensive Feature Alignment for Wireless Data:

  • General LLM Adaptation: Often involves tokenization and direct mapping of inputs/outputs to the LLM's semantic space. This might be insufficient for the unique structure and characteristics of wireless channel data (e.g., complex numbers, spatio-temporal correlations).
  • LLM4WM's Innovation: Designs specialized preprocessing modules to transform raw channel data (e.g., flattening CSI, DFT for angle domain conversion) before feeding it to the LLM. It also introduces multi-task adapter modules at both input and output to align the specific feature space of wireless tasks with the LLM's general semantic embedding space. Additionally, multi-task output layers with CNNs or MLPs are tailored for task-specific output formats, overcoming the textual output limitation of standard LLMs.

• Leveraging LLM's General Knowledge for Wireless:

  • LLM4WM explicitly aims to transfer the pre-trained LLM's general knowledge (acquired from massive diverse data) to the wireless domain. This is a significant advantage over training models from scratch on limited wireless datasets, leading to improved few-shot learning and generalization capabilities.

    In essence, LLM4WM innovatively combines the power of LLMs with a sophisticated MoE-LoRA fine-tuning strategy and customized domain-specific adapters to create a robust, generalized, and efficient framework for wireless multi-task learning, addressing the shortcomings of previous single-task LLM adaptations and traditional multi-task approaches.

4. Methodology

4.1. Principles

The core idea behind LLM4WM is to adapt a pre-trained Large Language Model (LLM) for diverse wireless channel-associated tasks by leveraging its inherent multi-task learning capabilities and general knowledge. The theoretical basis and intuition are:

1. Shared Representations: Many wireless tasks, despite their different objectives (e.g., estimation, prediction, beamforming), are fundamentally rooted in the characteristics of the wireless channel. A powerful model should be able to learn a joint representation that captures these common underlying features, leading to improved performance for individual tasks. LLMs, with their vast parameter counts and exposure to diverse data, are hypothesized to excel at learning such rich, generalized representations.
2. Transfer Learning: Pre-trained LLMs possess a broad "understanding" of patterns and relationships from their extensive training data. This general knowledge can be transferred to a new, specialized domain (wireless communication) through fine-tuning, allowing the model to quickly adapt and perform well even with limited domain-specific data (few-shot learning).
3. Parameter Efficiency: Fine-tuning an entire LLM for specific tasks can be computationally prohibitive. Parameter-efficient fine-tuning techniques like Low-Rank Adaptation (LoRA) allow adapting the LLM with a small fraction of trainable parameters, making the process feasible.
4. Task Specialization and Collaboration: While shared representations are beneficial, tasks also have unique requirements. A Mixture of Experts (MoE) architecture allows for flexible task specialization by dynamically activating different "expert" sub-networks for different tasks, while still enabling collaboration and knowledge sharing.
5. Domain Alignment: The input and output data in wireless communication (e.g., complex channel matrices, specific metrics) differ significantly from the textual data LLMs are typically trained on. Specialized preprocessing modules and multi-task adapter modules are necessary to effectively translate wireless data into the LLM's semantic feature space and vice-versa.

4.2. Core Methodology In-depth (Layer by Layer)

The LLM4WM framework is designed to integrate the learning of multiple channel-associated tasks within a unified LLM-based network. The complete workflow is illustrated in Figure 1 of the paper.

The following figure (Figure 1 from the original paper) shows the system architecture:

该图像是一个示意图，展示了无线单任务学习和多任务学习的框架。左侧展示了小模型分布式建模网络，右侧展示了大型模型的联合建模网络LLM4WM，包含了多个任务如信道估计、距离估计和路径损耗等，并利用 Mixture of Experts 模型和低秩适应方法。该框架旨在优化无线通信任务的学习效率。

The framework consists of four main components: a Pre-process Module, Multi-Task Adapter Module, Backbone LLM (fine-tuned with MoE-LoRA), and Multi-Task Output Module.

The following figure (Figure 2 from the original paper) shows the detailed structure of the LLM4WM framework:

该图像是一个示意图，展示了LLM4WM框架的结构，包括预处理模块、多任务适配器、主干LLM和多任务输出模块。它涉及数据的归一化、领域转换和线性投影等任务特定操作，并通过Mixture of Experts与Low-Rank Adaptation进行多任务微调。

4.2.1. System Description (Section II)

The paper considers a dual-frequency communication system with both sub-6G and mmWave frequency bands. This system comprises one Base Station (BS) and one User Equipment (UE), both equipped with transceivers for both bands. Multiple-Input Single-Output (MISO) and Orthogonal Frequency Division Multiplexing (OFDM) technologies are applied. The sub-6G and mmWave antennas are co-located and aligned, implying they share spatial features. The BS uses an analog beamforming architecture for mmWave ($N_t$ antennas, Uniform Linear Array (ULA)) and a fully digital beamforming architecture for sub-6G ($\tilde{N}_t$ ULA antennas). The UE has an omnidirectional antenna for both links.

4.2.1.1. Channel Model (II.A)

The channel for both sub-6G and mmWave is described by a classical cluster-based multipath channel model for downlink and uplink CSI at time $t$ and frequency $f$. The channel impulse response h(t, f) is given by: $ h ( t , f ) = \sum _ { n = 1 } ^ { N } \sum _ { p = 1 } ^ { P _ { n } } \beta _ { n , p } e ^ { j [ 2 \pi ( \upsilon _ { n , p } t - f \tau _ { n , p } ) + \Phi _ { n , p } ] } \pmb { a } ( \theta _ { n , p } ) . $ Here:

• $N$: Number of clusters.

• $P_n$: Number of paths in the $n$-th cluster.

• $\beta_{n,p}$: Complex path gain of the $p$-th path in the $n$-th cluster.

• $\upsilon_{n,p}$: Doppler frequency shift of the $p$-th path in the $n$-th cluster.

• $\tau_{n,p}$: Delay of the $p$-th path in the $n$-th cluster.

• $\Phi_{n,p}$: Random phase of the $p$-th path in the $n$-th cluster.

• $\pmb{a}(\theta_{n,p})$: Steering vector of the corresponding path, where $\theta_{n,p}$ is the azimuth angle of departure (AoD).

  For a ULA antenna array, the steering vector $\pmb{a}(\theta_{n,p})$ is expressed as: $ \begin{array} { r } { \pmb { a } ( \theta _ { n , p } ) = [ 1 , e ^ { j \frac { 2 \pi f d _ { \mathrm { v } } \sin ( \theta _ { n , p } ) } { c } } , . . . , e ^ { j \frac { 2 \pi ( N _ { \mathrm { t } } - 1 ) f d _ { \mathrm { v } } \sin ( \theta _ { n , p } ) } { c } } ] . } \end{array} $ Here:

• $d_{\mathrm{v}}$: Antenna spacing in the vertical direction.

• $c$: Speed of light.

• $N_t$: Number of transmit antennas.

4.2.1.2. Signal Model (II.B)

For mmWave links, a downlink MISO-OFDM signal transmission is considered with $K$ subcarriers. The downlink CSI at time $t$ and $k$-th subcarrier is $h_{t,k} = h(t, f_k)$. The received downlink mmWave signal $\pmb{y}_{t,k}$ at the user side for time $t$ and $k$-th subcarrier is: $ { \pmb y } _ { t , k } = { \pmb h } _ { t , k } ^ { \mathrm { H } } { \pmb w } _ { t } x _ { t , k } + { \pmb n } _ { t , k } , $ Here:

• $\pmb{h}_{t,k}^{\mathrm{H}}$: Conjugate transpose of the channel vector.

• $\pmb{w}_t \in \mathbb{R}^{N_v \times 1}$: Beam vector selected from a predefined codebook $\mathcal{W}$, i.e., $\pmb{w}_t \in \mathcal{W}$. $N_v$ is related to the number of antennas.

• $x_{t,k}$: Transmitted symbol.

• $\pmb{n}_{t,k}$: Additive white Gaussian noise (AWGN).

  The achievable spectral efficiency (SE) $R_t$ of the downlink transmission is: $ R _ { t } = \sum _ { k = 1 } ^ { K _ { \mathrm { s } } } \log _ { 2 } \left( 1 + \frac { \vert h _ { t , k } ^ { \mathrm { H } } w _ { t } \vert ^ { 2 } } { \sigma _ { n } ^ { 2 } } \right) . $ Here:

• $K_s$: Number of active subcarriers.

• $|h_{t,k}^{\mathrm{H}}w_t|^2$: Squared magnitude of the effective channel gain with beamforming.

• $\sigma_n^2$: Noise power.

  The optimal beam vector $\pmb{w}_t^*$ is selected through beam training to maximize $R_t$: $ \mathbf { \Delta } \mathbf { w } _ { t } ^ { * } = \arg \operatorname* { m a x } _ { \mathbf { \Delta } w _ { t } \in \mathcal { W } } R _ { t } . $

For sub-6G links, the transmission process is similar but uses fully digital precoding. The uplink channel at pilot positions is estimated using the Least Squares (LS) method: $ \tilde { h } _ { L S } ( t , f _ { k } ) = \tilde { y } _ { t , k } / \tilde { x } _ { t , k } , $ Here:

• $\tilde{x}_{t,k}$: Uplink pilot signal sent by the user.

• $\tilde{y}_{t,k}$: Signal received by the base station.

  To maximize SE, matched filtering based precoding is applied: $ \tilde { w } _ { t , k } ^ { * } = \frac { \tilde { h } _ { L S } ( t , f _ { k } ) } { \lVert \tilde { h } _ { L S } ( t , f _ { k } ) \rVert } , $ Here:

• $\tilde{w}_{t,k}^*$: Optimal precoding vector for time $t$ and $k$-th subcarrier. Its accuracy depends on $\tilde{h}_{LS}(t, f_k)$.

4.2.2. Task Description (Section III)

The paper defines a set of channel-associated tasks $\mathcal{T} = \{CE, CP, PF, BF, DE, PE\}$, categorized into three classes: Channel Reconstruction, Beam Management, and Radio Environment Mining. Each task $n \in \mathcal{T}$ corresponds to a dataset $D_n = \{X_n^I, X_n^L\}$, where $X_n^I$ is the input and $X_n^L$ is the label. All tasks are formulated as an optimization problem: $ \begin{array} { r l } { \underset { \Omega _ { n } } { \operatorname* { m a x } } } & { \mathrm { S c o r e } _ { n } = \mathbb { E } \left{ f _ { e v a l _ { n } } ( X _ { n } ^ { L } , X _ { n } ^ { O } ) \right} } \ { s . t . } & { X _ { n } ^ { O } = f _ { \Omega _ { n } } ( X _ { n } ^ { I } ) , \quad n \in \mathcal { T } , } \end{array} $ Here:

• $f_{eval_n}$: Evaluation function for task $n$.

• $f_{\Omega_n}$: Model function for task $n$, parametrized by $\Omega_n$.

• $X_n^O$: Output results (prediction/estimation) of the model for task $n$.

  The following are the results from Table I of the original paper:

  Task class	Task id	Task content
  Channel Reconstruction	CE	Channel estimation
  	CP PF	Temporal domain channel prediction Frequency domain channel prediction
  Beam Management	BF	Sub-6G assisted mmWave beamforming
  Radio Environment Mining	DE	Distance estimation
  	PE	Path loss estimation

4.2.2.1. Channel Reconstruction: Interpolation and Prediction (III.A)

These tasks aim to predict or interpolate target channel matrices from known ones, capturing inter-domain correlations across time, frequency, and antenna domains. The channel matrix $\pmb{H}$ is considered to have three dimensions: time, space, and antenna. $ H [ i , j , : ] = h ( i \Delta t , f _ { 1 } + ( j - 1 ) \Delta f ) , $ Here:

• $i \Delta t$: Time index with interval $\Delta t$.

• $f_1 + (j-1)\Delta f$: Frequency index with interval $\Delta f$ and lowest frequency $f_1$.

  For Channel Estimation (CE): A comb-type pilot pattern is used. The network interpolates missing channels. $ \begin{array} { l } { { { \cal { X } } _ { C E } ^ { I } = H [ 1 : \tilde { T } , 1 : \tilde { K } / n _ { p i l o t } : \tilde { K } , 1 : \tilde { N } _ { t } ] } } \ { { { \cal { X } } _ { C E } ^ { L } = H [ 1 : \tilde { T } , 1 : \tilde { K } , 1 : \tilde { N } _ { t } ] . } } \end{array} $ Here:

• $\tilde{T}$: Total number of timestamps.

• $\tilde{K}$: Number of OFDM symbols.

• $n_{pilot}$: Number of pilots, typically $\tilde{K} / n_{pilot} = 4$.

• $H[1:\tilde{T}, 1:\tilde{K}/n_{pilot}:\tilde{K}, 1:\tilde{N}_t]$: Input to CE, representing pilots sparse in frequency.

• $H[1:\tilde{T}, 1:\tilde{K}, 1:\tilde{N}_t]$: Label for CE, representing the full channel matrix.

  For Time-Domain Channel Prediction (CP): $ \begin{array} { r l } & { X _ { C P } ^ { I } = H [ 1 : \tilde { T } , 1 : \tilde { K } , 1 : \tilde { N } _ { t } ] } \ & { \boldsymbol { X } _ { C P } ^ { L } = H [ \tilde { T } + 1 : \tilde { T } + \tilde { P } , 1 : \tilde { K } , 1 : \tilde { N } _ { t } ] , } \end{array} $ Here:

• $X_{CP}^I$: Historical channel data up to time $\tilde{T}$.

• $X_{CP}^L$: Future channel data for $\tilde{P}$ moments to be predicted.

  For Frequency-Domain Channel Prediction (PF): $ \begin{array} { l } { { { \pmb X } _ { P F } ^ { I } = { \pmb H } [ 1 : \tilde { T } , 1 : \tilde { K } / 2 , 1 : \tilde { N } _ { t } ] } } \ { { { \pmb X } _ { P F } ^ { L } = { \pmb H } [ 1 : \tilde { T } , \tilde { K } / 2 : \tilde { K } , 1 : \tilde { N } _ { t } ] . } } \end{array} $ Here:

• $X_{PF}^I$: Channel data for the first half of frequencies.

• $X_{PF}^L$: Label for the second half of frequencies.

4.2.2.2. Beam Management: Sub-6G Aided Beamforming (III.B)

This task focuses on acquiring the optimal weight vector $\mathbf{w}_t$ from a codebook $\mathbf{W} \in \mathbb{R}^{\bar{N}_v \times N_c}$ (where $N_c > N_v$ for super-resolution DFT codebook). It leverages the spatial correlation between sub-6G and mmWave bands, particularly in Line-of-Sight (LoS) conditions, to assist mmWave beamforming and reduce pilot overhead. The input $X_{BF}^I$ and label $X_{BF}^L$ for this task are: $ \begin{array} { l } { { { \cal X } _ { B F } ^ { I } = { \cal H } [ 1 , 1 : { \tilde { K } } , 1 : { \tilde { N } } _ { t } ] } } \ { { { \cal X } _ { B F } ^ { L } = w _ { t } ^ { * } . } } \end{array} $ Here:

• $X_{BF}^I$: Sub-6G channel matrix at time 1, across all $\tilde{K}$ subcarriers and $\tilde{N}_t$ antennas.
• $X_{BF}^L$: Optimal mmWave beam vector $w_t^*$.

4.2.2.3. Radio Environment Mining: Distance Estimation and Path Loss Estimation (III.C)

These tasks extract environmental information from channel data, such as distance $x_d$ and path loss $x_{pl}$, to adjust communication system configurations. For Distance Estimation (DE): $ \begin{array} { l } { { { \cal { X } } _ { D E } ^ { I } = { \cal { H } } [ 1 , 1 : \tilde { K } , 1 : \tilde { N } _ { t } ] } } \ { { { \cal { X } } _ { D E } ^ { L } = x _ { d } . } } \end{array} $ Here:

• $X_{DE}^I$: Sub-6G channel matrix.

• $X_{DE}^L$: Distance $x_d$ from UE to BS.

  For Path Loss Estimation (PE): $ \begin{array} { l } { { { \cal X } _ { P E } ^ { I } = H [ 1 , 1 : \tilde { K } , 1 : \tilde { N } _ { t } ] } } \ { { { \cal X } _ { P E } ^ { L } = x _ { p l } . } } \end{array} $ Here:

• $X_{PE}^I$: Sub-6G channel matrix.

• $X_{PE}^L$: Path loss $x_{pl}$.

4.2.3. LLM for Wireless Channel-Associated Tasks (Section IV)

4.2.3.1. Preprocessor Module (IV.A)

Given that each task requires different channel characteristics, a specific preprocessing function $f_{pre,n}(\cdot)$ is designed for each task $n$. The preprocessed data $X_n^{pre}$ for task $n$ is obtained as: $ X _ { n } ^ { p r e } = f _ { \mathrm { p r e } , n } ( X _ { n } ^ { I } ) , $ Here:

• $X_n^I$: Input data for task $n$.

• $f_{pre,n}(\cdot)$: Preprocessing operation for task $n$.

  For channel reconstruction tasks (CE, CP, PF), the preprocessing involves tokenizing the CSI by flattening its spatial and frequency features: $ X _ { n } ^ { p r e } = \mathrm { F l a t t e n } ( X _ { n } ^ { I } , - 2 ) , $ Here:

• $\mathrm{Flatten}(X, i)$: Operation that flattens the $i$-th dimension of tensor $X$ and all subsequent dimensions into a single dimension. For example, if $X_n^I$ has shape $(T, K, N_t)$, and $i=-2$ refers to $K$, then it flattens $K$ and $N_t$ dimensions, resulting in a shape $(T, K \times N_t)$.

  For tasks requiring channel angle features (BF, DE, PE), the CSI data undergoes a domain transformation from the spatial domain to the angle domain using a Discrete Fourier Transform (DFT) matrix: $ X _ { n } ^ { p r e } = X _ { n } ^ { I } F _ { \tilde { N } _ { n } } , $ Here:

• $F_{\tilde{N}_t}$: An $\tilde{N}_t$-dimensional DFT matrix. This transforms the spatial domain CSI into its angular spectrum representation.

4.2.3.2. Multi-Task Adapter Module (IV.B)

This module extends conventional single-task adapters to address multiple tasks simultaneously. It consists of multiple individual adapters, each assigned to a specific task. These adapters perform dimensional alignment and intrinsic representation alignment between the wireless task features and the LLM's semantic space.

The following figure (Figure 3 from the original paper) shows the structure of the multi-task adapter module:

该图像是图示，展示了多任务适配器模块中的维度对齐和特征对齐过程。左侧展示了通过线性层和转置操作实现维度对齐的步骤，右侧则展示了特征对齐的处理流程，包括残差块和激活函数 GELU。

An individual adapter, denoted as $\mathrm{Adapter}_n^{in}$ for task $n$, first uses a linear alignment layer: $ \begin{array} { r } { \pmb { X } _ { n } ^ { f } = \mathrm { L i n e a r } ( \pmb { X } _ { n } ^ { p r e } ) \in \mathbb { R } ^ { L \times D _ { l l m } } , } \end{array} $ Here:

• $X_n^{pre}$: Preprocessed data for task $n$.

• $\mathrm{Linear}(\cdot)$: Operation including at least two fully connected layers that linearly map the first and second dimensions of the input to the specified dimensions.

• $L$: Token length of the LLM's input.

• $D_{llm}$: Hidden dimension of the LLM.

• $X_n^f$: Feature map after linear alignment.

  Following the linear alignment, residual feature extraction networks and an activation function are applied to obtain feature maps with aligned semantic features: $ \begin{array} { r } { \pmb { X _ { n } ^ { a } } = \mathbf { R e s } \big ( \mathbf { G E L U } \big ( \mathbf { R e s } ( \pmb { X _ { n } ^ { f } } ) \big ) \big ) \in \mathbb { R } ^ { L \times D _ { l l m } } , } \end{array} $ Here:

• $\mathrm{Res}(\cdot)$: Operation including $N_{a,i}$ Res-blocks. Each Res-block contains two 1-dimensional convolution kernels (size 3, stride 1) and a ReLU activation function.

• $\mathrm{GELU}(\cdot)$: Gaussian Error Linear Unit, a smooth activation function [30].

• $X_n^a$: Aligned feature map that is fed into the LLM.

  The entire process for the input adapter can be simplified as: $ X _ { n } ^ { a } = \mathrm { A d a p t e r } _ { n } ^ { i n } ( X _ { n } ^ { p r e } ) . $

4.2.3.3. Mixture-of-LoRA Based Fine-tuning (IV.C)

The backbone LLM module is fine-tuned using MoE-LoRA to adapt it to wireless channel tasks while keeping most of its parameters frozen.

First, standard LoRA is introduced. For pre-trained weights $W_0 \in \mathbb{R}^{d_{out} \times d_{in}}$ (where $d_{in}$ is input dimension, $d_{out}$ is output dimension), two trainable low-rank matrices $A \in \mathbb{R}^{r \times d_{in}}$ and $B \in \mathbb{R}^{d_{out} \times r}$ are used. The fine-tuned weights $W$ are: $ W = W _ { 0 } + { \frac { \alpha } { r } } B A , $ Here:

• $W_0$: Original pre-trained weights.

• A, B: Low-rank matrices ($A$ maps $d_{in}$ to $r$, $B$ maps $r$ to $d_{out}$).

• $r$: Rank of the low-rank approximation.

• $\alpha$: Scaling hyperparameter, typically set as $\alpha = 2 \times r$.

  If the input to the feed-forward network is $x_t$ and output is $y_t$, the forward propagation is: $ y _ { t } = W x _ { t } = W _ { 0 } x _ { t } + { \frac { \alpha } { r } } B A x _ { t } . $

To extend this to multi-task learning, MoE principles are incorporated. A collection of independent low-rank matrices (experts) are used, and a gating network selects and combines them. The MoE-LoRA forward propagation is: $ y _ { t } = W x _ { t } = W _ { 0 } x _ { t } + \frac { \alpha } { r } \sum _ { k = 1 } ^ { N _ { e } } \omega _ { k } B _ { k } A _ { k } x _ { t } , $ Here:

• $B_k \in \mathbb{R}^{d_{out} \times r}$ and $A_k \in \mathbb{R}^{r \times d_{in}}$: The $k$-th pair of low-rank matrices (an "expert").

• $N_e$: Number of experts.

• $\omega_k$: Weight of the $k$-th expert, determined by a gating network. A single-layer linear network followed by Softmax(\cdot) is used for the gating network to generate expert weights and normalize them.

  The following figure (Figure 4 from the original paper) illustrates the MoE-LoRA fine-tuning method:

  该图像是一个示意图，展示了MoE-LoRA微调方法的结构。图中显示了多任务学习中的不同LoRA组件如何与预训练权重W0相结合，并通过加法与归一化层和多头注意力层进行交互，以实现任务特定的表现。

MoE-LoRA is applied to the linear layers within the feed-forward network (FFN) of the LLM, freezing other parameters. This reduces trainable parameters and training costs.

4.2.3.4. Multi-Task Output Module (IV.D)

Standard LLMs map to a probability distribution over a vocabulary, which is unsuitable for wireless tasks. Therefore, a specialized output layer is designed.

First, a multi-task adapter (identical to the input adapter, denoted as $\mathrm{Adapter}_n^{out}$) is connected to the LLM's output for task $n$: $ \pmb { X _ { n } ^ { p } } = \mathrm { A d a p t e r } _ { n } ^ { o u t } ( \pmb { X } _ { n } ^ { L L M } ) $ Here:

• $X_n^{LLM}$: LLM's output feature for task $n$.

• $X_n^p$: Output of the multi-task adapter for task $n$.

  Then, different processing networks are applied based on task type:

• For channel estimation and channel prediction tasks (CE, CP, PF), which are sensitive to local features, CNNs are used for processing and dimensional alignment.

• For tasks requiring a global feature representation (BF, DE, PE), such as beamforming, distance estimation, and path loss estimation, the feature map is flattened and processed by an MLP network for feature processing and dimensional alignment. The final prediction/estimation result $X_n^o$ for task $n$ is: $ \begin{array} { r } { { \cal X } _ { n } ^ { \mathrm { o } } = \left{ \begin{array} { l l } { \mathrm { C N N } ( X _ { n } ^ { p } ) , } & { n \in { C E , C P , P F } } \ { \mathrm { M L P } ( X _ { n } ^ { p } ) , } & { n \in { B F , D E , P E } } \end{array} \right. } \end{array} $ Both CNN and MLP networks are typically three layers, with the final layer being a single fully connected layer to align output dimensions.

4.2.3.5. Training Configuration (IV.E)

The network is trained on a multi-task mixed dataset using a two-stage training approach:

1. Stage 1: Only the multi-task adapters and output layer are trained. The LLM parameters are frozen. In this stage, the model learns the mapping between the task feature space and the pre-trained LLM's text feature space.

2. Stage 2: The LLM is fine-tuned using MoE-LoRA. The multi-task adapters become frozen, but the output layer remains trainable. This stage leverages the LLM for joint modeling and generalized representations.

   The same loss function is used for both stages: $ \operatorname { L o s s } = \sum _ { n } \omega _ { n } f _ { l o s s , n } ( X _ { n } ^ { \mathrm { o } } , X _ { n } ^ { \mathrm { l } } ) $ Here:

• $f_{loss,n}$: Loss function for task $n$.

• $X_n^o$: Predicted/estimated output for task $n$.

• $X_n^L$: True label for task $n$.

• $\omega_n$: Task weighting. The Dynamic Weight Average (DWA) algorithm [33] is used to dynamically adjust $\omega_n$ each epoch based on task losses, ensuring all tasks are well-trained.

  The choice of $f_{loss,n}$ depends on the task:

• For classification problems (e.g., BF), cross-entropy loss is used.

• For regression problems (e.g., CP), normalized mean square error (NMSE) [23] is used.

5. Experimental Setup

5.1. Datasets

The experiments utilize time-varying CSI datasets simulated using the QuaDRiGa channel generator [34], which is compliant with 3GPP standards.

• System Setup: A dual-frequency wireless system with a 1.9 GHz sub-6G link and a 28 GHz mmWave link is considered.

• Scenario: $3GPP_38.901_UMa_LOS$ (Urban Macrocell Line-of-Sight) is the primary scenario for dataset generation.

• Data Generation Parameters: The following are the results from Table II of the original paper:

  Parameter	mmWave	sub-6G
  Scenario	3GPP_38.901_UMa_LOS
  Active BSs	1	1
  Codebook size	256	N/A
  Transmit antennas	64	8
  Center frequency (GHz)	28	1.9
  Bandwidth (GHz)	0.5	0.06
  Antenna spacing	0.5	0.5
  OFDM sub-carriers	64	64
  Clusters	N/A	21
  Paths per cluster	N/A	20

• Sub-6G Link Details: Operates in FDD mode. Uplink and downlink channels are adjacent. A pilot is placed every 8 subcarriers for the uplink channel.

  • For channel prediction (CP): Future $\tilde{P}=4$ RBs are predicted based on historical $\tilde{T}=16$ RBs. The time interval of pilots is $0.5 \mathrm{~ms}$.
  • For frequency-domain prediction (PF): Downlink channel at pilot locations is inferred from the uplink channel (estimated or predicted).

• mmWave Link Details: Employs TDD mode.

  • For sub-6G assisted mmWave beamforming (BF): Downlink analog precoding is derived based on the spatial correlation of the uplink sub-6G channel estimated by the uplink pilot.

• User Mobility: Initial user position is randomized. Motion trajectory is linear at a speed of $30 \mathrm{~km/h}$.

• Dataset Size: Total of 20,000 samples.

  • Training set: 15,000 samples.

  • Validation set: 1,600 samples.

  • Test set: 3,400 samples.

    These datasets are effective for validating the method's performance because they are simulated using a widely accepted channel generator (QuaDRiGa) and adhere to 3GPP standards, making them representative of realistic wireless communication environments. The inclusion of dual-frequency bands and user mobility adds to the complexity and realism, allowing for a comprehensive evaluation of LLM4WM's capabilities across diverse tasks.

5.2. Evaluation Metrics

The paper uses task-specific metrics for evaluation, along with an average metric and Spectral Efficiency (SE).

1. Normalized Mean Square Error (NMSE): Used for channel reconstruction tasks (CE, CP, PF) and path loss estimation (PE).

   • Conceptual Definition: NMSE is a normalized measure of the average squared difference between predicted values and ground truth values. It is commonly used to quantify the accuracy of regression models, especially when the magnitude of the target variable can vary significantly. Normalization by the power of the ground truth makes it scale-independent and comparable across different channel conditions. A lower NMSE indicates better performance.
   • Mathematical Formula: $ \mathrm{NMSE} = \frac { \mathbb { E } { | \mathbf { \hat { X } } - \mathbf { X } | _ { 2 } ^ { 2 } } } { \mathbb { E } { | \mathbf { X } | _ { 2 } ^ { 2 } } } $
   • Symbol Explanation:
     • $\mathbb{E}\{\cdot\}$: Statistical expectation, typically approximated by averaging over the test samples.
     • $\| \cdot \|_2^2$: Squared $l_2$-norm, representing the squared Euclidean distance or power.
     • $\mathbf{\hat{X}}$: Predicted channel matrix, path loss, or other regression target.
     • $\mathbf{X}$: Ground truth (actual) channel matrix, path loss, or other regression target.

2. Top-1 Accuracy (Acc): Used for the beam management task (BF).

   • Conceptual Definition: Top-1 accuracy measures the proportion of samples for which the model's highest-probability prediction matches the true label. In the context of beamforming, it indicates how often the model correctly predicts the optimal beam index. A higher accuracy indicates better performance.
   • Mathematical Formula: $ \mathrm{Acc} = \frac{1}{N_{samples}} \sum_{i=1}^{N_{samples}} \mathbb{I}(\hat{y}_i = y_i) $
   • Symbol Explanation:
     • $N_{samples}$: Total number of samples in the evaluation set.
     • $\mathbb{I}(\cdot)$: Indicator function, which equals 1 if the condition inside is true, and 0 otherwise.
     • $\hat{y}_i$: The model's predicted label (e.g., beam index) for the $i$-th sample.
     • $y_i$: The true label for the $i$-th sample.

3. Mean Absolute Error (MAE): Used for distance estimation (DE).

   • Conceptual Definition: MAE measures the average magnitude of the errors between predicted values and ground truth values. It provides a linear score where all individual differences are weighted equally. MAE is less sensitive to outliers compared to MSE. A lower MAE indicates better performance.
   • Mathematical Formula: $ \mathrm{MAE} = \frac{1}{N_{samples}} \sum_{i=1}^{N_{samples}} |\hat{y}_i - y_i| $
   • Symbol Explanation:
     • $N_{samples}$: Total number of samples in the evaluation set.
     • $\hat{y}_i$: The model's predicted value for the $i$-th sample.
     • $y_i$: The true value for the $i$-th sample.
     • $|\cdot|$: Absolute value.

4. Average Metric (Avg.): A composite metric across all tasks for intuitive comparison.

   • Conceptual Definition: This metric provides a single normalized score that averages the performance across all six diverse tasks. It's constructed such that a lower value always indicates better overall performance. For accuracy-based metrics, (1 - Accuracy) is used to align with the "lower is better" philosophy of error metrics.
   • Mathematical Formula: $ \mathrm { A v g . } = \frac { 1 } { 6 } \ast [ \mathrm { N M S E } \left( \mathrm { C E } \right) + \mathrm { N M S E } \left( \mathrm { C P } \right) + \mathrm { N M S E } \left( \mathrm { P F } \right) + \ ( 1 - \mathrm { A c c } \left( \mathrm { B F } \right) ) + \mathrm { M A E } \left( \mathrm { D E } \right) + \mathrm { N M S E } \left( \mathrm { P E } \right) ] . $
   • Symbol Explanation:
     • $\mathrm{NMSE}(\cdot)$: Normalized Mean Square Error for the specified task.
     • $\mathrm{Acc}(\mathrm{BF})$: Top-1 Accuracy for the Beamforming task.
     • $\mathrm{MAE}(\mathrm{DE})$: Mean Absolute Error for the Distance Estimation task.
     • $\mathrm{CE}, \mathrm{CP}, \mathrm{PF}, \mathrm{BF}, \mathrm{DE}, \mathrm{PE}$: Task IDs representing Channel Estimation, Channel Prediction, Frequency-domain Prediction, Beamforming, Distance Estimation, and Path Loss Estimation, respectively.

5. Spectral Efficiency (SE): Reflects the overall performance of the communication system.

   • Conceptual Definition: SE measures the achievable data rate per unit of bandwidth (bits/s/Hz) that can be transmitted over a wireless communication link. It is a crucial metric for evaluating the efficiency of a communication system. Higher SE indicates a more efficient use of the available spectrum. In this context, it's calculated using the channel information predicted or estimated by the model.
   • Calculation: Referred back to Eq. (4) (see Section 4.2.1.2) where $h_{t,k}$ is the actual CSI and $\pmb{w}_t$ is obtained as per Eq. (7) using the predicted $h_{t,k}$. The communication SNR is set as $1 / \sigma_n^2 = 10 \mathrm{~dB}$.

5.3. Baselines

To demonstrate the superiority of LLM4WM, the authors compare it against a comprehensive set of baselines, categorized by their approach:

1. Traditional Methods (without deep learning): These methods do not involve a training process and rely on inherent channel characteristics.

   • BI (Bilinear Interpolation): Treats CSI as a time series and uses bilinear interpolation to complete channel reconstruction tasks.
   • Codebook [35]: For beam management, this method uses a super-resolution codebook and spatial correlation in the sub-6G band to find the optimal mmWave downlink beam.
   • FIFS (Fine-Grained Indoor Fingerprinting System) [36]: A CSI-based fingerprinting system that utilizes a coherence bandwidth-enhanced probability algorithm and a correlation filter to map objects to fingerprints, implemented for radio environment mining tasks.

2. Single-Task Small Model Methods: These employ specially designed, relatively small-parameter models for specific downstream tasks.

   • MLP (MultiLayer Perceptron) [37], [38]: Used for radio environment sensing and beam management.
   • LSTM (Long Short-Term Memory) [39]: A recurrent neural network designed for sequence data, implemented with 4 LSTM layers for channel reconstruction tasks.
   • CNN (Convolutional Neural Network) [24]: A CNN-based predictor (ten convolutional layers, $3 \times 3$ kernel) for FDD systems, treating time-frequency CSI data as a 2D image processing task. Implemented for channel reconstruction tasks.
   • WiT (Wireless Transformer) [26]: A transformer-based location estimation method leveraging the attention mechanism for robust learning. Implemented for radio environment sensing tasks.
   • Transformer [23]: A transformer-based parallel channel predictor (3 encoders, 2 decoders) for TDD systems to mitigate error propagation. Implemented for channel reconstruction tasks.

3. Multi-Task Small Model Methods: These use techniques like low-level sharing and cross-feature fusion to enable feature sharing across different tasks.

   • Cross-stitch [40]: A convolutional multi-task learning neural network with "cross-stitch" units to combine activations from multiple networks. Uses ResNet [41] as the backbone.
   • Cross-stitch(s): A variant of Cross-stitch where it performs only a single task, serving as a baseline to illustrate the impact of multi-task learning for small models.

4. Single-Task Large Model Methods: These fine-tune large pre-trained models for a single downstream task.

   • LLM4CP [19]: The first method to apply LLMs to the channel prediction task through fine-tuning. Uses GPT-2 as the backbone LLM and LN Tuning [42] as the fine-tuning method, implemented for channel reconstruction tasks.

   • LLM4WM(s): A single-task fine-tuning version of the proposed LLM4WM framework, performing only a single task. This serves as an ablation to show the benefit of multi-task learning within the LLM4WM architecture.

     These baselines are representative because they cover a wide spectrum of approaches, from traditional signal processing to state-of-the-art deep learning models, including both small and large models, and both single-task and multi-task learning paradigms relevant to wireless communication. This diverse comparison provides a robust validation of LLM4WM's performance.

5.4. Network and Training Parameters

• Multi-Task Adapter Module: Uses $N_a = 8$ for input and output feature alignment.

• MoE-LoRA Fine-tuning:

  • Number of experts: 8.
  • LoRA rank ($r$): 8 for each LoRA matrix.

• Multi-Task Output Module:

  • For CE, CP, PF: Three-layer CNN with $3 \times 3$ kernels.
  • For BF, DE, PE: Three-layer MLP with 768-dimensional features.
  • Both types of output modules use only one single-layer fully connected network for output dimension alignment.

• Backbone LLM: Smallest version of GPT-2 [43] with a feature dimension of $F = 768$. The first $N_L = 6$ layers of GPT-2 are deployed.

• Scheduler: Both warm-up and cosine annealing scheduler are employed.

  • Warm-up phase: First 50 epochs, learning rate increases linearly from $1 \times 10^{-5}$ to $1 \times 10^{-3}$.
  • Subsequent training: Learning rate dynamically adjusted using cosine annealing.

• Hyperparameters for Network Training: The following are the results from Table III of the original paper:

  Parameter	Value
  Batch size	512
  Epochs	250
  Optimizer	Adam (betas=(0.9, 0.999))
  Learning rate scheduler	Cosine Annealing
  Cosine annealing period	100 epochs
  Learning rate range	[1 × 10−5, 1 × 10−3]

6. Results & Analysis

6.1. Core Results Analysis

The LLM4WM framework is evaluated against various baselines across six channel-associated tasks. The primary performance indicators are NMSE (for channel reconstruction and path loss estimation, lower is better), Top-1 Accuracy (for beamforming, higher is better), MAE (for distance estimation, lower is better), and Spectral Efficiency (SE) (for overall system performance, higher is better). The custom Avg. metric provides a unified measure where lower values indicate better overall performance.

The following are the results from Table IV of the original paper:

Analysis:

1. Overall Dominance of LLM4WM: LLM4WM consistently achieves the best performance across almost all tasks and metrics, as indicated by the bolded values, and achieves the lowest Avg. ↓ score of 0.087. This demonstrates its robust multi-task joint modeling and transfer learning capabilities.

2. Superiority over Traditional and Small Models: Traditional methods (BI, Codebook, FIFS) and many small models (CNN, LSTM, MLP) perform significantly worse than LLM4WM, highlighting the limitations of non-learning or capacity-constrained approaches. For instance, BI and LSTM show very high NMSE for CE, CP, PF tasks, and Codebook has very low Acc for BF.

3. Advantage over Single-Task LLM Adaptation (LLM4CP, LLM4WM(s)):

   • LLM4WM outperforms LLM4CP in most tasks (e.g., lower NMSE for CE, PF; higher Acc for BF; lower MAE for DE; lower NMSE for PE). This indicates the benefit of LLM4WM's multi-task design over single-task LLM fine-tuning.
   • When comparing LLM4WM to its single-task variant LLM4WM(s), LLM4WM still performs better across most metrics (e.g., lower NMSE for CE, PF; higher Acc for BF; lower MAE for DE; lower NMSE for PE). This clearly validates the effectiveness of the multi-task learning aspect of the proposed framework. The authors note that single-task fine-tuning is prone to overfitting, which LLM4WM addresses through its multi-task approach.

4. Impact of Multi-Task Learning (Small vs. Large Models): The paper highlights a crucial distinction in the benefits of multi-task learning for small vs. large models.

   • Small models (Cross-stitch vs. Cross-stitch(s)): Cross-stitch (multi-task) shows only a marginal improvement over Cross-stitch(s) (single-task), with Avg. scores of 0.134 vs. 0.140. The average improvement is only $0.19 \mathrm{~dB}$ (likely referring to an overall SE metric or similar aggregated performance not explicitly listed for small models). This suggests small models struggle to effectively extract generalized representations due to conflicting task knowledge.
   • Large models (LLM4WM vs. LLM4WM(s)): LLM4WM (multi-task) achieves a much larger improvement over LLM4WM(s) (single-task), with Avg. scores of 0.087 vs. 0.109. This represents an improvement of $0.99 \mathrm{~dB}$ (again, likely an aggregated metric). This finding strongly supports the hypothesis that large models possess a greater capacity to learn and utilize joint representations across diverse tasks.

5. Spectral Efficiency (SE) Improvement: LLM4WM achieves the highest SE values across CE, CP, PF, and BF tasks, reaching up to $6.081 \mathrm{~bit}(\mathrm{s} \cdot \mathrm{Hz})^{-1}$ for PF and $8.557 \mathrm{~bit}(\mathrm{s} \cdot \mathrm{Hz})^{-1}$ for BF. This directly translates to improved communication system performance.

   The following figure (Figure 5 from the original paper) illustrates the performance comparison of large and small models before and after wireless multi-task learning:

   该图像是六边形雷达图，展示了大模型（LM）和小模型（SM）在多任务学习（MTL）和单任务学习（STL）中各评估指标的性能比较。图中分别标识了不同模型在通道效率（CE）、通道潜力（CP）、信号质量（PQ）、边缘性能（PE）、延迟（DE）和频谱效率（PF）等方面的表现。

This radar chart visually reinforces the findings from Table IV regarding the multi-task learning impact. It shows that Large Models (LM) exhibit a more significant improvement when transitioning from Single-Task Learning (STL) to Multi-Task Learning (MTL) compared to Small Models (SM). The LM-MTL polygon (blue) covers a larger area than LM-STL (green), while the difference between SM-MTL (orange) and SM-STL (red) is less pronounced. This confirms that LLMs are better suited for multi-task learning in this context.

The following figure (Figure 6 from the original paper) shows the Pearson correlation coefficient heatmap of expert combination weights for various tasks:

Analysis of Expert Combination Weights:

This heatmap visualizes the Pearson correlation coefficient between the expert combination weights (i.e., $\omega_k$ from Eq. 24) learned by the gating network in two randomly selected MoE-LoRA layers.

• Distinct Expert Allocation: The overall low correlation between expert weights for most tasks (indicated by lighter colors or smaller coefficient values) suggests that the gating network successfully learns to activate distinct expert combinations for different task types. This is a key mechanism of MoE, allowing for task-specific specialization without having to train entirely separate models.
• Task Similarity Reflected: Tasks with similar characteristics (e.g., likely those within the Channel Reconstruction class or related Radio Environment Mining tasks) tend to exhibit higher correlations (darker cells). This indicates that their optimal expert combinations might share common experts, allowing for efficient knowledge sharing. This finding supports the design principle of MoE-LoRA to balance shared knowledge and task-specific differentiation.

6.2. Generalization Experiments

Generalization is crucial for real-world deployment. The paper tests LLM4WM's generalization in two scenarios:

1. Scenario Transfer: Training on the UMa (Urban Macrocell) dataset and testing on the RMa (Rural Macrocell) dataset (using only 10% of the RMa data for transfer).

2. Frequency Transfer: Training on the 1.9 GHz sub-6G link dataset and testing on a 2.4 GHz sub-6G link dataset.

   The following are the results from Table V of the original paper:

   Train Set	Test Set	Method	CE	CP	PF	Method	BF	Method	DE	PE	Avg. ↓
   			NMSE ↓	NMSE ↓	NMSE ↓		Acc ↑		MAE ↓	NMSE ↓
   UMa1.9GHz	RMa1.9GHz	LLM4WM	0.143	0.145	0.162	LLM4WM	0.413	LLM4WM	0.336	0.285	0.276
   		LLM4CP	0.177	0.133	0.292	LLM4CP	0.306	LLM4CP	0.370	0.311	0.330
   		CNN	0.187	0.137	0.384	CNN	0.215	WiT	0.339	0.220	0.376
   		LSTM	1.000	0.309	0.545	MLP	0.365	MLP	0.539	0.473	0.584
   	UMa2.4GHz	LLM4WM	0.101	0.110	0.135	LLM4WM	0.785	LLM4WM	0.126	0.047	0.122
   		LLM4CP	0.110	0.113	0.196	LLM4CP	0.685	LLM4CP	0.182	0.073	0.165
   		CNN	0.115	0.121	0.381	CNN	0.375	WiT	0.143	0.047	0.239
   		LSTM	1.000	0.174	0.340	MLP	0.769	MLP	0.256	0.134	0.356

Analysis:

1. Robust Generalization of LLM4WM: In both scenario transfer (UMa to RMa) and frequency transfer (1.9GHz to 2.4GHz), LLM4WM consistently achieves the best overall Avg. ↓ metric (0.276 and 0.122 respectively), indicating superior generalization capabilities compared to other methods.
2. Performance in Complex Tasks: LLM4WM particularly excels in complex tasks like channel estimation (CE) and channel prediction (CP, PF), which require understanding multi-dimensional features. For instance, in UMa to RMa transfer, LLM4WM has significantly lower NMSE for CE and PF compared to LLM4CP and small models. This confirms the value of large models in handling dynamic real-world communication scenarios.
3. Small Models Struggle with Generalization: Small models like LSTM and CNN generally show much poorer generalization, especially LSTM with an NMSE of 1.000 for CE in both transfer scenarios, indicating complete failure for that task under new conditions. MLP also performs poorly.
4. Radio Environment Mining Anomaly: The paper notes a "slight dip" in radio environment mining performance for LLM4WM in some generalization cases, where smaller models like WiT might perform comparably or slightly better for specific metrics (e.g., WiT has comparable NMSE for PE in the 2.4GHz test set). This is attributed to the relative simplicity of these tasks in the LOS (Line-of-Sight) scenario, where simpler models can be sufficient. However, the overall Avg. metric still strongly favors LLM4WM.
5. Multi-Task Generalization: The ability of LLM4WM to generalize well across multiple tasks simultaneously in new environments/frequencies is a key strength, reducing the need for frequent retraining or task-specific models in dynamic wireless settings.

6.3. Hyper-parameter Analysis

The paper investigates the impact of LoRA rank and the number of experts on LLM4WM's performance.

The following figure (Figure 7 from the original paper) shows the performance of LLM4WM under different Lora ranks and number of experts:

Analysis:

1. Effect of LoRA Rank:

   • As the LoRA rank increases (from 2 to 10 on the left plot), the performance of LLM4WM (measured by Average Loss) gradually improves. This is expected because a higher rank means more trainable parameters are introduced, allowing the model to adapt more finely to the data distribution.
   • However, increasing the rank also leads to higher training overhead (more trainable parameters).
   • The authors found that a LoRA rank of 8 provided the "optimal balance" between performance gain and computational efficiency, likely reaching a point of diminishing returns for further increases.

2. Effect of Number of Experts:

   • With the LoRA rank fixed at 8, increasing the number of experts (from 2 to 10 on the right plot) also generally leads to a decrease in Average Loss, meaning improved performance.

   • A larger number of experts enhances the model's analytical and representational capacity, allowing for more specialized learning paths for different tasks or input types.

   • Similar to LoRA rank, the increase in experts also linearly increases training and inference costs.

   • The authors concluded that setting the number of experts to 8 was the most appropriate choice, balancing performance and computational efficiency.

     This analysis validates the selected hyperparameter settings as a result of a careful trade-off between model accuracy and practical inference/training costs.

6.4. Ablation Experiments

Ablation studies were conducted to determine the effectiveness and contribution of individual modules within the LLM4WM framework.

The following are the results from Table VI of the original paper:

Analysis:

1. Effectiveness of Multi-Task Adapters:

   • w/o Adapterin: Removing the input adapter leads to a 6.50% increase in Average Loss (from 0.087 to 0.092).
   • w/o Adapterout: Removing the output adapter leads to a 9.54% increase in Average Loss (from 0.087 to 0.095).
   • w/o Adapter: Removing both input and output adapters results in the largest performance drop among adapter-related ablations, with a 17.62% increase in Average Loss (from 0.087 to 0.102).
   • These results clearly demonstrate the critical role of the multi-task adapter modules in aligning feature spaces between wireless data and the LLM's semantic space. Both input and output adapters are necessary, with the output adapter appearing slightly more impactful in this specific setup.

2. Critical Role of the Backbone LLM:

   • w/o LLM: Completely removing the large model (LLM backbone) results in the most significant performance degradation, with a 34.40% increase in Average Loss (from 0.087 to 0.117). This highlights that the backbone LLM is the "critical engine" for the success of multi-task joint learning in wireless tasks, confirming the hypothesis that large models' general knowledge and capacity are essential.

   • Frozen LLM: Freezing the LLM parameters entirely (i.e., not applying MoE-LoRA fine-tuning) leads to a 6.15% increase in Average Loss (from 0.087 to 0.092). This shows that while the pre-trained LLM provides a strong foundation, the MoE-LoRA fine-tuning is necessary to adapt its general knowledge effectively to the specific nuances of wireless tasks and achieve optimal performance.

     In summary, all modules (input adapters, output adapters, and the fine-tuned LLM backbone) contribute significantly to LLM4WM's performance, validating the integrated design of the framework.

6.5. Efficiency Evaluation

The paper assesses the training and inference costs of LLM4WM compared to baselines to evaluate its practical deployability.

The following are the results from Table VII of the original paper:

Analysis:

1. Parameter Efficiency of MoE-LoRA:

   • LLM4WM has a Total Network parameters (M) of 88.71 M, which is large due to the GPT-2 backbone.
   • However, its Trainable Network parameters (M) is only 1.13 M. This is remarkably low, even smaller than MLP (1.29 M), CNN (2.14 M), WiT (19.19 M), and LLM4CP (1.80 M), and comparable to LSTM (1.17 M).
   • This low number of trainable parameters is a direct result of the MoE-LoRA fine-tuning method, which freezes most of the LLM's backbone and only trains small, low-rank matrices and the adapters. This significantly reduces training costs and makes the adaptation of a large model feasible. The paper emphasizes that adding new tasks only increases the model's parameters by about 1.13 M, which is a small fraction of the total, indicating high parameter efficiency.

2. Acceptable Inference Time:

   • The Interference time (ms) for LLM4WM is 6.00 ms. While higher than very small models like MLP (0.32 ms) and CNN (0.49 ms), it is comparable to or even better than other complex models like LSTM (6.49 ms) and significantly faster than LLM4CP (8.62 ms).

   • This "acceptable" inference speed, despite using a large LLM backbone, is achieved by utilizing a lightweight GPT-2 version and the parameter-efficient nature of MoE-LoRA which means that during inference, only the activated experts and small LoRA matrices contribute to the computation alongside the frozen backbone.

     The efficiency evaluation demonstrates that LLM4WM successfully addresses the challenge of deploying large models in practical wireless scenarios. Its parameter efficiency makes training manageable, and its acceptable inference speed makes it suitable for real-time or near real-time communication systems. This suggests LLM4WM has significant potential for future wireless communication systems that require robust multi-tasking and customization.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper introduces LLM4WM, a novel multi-task fine-tuning framework specifically designed to adapt Large Language Models (LLMs) for various channel-associated tasks in wireless communication systems. By leveraging a diverse multi-task dataset, LLM4WM enables concurrent execution of tasks like channel estimation, prediction, beam management, and radio environment mining.

The core innovations enabling this are:

1. MoE-LoRA Integration: The framework incorporates Mixture of Experts with Low-Rank Adaptation (MoE-LoRA) into the fine-tuning process. This empowers the LLM backbone to dynamically adapt by optimally combining specialized expert modules, facilitating the extraction of shared representations across tasks while also improving task-specific performance and ensuring parameter efficiency.

2. Multi-Task Adapter for Feature Alignment: Custom-designed multi-task adapter modules are employed at both the input and output layers to harmonize the distinct feature spaces of wireless tasks with the LLM's general semantic embedding space, ensuring coherent task alignment and improved adaptability. Task-specific preprocessing modules and output layers further refine this alignment.

   Extensive simulation results demonstrate that LLM4WM exhibits robust multi-task learning and generalization capabilities, outperforming existing methodologies in both full-sample and few-shot evaluations. Ablation studies confirm the critical contributions of each module (adapters, MoE-LoRA, and the LLM backbone) to the overall system performance. The expert weight heatmap further validates the efficacy of the MoE mechanism in adaptively allocating resources, enhancing model specialization and flexibility. The efficiency evaluation indicates practical deployability due to parameter-efficient fine-tuning and acceptable inference times.

7.2. Limitations & Future Work

The paper's conclusion section highlights the "preliminary simulation results" which, while robust, implies that real-world deployment or validation remains a future step. While the authors don't explicitly list a "Limitations" section, some can be inferred:

• Simulation-Based Evaluation: All experiments are conducted on simulated datasets (QuaDRiGa). Real-world wireless channels are notoriously complex and diverse, posing challenges that simulations might not fully capture (e.g., unexpected interference, hardware impairments, diverse environmental dynamics).

• Specific LLM Backbone: The paper uses a specific, smaller version of GPT-2. While MoE-LoRA makes it efficient, the choice of LLM backbone and its pre-training data might inherently limit the types of knowledge transferable to wireless. The current LLM might not have "seen" anything analogous to wireless channel data during its initial language training.

• Static Expert Allocation: While MoE allows dynamic selection of experts, the number of experts and their underlying structure are fixed. Adapting this dynamically could be a future enhancement.

• Interpretability: LLMs are often black-box models. Understanding why an LLM makes certain predictions or how it internally represents wireless channel features could be challenging, which is important for critical communication systems.

• Computational Overhead: While MoE-LoRA significantly reduces trainable parameters and achieves acceptable inference speeds, the total parameter count of the LLM backbone is still substantial (88.71 M). This might still pose challenges for deployment on resource-constrained edge devices common in wireless networks.

  The paper implicitly suggests future work by stating that LLM4WM "demonstrates significant potential for deployment in future communication scenarios marked by increasing demand and the need for customized services involving numerous tasks." This implies:

• Real-world Deployment and Validation: Moving beyond simulations to evaluate LLM4WM in live communication networks.

• Larger Scale and Diversity of Tasks: Exploring the framework's scalability to an even greater number and more diverse set of wireless tasks.

• Adaptive LLM Architectures: Potentially exploring other LLM architectures or more advanced MoE designs that could further optimize performance or efficiency.

• On-Device Deployment: Researching optimizations for deploying LLM4WM on edge devices with limited computational resources.

7.3. Personal Insights & Critique

This paper presents an exciting and highly relevant direction for integrating cutting-edge AI (LLMs) with fundamental wireless communication challenges. The idea of leveraging the vast pre-trained knowledge of LLMs for multi-task learning in wireless is conceptually very powerful.

Insights:

• Paradigm Shift for Wireless AI: The work represents a significant step towards "foundational models" in wireless. Instead of designing a new neural network from scratch for each wireless task, this approach suggests fine-tuning a powerful, general-purpose model, potentially accelerating research and development in the field.
• Implicit Feature Extraction: LLMs, despite being language models, learn complex patterns and relationships. The success of LLM4WM implies that the abstract numerical patterns in wireless channel data can be effectively mapped and processed by these models, possibly by aligning wireless data to a "semantic space" where LLMs can find analogies to their learned representations. This is a fascinating cross-domain transfer.
• Efficiency of MoE-LoRA: The demonstration of parameter efficiency and acceptable inference times with MoE-LoRA is crucial. Without these techniques, the direct application of LLMs to wireless would likely be impractical. This makes the approach much more viable.
• Multi-Task Synergy: The compelling evidence that large models benefit significantly more from multi-task learning than small models is a key takeaway. It highlights that the sheer capacity of LLMs allows them to learn truly generalized representations without the conflicting gradients often seen in smaller multi-task models.

Critique / Areas for Improvement:

• Interpretability for Wireless Engineers: While LLMs perform well, the lack of interpretability can be a hurdle in mission-critical wireless systems. For example, if the model predicts a specific beam, understanding why it chose that beam (e.g., what channel features it prioritized) would be invaluable for system validation and debugging. Future work could explore methods to make the LLM's internal "reasoning" for wireless tasks more transparent.

• Real-World vs. Simulation Gap: The reliance on simulated data, while standard, means there's a significant gap to bridge before practical deployment. Real-world channels have complexities (e.g., non-stationary statistics, hardware imperfections, measurement noise, diverse interference) that QuaDRiGa might not fully model. Validation with measured channel data would be a strong next step.

• Computational Cost for Fine-tuning: While trainable parameters are low, the initial pre-training of the LLM and the full-model loading still require significant computational resources. This might limit who can replicate or further develop such models, favoring well-resourced institutions.

• Cold Start Problem for New Tasks: While the MoE-LoRA handles new tasks efficiently, the process of designing the specific preprocessing modules, multi-task adapters, and output layers for each new type of wireless task still requires domain expertise and effort.

• Data Representation Nuances: The paper details how wireless data is preprocessed to be compatible with the LLM (e.g., flattening, DFT). A deeper analysis of how these representations align with the LLM's internal token embeddings (e.g., what "tokens" actually represent in the wireless domain) could offer more theoretical understanding.

  Overall, LLM4WM offers a promising new direction for enhancing wireless communication systems using the power of LLMs, demonstrating the feasibility and benefits of transfer learning and multi-task learning in this domain. Its robust performance and efficiency indicate that foundational models for wireless are becoming a tangible reality.