1. Bibliographic Information

• Title: OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models
• Authors: Huanpeng Chu, Wei Wu, Guanyu Fen, Yutao Zhang
• Affiliations: The authors are affiliated with Zhipu AI and Nanjing University.
• Journal/Conference: The paper is available on arXiv, which is a preprint server for academic papers. This means it has not yet undergone formal peer review for a conference or journal.
• Publication Year: The paper's content and references (citing works from 2024 and even a placeholder for 2025) indicate it is a very recent work, likely from 2024.
• Abstract: The paper introduces OmniCache, a novel training-free method to accelerate the inference of Diffusion Transformer (DiT) models. The core problem addressed is the high computational cost of these models. Unlike existing methods that rely on local, inter-step similarity to decide which computations to cache (often focusing on later sampling steps), OmniCache takes a global perspective. It analyzes the entire sampling trajectory of the diffusion process and strategically distributes cache reuse based on the trajectory's geometric properties (curvature). Additionally, it introduces a mechanism to dynamically estimate and filter the noise introduced by caching, thereby preserving generation quality. The authors claim significant speedups on various DiT models with competitive performance.
• Original Source Link: The paper is available at https://arxiv.org/abs/2508.16212 (PDF: http://arxiv.org/pdf/2508.16212v2). Note: The arXiv ID format appears non-standard, but the analysis proceeds based on the provided text.

2. Executive Summary

• Background & Motivation (Why):

  • Core Problem: Diffusion models, especially those using a Transformer backbone (DiTs), have achieved state-of-the-art results in generative tasks like video synthesis. However, their iterative sampling process is computationally intensive and slow, posing a major barrier to real-time applications.
  • Existing Gaps: Current acceleration techniques that use caching often make decisions based on local similarity between adjacent time steps. Since similarity is higher in the later stages of denoising, these methods tend to concentrate cache reuse there. The authors argue this is a flawed strategy. As shown in Figure 1, late-stage caching introduces errors that the model has little opportunity to correct, leading to "irrecoverable trajectory deviations" and a drop in quality (Signal-to-Noise Ratio).
  • Innovation: OmniCache proposes a fundamentally different approach. Instead of a local view, it adopts a global, trajectory-oriented perspective. It analyzes the entire geometric path the model takes from noise to a clean sample. This allows it to identify the most stable parts of the trajectory for caching, regardless of whether they are early or late in the process. This global strategy is combined with an explicit noise correction mechanism to mitigate the negative effects of caching.

• Main Contributions / Findings (What):

  1. Trajectory-Oriented Caching Strategy: The paper pioneers a method that determines where to apply cache reuse by analyzing the curvature of the diffusion sampling trajectory. It caches computations in low-curvature regions (stable, straight-line progress) and avoids caching in high-curvature regions (critical turns).
  2. Dynamic Noise Correction: A novel module is introduced to estimate the noise introduced by the caching process. It leverages the high correlation of this noise between adjacent steps and applies frequency-based filtering (low-pass in early stages, high-pass in later stages) to correct the model's output without damaging important signal components.
  3. State-of-the-Art Performance: OmniCache achieves a 2-2.5x speedup on powerful video generation models like OpenSora and Latte with almost no loss in generation quality. Critically, it also successfully accelerates a highly optimized, distilled model (CogVideoX-5b-I2V-distill) by 1.45x where other caching methods fail and cause model collapse.

3. Prerequisite Knowledge & Related Work

• Foundational Concepts:

  • Diffusion Models (DDPMs): These are generative models that learn to create data by reversing a noise-adding process. The forward process gradually adds Gaussian noise to an image until it becomes pure noise. The reverse process learns to denoise it step-by-step, starting from random noise to generate a new image. This iterative denoising requires many steps, making it slow.
  • Diffusion Transformers (DiTs): Instead of the commonly used U-Net architecture, DiTs use a Transformer as the backbone for the denoising network. Transformers, known for their success in natural language processing, have proven to be highly scalable and effective for diffusion models, leading to state-of-the-art performance in image and video generation. A DiT block typically consists of a self-attention module and an MLP (Multi-Layer Perceptron) module.
  • Cache Reuse: In iterative processes like diffusion sampling, the inputs to the model at adjacent steps ($t$ and t-1) can be very similar. Cache reuse exploits this by storing intermediate computations (e.g., the output of an attention or MLP block) from step $t$ and reusing them at step t-1, skipping the actual computation to save time.

• Previous Works:

  • Efficient Process Strategies (EPS): These methods aim to reduce the number of sampling steps required. Examples include knowledge distillation (training a smaller model to mimic a larger one in fewer steps) and developing more efficient differential equation solvers (DPM-Solver, DDIM).
  • Efficient Design Strategies (EDS): These methods focus on making the model itself more efficient. This includes model compression techniques like network pruning, quantization, or using a latent space (Latent Diffusion Models).
  • Existing Cache Mechanisms: Several methods have applied caching to diffusion models.
    • DeepCache: Caches feature maps in the U-Net's upsampling blocks.
    • Δ-DIT: Stores the residuals (differences) between blocks to accelerate computation.
    • AdaCache, TOCA, TGATE: These methods are more recent and focus on DiTs. They typically use similarity metrics to decide when and what to cache, often focusing on later sampling stages where outputs are more similar.

• Differentiation: OmniCache distinguishes itself from prior work in two key ways:

  1. Global vs. Local Strategy: While methods like AdaCache use local similarity between step outputs to make caching decisions, OmniCache uses a global property of the entire sampling process—the trajectory curvature. This prevents it from falling into the trap of caching too aggressively in the late stages where errors are most damaging.
  2. Proactive Noise Correction: OmniCache is the first method to systematically model, estimate, and correct the noise artifacts introduced by caching. Other methods implicitly rely on the diffusion model's inherent robustness to handle these errors, which, as the paper shows, is insufficient, especially in later steps or in distilled models.

4. Methodology (Core Technology & Implementation)

The core of OmniCache is built on analyzing the sampling trajectory and correcting the errors introduced by caching.

Figure . The diagram of our OmniCache. In the calibration stage, we store the states at different time steps $\boldsymbol { x } _ { t _ { n } }$ to obtain the corresponding time and correct the denoisng mode's output bason the noise coelation and high-pass/low-pas fteing.

The diagram above (labeled as Figure 2 in the paper) illustrates the two main stages of OmniCache.

• (a) Calibration Stage: This is an offline analysis step. The model is run on sample inputs to generate a typical sampling trajectory. This trajectory's curvature is analyzed to determine the optimal steps for cache reuse (the reuse set). The correlation of cache-induced noise between steps is also computed.
• (b) Inference Stage: During actual generation, when a step is part of the reuse set, computation is skipped. The model estimates the noise that would have been introduced and subtracts a filtered version of it from the output, correcting the sampling direction.

4.1. Preliminaries of Diffusion and Caching

The standard DDPM sampling step to get from a noisy sample $x_t$ to a slightly less noisy sample $x_{t-1}$ is given by: $$x _ { t - 1 } = \frac { 1 } { \sqrt { \alpha _ { t } } } \left( x _ { t } - \frac { \beta _ { t } } { \sqrt { 1 - \overline { { \alpha _ { t } } } } } \epsilon _ { \theta } ( x _ { t } , t ) \right) + \sigma _ { t } z , \quad z \in N ( 0 , I )$$ where:

• $x_t$ is the noisy input at timestep $t$.

• $\epsilon_{\theta}(x_t, t)$ is the noise predicted by the DiT model.

• $\alpha_t$, $\beta_t$, $\sigma_t$ are pre-defined noise schedule constants.

• $z$ is random Gaussian noise.

  When cache reuse is applied, for example, to an attention block, the computation is replaced: $$\widetilde { f } _ { t } ^ { l } = \left\{ \begin{array} { l l } { f _ { t } ^ { l } + A t t e n ( f _ { t } ^ { l } ) } & { \text{Original Forward} } \\ { f _ { t } ^ { l } + A t t e n C a c h e } & { \text{Cache Reuse} } \end{array} \right.$$ This introduces an error, causing the model to predict a slightly incorrect noise $\widetilde{\epsilon}_{\theta}(x_t, t)$. The difference between the true and cached prediction is the cache-induced noise, $q_{\theta}$: $$\widetilde{\epsilon}_{\theta}(x_t, t) = \epsilon_{\theta}(x_t, t) + q_{\theta}(x_t, t)$$ This noise accumulates and can derail the entire sampling process.

4.2. Sampling Trajectory Analysis

The key insight of OmniCache is that the path from pure noise ($x_T$) to a clean image ($x_0$) has a consistent geometric shape.

• Trajectory Visualization: The high-dimensional trajectory is projected into a 3D subspace for analysis using Principal Component Analysis (PCA). The primary axis $u$ connects the start and end points, and two additional orthogonal axes ($w_1, w_2$) capture the main deviation from a straight line. $$x _ { t _ { i } } ^ { \mathrm { p r o j } } = ( x _ { t _ { i } } \cdot u ) u + ( x _ { t _ { i } } \cdot w _ { 1 } ) w _ { 1 } + ( x _ { t _ { i } } \cdot w _ { 2 } ) w _ { 2 }$$

• Trajectory Shape & Properties: The paper observes that these trajectories consistently exhibit a "boomerang shape", independent of the specific content being generated.

  该图像是图1，展示了扩散模型中的采样过程与信噪比（SNR）变化。图(a)为几何采样模型，描绘了从噪声到数据流形的采样轨迹，并在不同时间步 $x_{t_n}$ 和 $x_{t_m}$ 处进行缓存复用。早期复用有助于校正噪声偏差，而后期复用可能导致不可逆的轨迹偏离。图(b)显示了采样过程中不同缓存复用策略下的真实SNR曲线，表明现有方法在后期采样步长中会导致SNR持续下降，而本文方法（Ours）能更好地维持SNR。

  该图像是图3，展示了CogVideoX模型16步去噪过程的采样轨迹及其缓存复用效果。图a和图b分别可视化了在早期($x_{t_{11}}$)和后期($x_{t_3}$)步骤应用缓存复用时，模型在三维空间中的采样轨迹，并对比了相应的中间输出和最终生成图像$x_{t_0}$，红框突出显示细节区域。图c则以热力图形式呈现了相邻步骤间输出噪声的相对L2范数，颜色深浅（约0到0.14）反映了误差大小。该图揭示了缓存复用对采样轨迹和生成质量的影响。

• Impact of Cache Reuse Timing: As visualized in Figure 3, applying cache reuse has dramatically different effects depending on the timing:

  • Early Stage (Fig. 3a): The input $x_{t_{12}}$ is mostly noise. The error introduced by caching is small relative to the existing noise. Subsequent denoising steps have ample opportunity to correct this deviation, acting as a self-correction mechanism.
  • Late Stage (Fig. 3b): The input $x_{t_4}$ is already quite clean. The cache-induced noise is significant relative to the remaining noise, causing an irreversible deviation. The final image shows quality degradation (ripple artifacts).
  • The Similarity Trap: Figure 3c shows that the similarity between adjacent steps (measured by the L2 norm of output differences) is higher in the late stages. This explains why similarity-based methods are drawn to caching late, which is precisely the wrong choice from a global trajectory perspective.

• Curvature-Based Cache Selection: Based on this analysis, OmniCache proposes to use trajectory curvature as the criterion for caching.

  • Low Curvature: Indicates the trajectory is moving in a stable, predictable direction. Caching here is safe as it's unlikely to cause a major directional change.
  • High Curvature: Indicates a critical "turn" in the sampling path. Caching here is risky and should be avoided.

4.3. Cache-Induced Noise Correction

OmniCache actively corrects for the caching error $q_{\theta}$. The authors find that the noise $q_{\theta}$ at step t-1 is highly correlated with the noise at the previous step $t$. Using a first-order Taylor expansion, they approximate this relationship: $$q _ { \theta } ( x _ { t - 1 } , t - 1 ) \approx \gamma _ { t - 1 } q _ { \theta } ( x _ { t } , t )$$ where $\gamma_{t-1}$ is a correlation coefficient. This means the noise at the next step can be estimated from the noise at the current step.

The quality of this approximation depends on the second term of the Taylor expansion, which is shown to be related to the trajectory's second derivative (i.e., curvature): $$- \frac { d q _ { \theta } ( x _ { t } , t ) } { d t } \approx \frac { \sqrt { \alpha _ { t } - \alpha _ { t } \overline { { \alpha } } _ { t } } } { \beta _ { t } } \frac { d ^ { 2 } x _ { t } } { d t ^ { 2 } }$$ This elegantly connects the two core ideas: selecting low-curvature points for caching not only minimizes trajectory deviation but also makes the cache-induced noise more predictable and correctable.

• Frequency Filtering: Recognizing that diffusion models focus on low-frequency (global structure) information in early stages and high-frequency (fine details) in late stages, the estimated noise is filtered before being subtracted:
  • Early Stages: A low-pass filter is applied to the noise estimate to avoid corrupting high-frequency details that haven't formed yet.
  • Late Stages: A high-pass filter is applied to avoid damaging the already-established low-frequency structure.

5. Experimental Setup

• Datasets:

  • ImageNet: Used for class-conditional image generation with DiT-XL. 50,000 samples were generated for evaluation.
  • VBench: A comprehensive benchmark for text-to-video generation. Used with OpenSora and Latte. 4,750 videos were generated across 950 prompts.
  • Custom Set: 100 randomly generated videos for the image-to-video task with CogVideoX-5b-I2V-distill.

• Evaluation Metrics:

  1. FID (Fréchet Inception Distance): Measures the similarity between the distribution of generated images and real images. Lower is better.
     • Conceptual Definition: It calculates the distance between two multivariate Gaussian distributions fitted to the feature representations (from an InceptionV3 network) of real and generated images. It captures both quality and diversity.
     • Formula: $FID(x, g) = ||\mu_x - \mu_g||^2 + \text{Tr}(\Sigma_x + \Sigma_g - 2(\Sigma_x \Sigma_g)^{1/2})$
     • Symbols: $\mu_x, \mu_g$ are the mean feature vectors for real and generated images. $\Sigma_x, \Sigma_g$ are their covariance matrices. $\text{Tr}$ is the trace of a matrix.
  2. Precision and Recall: Measures the fidelity (quality) and diversity of generated samples, respectively. Higher is better for both.
  3. VBench Score: An aggregate score from the VBench framework, which evaluates video generation across 16 different dimensions (e.g., temporal consistency, object permanence, motion quality). Higher is better.
  4. PSNR (Peak Signal-to-Noise Ratio): Measures the quality of a reconstructed image/video compared to an original. Higher is better.
     • Formula: $PSNR = 20 \log_{10}(\text{MAX}_I) - 10 \log_{10}(\text{MSE})$
     • Symbols: $\text{MAX}_I$ is the maximum possible pixel value (e.g., 255). MSE is the Mean Squared Error between the original and generated video frames.
  5. SSIM (Structural Similarity Index Measure): Measures perceived image quality based on luminance, contrast, and structure. Ranges from -1 to 1, where 1 is a perfect match. Higher is better.
  6. LPIPS (Learned Perceptual Image Patch Similarity): Measures the perceptual distance between two images using features from a deep neural network. Lower is better.
  7. Q-Align: A metric designed for image-to-video generation that measures the quality and alignment of the generated video with the input image. Higher is better.

• Baselines: The paper compares OmniCache against several state-of-the-art training-free acceleration methods, including FORA, Δ-DiT, T-GATE, PAB, AdaCache, TeaCache, and ToCA.

6. Results & Analysis

6.1. Core Results

• Text-to-Video Generation (Table 1): This table shows results for OpenSora and Latte. OmniCache-slow (50% steps cached) and OmniCache-fast (60% steps cached) are presented.

  • On OpenSora, OmniCache-slow achieves a 2.00x speedup with a negligible VBench score drop (79.22 to 78.83). OmniCache-fast achieves a 2.50x speedup while still outperforming most baselines in quality.

  • On Latte, OmniCache-slow gets a 2.00x speedup with a VBench drop from 77.40 to 77.24, which is excellent. OmniCache-fast provides a 2.50x speedup with a score of 77.09, demonstrating a strong trade-off.

    This is a manual transcription of the data from Table 1 in the paper.

    Method	VBench (%) ↑	PSNR ↑	LPIPS ↓	SSIM ↑	FLOPs (T) Speedup
    Open-Sora	79.22				3230.24	1.00×
    + OmniCache-slow	78.83	22.37	0.1553	0.8180	1615.12	2.00×
    + OmniCache-fast	78.50	21.27	0.1841	0.7930	1292.10	2.50×
    Latte	77.40	−	−		3439.47	1.00×
    + OmniCache-slow	77.24	22.48	0.1955	0.7903	1719.74	2.00×
    + OmniCache-fast	77.09	21.06	0.2463	0.7575	1375.79	2.50×

• Image-to-Video Generation (Table 2): This experiment on the distilled CogVideoX-5b-I2V-distill model is a stress test. Distilled models have very little redundancy, making them hard to accelerate with caching.

  • Other methods reportedly lead to "model collapse."

  • AdaCache (adapted for this model) achieves a 1.33x speedup but sees a drop in the Q-Align score (0.79 to 0.77).

  • OmniCache achieves a higher 1.45x speedup while maintaining the Q-Align score (0.792 vs. 0.79), indicating nearly lossless acceleration.

    This is a manual transcription of the data from Table 2 in the paper.

    Method	Aesthetic Quality ↑	Q-Align↑	Speedup
    CogVideoX-5b-I2V-distill	0.59	0.79	1.00×
    +AdaCache*	0.57	0.77	1.33×
    +OmniCache	0.621	0.792	1.45×

    该图像是对比不同方法生成图像质量的示意图。它展示了Baseline、AdaCache和OmniCache三种方法在生成卡通角色、炸薯条、皮划艇运动场景和动漫人物序列时的表现。其中，AdaCache在生成炸薯条时出现明显伪影（红框所示），而OmniCache方法生成的图像质量与Baseline接近，且未出现类似伪影，突显了OmniCache在维持生成质量方面的优势。

The visual results in the figure above (labeled as Figure 4) corroborate this. AdaCache introduces blurry artifacts (red box), while OmniCache's quality is visually indistinguishable from the baseline.

• Class-Conditional Image Generation (Table 3): On the DiT-XL/2 model with 250 steps, the task has high redundancy. OmniCache achieves a 2.50x speedup with FID scores (5.86) comparable to the state-of-the-art method ToCa (5.81), demonstrating its competitiveness even in high-redundancy scenarios.

  This is a manual transcription of the data from Table 3 in the paper.

  Method	FLOPs(T)	Speed	sFID↓	FID↓	Precision↑	Recall↑
  DiT-XL/2-G (cfg = 1.50)	118.68	1.00x	4.98	2.31	0.82	0.58
  +ToCa (N = 4, R = 93%)	43.22	2.75x	5.81	2.68	0.80	0.57
  +OmniCache-fast	47.47	2.50x	5.86	2.69	0.81	0.56

6.2. Ablation Study (Table 4)

This study, conducted on the challenging CogVideoX-5b-I2V-distill model, systematically validates each component of OmniCache. All variants accelerate 5 out of 16 steps (1.45x speedup).

1. Cache Reuse Only: Applying the trajectory-based caching strategy without any correction drops the Q-Align score from 0.79 to 0.778.

2. + Noise Correction: Adding the noise estimation and correction module brings the score up to 0.788, recovering most of the quality loss.

3. + Noise Correction/Filtering: Adding the frequency-based filtering on top of correction fully restores and even slightly improves performance (Q-Align of 0.792), confirming the effectiveness of each proposed component.

   This is a manual transcription of the data from Table 4 in the paper.

   Method	Aesthetic Quality↑	Q-Align↑
   CogVideoX-5b-I2V-distill	0.59	0.79
   + OmniCache (Cache Reuse)	0.58	0.778
   + OmniCache (Cache Reuse + Noise Correct)	0.593	0.788
   + OmniCache (Full)	0.621	0.792

6.3. Qualitative Visualizations

该图像是图5，展示了使用OpenSora V1.2模型生成的视频的第一帧可视化结果。图像由8行场景组成，每行包含4幅不同条件下的生成图像，可能对应OmniCache方法的不同加速倍数（如2倍、2.5倍）。这些图像涵盖了酒吧、卧室、笔记本电脑、长凳、咖啡杯、电话亭、游泳池和停车场等多样化场景，旨在评估OmniCache在加速采样过程同时保持生成质量的能力。

该图像是对比图，展示了扩散Transformer模型中TeaCache-fast、OmniCache-slow和OmniCache-fast三种加速方法生成的图像与原始图像的视觉质量对比。它通过七组不同场景的图像直观地呈现了OmniCache在加速采样过程的同时，保持了与原始图像相近的生成质量。

Figures 5 and 6 show visual comparisons against TeaCache on OpenSora and Latte, respectively. In both cases, the competing method introduces noticeable artifacts or distortions (e.g., malformed objects), while OmniCache's outputs remain visually faithful to the original, non-accelerated model, even at a 2.5x speedup.

7. Conclusion & Reflections

• Conclusion Summary: OmniCache presents a powerful, training-free method for accelerating Diffusion Transformer models. By shifting the paradigm from local similarity to a global analysis of the sampling trajectory's geometry, it makes more intelligent decisions about when to cache computations. This is complemented by a sophisticated noise estimation and correction mechanism that actively counteracts the negative side effects of caching. The method achieves significant speedups while maintaining high generation quality, even on highly efficient distilled models where previous methods fail.

• Limitations & Future Work: The authors acknowledge one primary limitation: to ensure the noise estimation is reliable, the method is constrained to not perform cache reuse for three or more consecutive steps. This constraint may cap the maximum achievable acceleration. Future work could explore more robust multi-step noise estimation techniques to overcome this.

• Personal Insights & Critique:

  • Novelty and Impact: The core idea of using trajectory curvature is both elegant and highly effective. It provides a principled reason for why and when caching is safe, moving beyond simple heuristics like similarity. This global perspective is a significant contribution to the field of efficient generative model inference.
  • Practicality: The method consists of a one-time, offline calibration stage to analyze the trajectory and a lightweight inference stage. While the calibration adds an initial overhead, it does not need to be repeated for every generation. The paper implies the trajectory shape is inherent to the model architecture, meaning one calibration could serve for many tasks. However, the sensitivity of this calibration to different domains or prompt types is an open question.
  • Robustness: The most compelling evidence for OmniCache's effectiveness is its success on the distilled CogVideoX model. This shows that the method is not just exploiting easy-to-remove redundancy but can operate effectively in low-redundancy regimes, highlighting the power of the noise correction module.
  • Future Directions: The concept of analyzing the global dynamics of the generation process could be extended beyond caching. It might inform better sampling schedules, adaptive model architectures, or more efficient distillation strategies. The frequency-based filtering of noise is also a clever insight that could be applied in other areas of generative modeling.