1. Bibliographic Information

1.1. Title

ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions

1.2. Authors

The authors are Xiaoxue Wu (affiliated with Fudan University and Shanghai Artificial Intelligence Laboratory), Xinyuan Chen (Shanghai Artificial Intelligence Laboratory), Yaohui Wang (Shanghai Artificial Intelligence Laboratory), and Yu Qiao (Shanghai Artificial Intelligence Laboratory).

1.3. Journal/Conference

This paper was published on arXiv, indicating it is a preprint. Given the publication date in late 2025, it is likely submitted or intended for a major computer vision or machine learning conference (e.g., CVPR, ICCV, NeurIPS, ICLR), which are highly reputable venues in the field.

1.4. Publication Year

2025

1.5. Abstract

The paper addresses a critical gap in multi-shot video generation: the lack of intentional design and cinematographic language in shot transitions, which often results in mere sequential changes rather than coherent narrative expression. To overcome this, the authors propose ShotDirector, an efficient framework integrating parameter-level camera control and hierarchical editing-pattern-aware prompting. Specifically, it uses a camera control module with 6-DoF poses and intrinsic settings for precise camera information injection. Additionally, a shot-aware mask mechanism is introduced to guide hierarchical prompts that are aware of professional editing patterns, enabling fine-grained control over shot content. This design effectively combines low-level parameter conditions with high-level semantic guidance to achieve film-like controllable shot transitions. To support this, the authors constructed ShotWeaver40K, a dataset capturing film-like editing patterns, and developed new evaluation metrics. Extensive experiments demonstrate the framework's effectiveness.

1.6. Original Source Link

https://arxiv.org/abs/2512.10286 The publication status is a preprint on arXiv.

1.7. PDF Link

https://arxiv.org/pdf/2512.10286v1.pdf The publication status is a preprint on arXiv.

2. Executive Summary

2.1. Background & Motivation

The core problem the paper aims to solve is the limitation of current multi-shot video generation models in creating cinematographically meaningful and directorially controllable shot transitions. While recent advances in diffusion-based video generation have made strides in synthesizing realistic single-shot videos and even multi-shot sequences, they largely focus on low-level visual consistency and treat shot transitions as simple frame-level changes. This often leads to generated multi-shot videos that are sequential but lack intentional film-editing patterns (like cut-in, cut-out, shot/reverse-shot, multi-angle) and coherent narrative expression.

This problem is important because shot transitions are a fundamental directorial tool in visual storytelling. They determine the overall narrative rhythm, guide audience perception, and convey cinematic language. Without explicit modeling of how transitions are designed and controlled, generated multi-shot videos remain "collections of single-shot clips" rather than cohesive, film-like narratives with intentional shot design. The challenge lies in enabling generative models to understand and execute these high-level cinematographic conventions while maintaining visual fidelity and consistency.

The paper's entry point is to reinterpret a cut not just as an abrupt visual break, but as a deliberate directorial choice. It seeks to integrate directorial decision-making into the generative process, allowing explicit control over how the next shot unfolds, beyond just visual coherence.

2.2. Main Contributions / Findings

The primary contributions of this paper are:

• A Novel Framework (ShotDirector): Proposing ShotDirector, an efficient framework that integrates parameter-level camera control and hierarchical editing-pattern-aware prompting for multi-shot video generation. This framework explicitly models cinematographic language and directorial design in shot transitions.

• Precise Camera Control Module: Developing a camera control module that incorporates 6-DoF poses (six degrees of freedom, referring to camera position and orientation) and intrinsic settings to enable precise injection of camera information into the generative process. This allows for refined control over viewpoint shifts and reduces discontinuities.

• Shot-aware Mask Mechanism: Introducing a shot-aware mask mechanism that guides hierarchical prompts. This mechanism, aware of professional editing patterns, provides fine-grained control over shot content by balancing global coherence with shot-specific diversity through structured token interactions.

• New Dataset (ShotWeaver40K): Constructing ShotWeaver40K, a high-quality multi-shot video dataset. This dataset is curated to capture the priors of film-like editing patterns and includes cinematography-aware captions and detailed camera parameters, facilitating the training of models that understand professional editing conventions.

• Comprehensive Evaluation Metrics: Developing a set of evaluation metrics designed specifically for controllable multi-shot video generation, encompassing controllability, consistency, and visual quality.

  Key conclusions and findings reached by the paper include:

• ShotDirector effectively combines parameter-level conditions with high-level semantic guidance, producing multi-shot videos that exhibit film-like cinematographic expression and coherent narrative flow.

• The framework achieves controllable shot transitions that align with specified editing patterns (e.g., cut-in, cut-out, shot/reverse-shot, multi-angle), outperforming existing baselines in terms of transition control accuracy.

• The constructed ShotWeaver40K dataset proves effective in training models to internalize shot transition design priors.

• Ablation studies confirm the positive contributions of both the camera information injection (Plücker and extrinsic branches) and the shot-aware mask mechanism (visual and semantic masks) to the model's performance, particularly in transition control and consistency.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To understand ShotDirector, a novice should be familiar with the following foundational concepts:

• Diffusion Models: These are a class of generative models that learn to reverse a diffusion process. They start with random noise and gradually refine it to generate data (like images or videos) that resembles real data. The process typically involves denoising a noisy input conditioned on some information (e.g., text, camera parameters). Key to their success are denoising diffusion probabilistic models (DDPMs) and latent diffusion models (LDMs), which perform this process in a lower-dimensional latent space for efficiency.
• Diffusion Transformer (DiT): A specific architecture for diffusion models, where the U-Net backbone (common in many diffusion models) is replaced by a Transformer architecture. Transformers are neural networks originally designed for sequence processing (like natural language) but have been adapted for image and video generation. DiT models treat image/video patches as tokens and use self-attention mechanisms to process them, enabling better long-range dependency modeling. $ \mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V $ Here, $Q$ represents the query matrix, $K$ the key matrix, and $V$ the value matrix. These are derived from the input embeddings. $d_k$ is the dimension of the key vectors, used for scaling to prevent vanishing gradients. The softmax function is applied row-wise to the result of $QK^T$ (the dot product attention scores) to obtain attention weights, which are then multiplied by $V$ to produce the output. This mechanism allows each token to "attend" to other tokens, dynamically weighing their importance based on their relevance.
• Camera Control (6-DoF poses, Intrinsics, Extrinsics):
  • 6-Degrees of Freedom (6-DoF): Refers to the six independent parameters that define the position and orientation of a rigid body in 3D space. For a camera, this means its (x, y, z) position in space and its roll, pitch, yaw (rotation around its axes).
  • Intrinsic Parameters ($K$): These describe the internal properties of the camera that project 3D points onto a 2D image plane. They include focal lengths ($f_x, f_y$), principal point ($c_x, c_y$ which is the image center), and skew coefficient. Often represented as a $3 \times 3$ matrix: $ K = \begin{pmatrix} f_x & s & c_x \ 0 & f_y & c_y \ 0 & 0 & 1 \end{pmatrix} $ where $s$ is the skew coefficient, often assumed to be 0.
  • Extrinsic Parameters ($E = [R | t]$): These describe the camera's position and orientation (the 6-DoF pose) relative to a world coordinate system. They consist of a rotation matrix ($R \in \mathbb{R}^{3 \times 3}$) and a translation vector ($t \in \mathbb{R}^{3 \times 1}$).
• Plücker Embedding: A mathematical representation used to encode 3D lines (or rays, in this context). In computer vision, it's used to represent the viewing ray from a camera's optical center through each pixel. Each ray is represented by a 6-dimensional vector combining its moment (cross product of origin and direction) and direction vector. This embedding provides a rich geometric representation that captures both the camera's position and the direction of light rays from the scene.
• Multi-Shot Video Generation: The task of generating a video composed of multiple distinct "shots," where each shot is a continuous sequence of frames captured from a single camera position or motion, and shots are separated by shot transitions (cuts).
• Cinematographic Editing Patterns: Standard conventions and techniques used in filmmaking to transition between shots, conveying narrative, emotion, and rhythm. The paper focuses on four types:
  • Cut-in: A transition from a wider shot to a closer detail of the same subject or scene, often to emphasize something.
  • Cut-out: A transition from a close-up or detail shot to a wider, contextual view of the same subject or scene.
  • Shot/Reverse Shot: An editing pattern used in dialogue scenes, alternating between shots of two characters (or a character and what they are looking at) from opposing camera angles.
  • Multi-Angle: Switching between different viewpoints of the same action or subject within the same scene, providing different perspectives without necessarily changing shot scale dramatically.

3.2. Previous Works

The paper categorizes previous multi-shot video generation approaches into two main types:

• Stitching-based approaches: (e.g., StoryDiffusion [53], VideoStudio [29], VGoT [52], Phantom [28]) These methods synthesize individual shots independently and then concatenate them. They emphasize cross-shot consistency through external constraints like keyframe alignment or identity-preserving embeddings. The limitation is that inter-shot dependencies are enforced artificially rather than learned from cinematic priors, leading to sequences that feel like "collections of single-shot clips" without genuine narrative continuity. Cut2Next [16] considers editing patterns but is image-level, limiting its applicability to complex video.
• End-to-end diffusion frameworks: (e.g., Mask2DiT [33], CineTrans [46], LCT [13], MoGA [21], TTT [9]) These models allow for interaction among different shots within the generative process itself, yielding higher visual and temporal consistency. However, they typically treat shot transitions as mere frame-level changes, lacking explicit modeling of editing conventions or controllable transition dynamics. They produce high-quality multi-shot sequences but often lack cinematic narrative structure.

Camera-Controlled Video Generation: This area focuses on explicitly controlling camera motion during video synthesis.

• Early approaches ([10, 14]) injected camera parameters into pre-trained video diffusion models.
• Subsequent works ([2, 3, 19, 49]) incorporated geometric constraints or 3D priors.
• Multi-camera setups were explored by SynCamMaster [5] (synchronized multi-camera generation) and ReCamMaster [4] (3D-consistent scene modeling).
• CameraCtr1II [15] aimed for enhanced temporal consistency and continuous camera motion.

3.3. Technological Evolution

Video generation has evolved rapidly, driven by advances in deep learning, particularly diffusion models. Initially, research focused on generating single, short, coherent videos from text (text-to-video models like Sora, Stable Video Diffusion, HunyuanVideo, Wan2.2). These models achieved remarkable photorealism and temporal fidelity.

The natural next step was multi-shot video generation to create longer, narrative-driven content. Initial attempts were often stitching-based, generating shots separately and then combining them, which struggled with seamless transitions and narrative flow. The field then moved towards end-to-end generation, where models learned to generate multiple shots within a single generative process, improving consistency.

However, even these end-to-end models often treated transitions as generic visual changes, missing the directorial intent behind cinematic cuts. Concurrently, camera-controlled generation emerged, allowing users to specify camera movements (zoom, pan, tilt, orbit) within a single continuous shot.

ShotDirector fits into this evolution by combining the strengths of end-to-end multi-shot generation with fine-grained camera control and, crucially, integrating an understanding of high-level cinematographic editing patterns. This bridges the gap between purely visual generation and narrative-driven, directorially informed video synthesis.

3.4. Differentiation Analysis

Compared to existing methods, ShotDirector introduces several core differentiators:

• Explicit Modeling of Cinematographic Transitions: Unlike most end-to-end multi-shot models that treat transitions as mere frame changes (Mask2DiT, CineTrans), ShotDirector explicitly models and controls specific editing patterns (e.g., cut-in, shot/reverse-shot). This moves beyond low-level visual consistency to high-level narrative coherence.
• Dual-Perspective Control: It combines parameter-level camera control (6-DoF poses, intrinsics) with semantic-level hierarchical prompting. Previous camera-controlled methods (SynCamMaster, ReCamMaster) often focused on continuous camera motion or multi-view synthesis without explicit shot transition type control, or struggled with abrupt cuts. ShotDirector leverages precise camera parameters to inform the discrete nature of shot changes.
• Editing-Pattern-Aware Prompting: The shot-aware mask mechanism allows for hierarchical text guidance, integrating global narrative context with shot-specific cinematographic cues and explicit transition types. This is more sophisticated than general semantic-driven prompts or visual masks used in prior work.
• Specialized Dataset: The ShotWeaver40K dataset, with its focus on "film-like editing patterns" and detailed cinematography-aware captions and camera parameters, is specifically designed to train models in understanding and executing professional shot transitions, which is a unique contribution compared to generic video datasets.
• Balanced Consistency and Controllability: While some methods achieve consistency at the cost of visual fidelity (SynCamMaster) or lack explicit transition control, ShotDirector aims for a balance, maintaining high visual quality and cross-shot consistency while offering fine-grained control over transition types and camera parameters.

4. Methodology

4.1. Principles

The core idea behind ShotDirector is to enable directorially controllable multi-shot video generation by treating shot transitions as fundamental directorial tools rather than just visual breaks. This is achieved by combining two complementary control mechanisms: parameter-level camera settings and semantic-level hierarchical prompting. The underlying principle is that a generative model, specifically a diffusion model, can learn to produce film-like narratives if it's explicitly conditioned on both the precise geometric information of camera movements and high-level textual guidance that encodes professional editing patterns. This allows the model to understand how and why a shot transition should occur, ensuring narrative coherence and artistic expression.

4.2. Core Methodology In-depth (Layer by Layer)

The ShotDirector framework integrates precise camera control and editing-pattern-aware prompting into a diffusion model, building upon architectures like DiT (Diffusion Transformer) or latent diffusion frameworks. The overall architecture is illustrated in Figure 3b (from the original paper).

Figure 3b (from the original paper): The architecture of our method. We adopt a dual-branch design to inject camera information and employ a shot-aware mask mechanism to regulate token visibility across global and local contexts. Professional transition design is incorporated into global text tokens, working with camera information to achieve multi-granular control over shot transitions.

4.2.1. ShotWeaver40K Dataset Construction

To train the model to understand and generate film-like shot transitions, a high-quality dataset named ShotWeaver40K is constructed through a refined data processing pipeline, as shown in Figure 3a (from the original paper).

Figure 3a (from the original paper): Dataset curation pipeline. Starting from a raw movie dataset, we apply Seg-and-Stitch and filtering to obtain a refined video set, annotated with hierarchical captions and transition camera poses to form ShotWeaver40K.

The data collection process involves several steps:

1. Raw Video Source: The process begins with a large corpus of 16K full-length films. This ensures a rich source of cinematography language and narrative flow.

2. Segmentation and Stitching:

   • Each video is first segmented into individual shots using TransNetV2 [38]. TransNetV2 is an effective deep network architecture for fast shot transition detection.

   • ImageBind [12] is then used to extract image features for adjacent shots. ImageBind is a model that learns a single embedding space for various modalities (like images, text, audio), allowing for cross-modal similarity calculations.

   • Similar segments are stitched together to form coherent multi-shot clips. A clip is discarded if the similarity between its first and last frames falls below a predefined threshold. This ensures that the stitched clips maintain some visual or semantic connection across the shot boundary.

   • The thresholds used during this stage are detailed in Table 4 from the supplementary material:

     The following are the results from Table 4 of the original paper:

     Threshold Type	Value
     Segmentation threshold	0.45
     First/last frame similarity threshold	0.90
     Stitching threshold	0.65

     • Segmentation threshold: A value of 0.45 for TransNetV2 likely determines the sensitivity for detecting shot changes.
     • First/last frame similarity threshold: A value of 0.90 means that if the similarity between the first frame of a clip and its last frame (which is the last frame of the second shot in a two-shot clip) is below 0.90 (as measured by ImageBind features), the clip is discarded. This helps ensure some level of visual consistency or flow across the entire multi-shot sequence.
     • Stitching threshold: A value of 0.65 for ImageBind feature similarity is used to stitch similar segments, ensuring that segments being joined are sufficiently related.

3. Preliminary/Coarse Filtering:

   • Initial multi-shot clips undergo filtering based on fundamental video attributes:
     • Resolution: Ensuring a minimum quality standard.
     • Frame Rate (FPS): Standardizing temporal density.
     • Temporal Duration: Clips are constrained to be between 5-12 seconds. The paper focuses on clips containing exactly two shots to study shot transitions effectively.
     • Overall Aesthetic Quality: An aesthetic predictor is used, with particular attention to frames near shot boundaries to prevent visually unclear transitions.
   • This stage yields approximately 500K candidate videos.

4. Fine-Grained Transition Filtering:

   • This step addresses both overly similar and excessively dissimilar shots.

   • Overly similar: CLIP [34] feature similarity is computed, and pairs with similarity greater than 0.95 are removed. This filters out incorrect segmentations or "no-transition" scenes (e.g., mere light flickering) that don't represent genuine narrative cuts. CLIP (Contrastive Language-Image Pre-training) is a neural network trained on a wide variety of image-text pairs, capable of understanding image content in relation to text.

   • Low similarity: For cases where image-feature-based metrics (like ImageBind or CLIP) might not capture causal or spatial relationships well (as they tend to focus on vibe/style), a VLM-based (Vision-Language Model) method [43] is employed. Qwen [43] is a powerful VLM capable of understanding complex visual and textual prompts. This VLM is prompted to assess if the two shots are different scenes or the same scene and to evaluate quality. Videos lacking clear transition logic are filtered out to prevent confusion for the model. An example of the VLM filtering prompt is provided in Figure 7 (from the supplementary material):

     Figure 7 (from the original paper): Prompt for VLM-based filtering.

     The VLM is asked to classify the quality and scene relationship (e.g., "High-quality, Different Scene" or "Low-quality, Same Scene") between two consecutive frames, helping to ensure that the transition logic is intelligible.

5. Caption Generation:

   • GPT-5-mini (a large language model) is used to generate hierarchical text annotations for each curated multi-shot video.

   • These captions cover:

     • Subject: Main characters across both shots.
     • Overall Description: General narrative of the multi-shot clip.
     • Shot-wise Descriptions: Detailed content and cinematographic features (framing, focus, lighting, camera angle) for each individual shot.
     • Transition Type: One of four representative types: shot/reverse shot, cut-in, cut-out, and multi-angle.
     • Transition Description: Explaining how the transition occurs.

   • This hierarchical scheme imbues the dataset with structured, professionally oriented cinematic transition knowledge. An example of the captioning prompt is provided in Figure 8 (from the supplementary material):

     Figure 8 (from the original paper): Prompt for hierarchical captioning using GPT-5-mini.

6. Camera Pose Estimation:

   • VGGT [42] (Visual Geometry Grounded Transformer) is used to estimate camera rotation ($R$) and translation ($t$) relative to the first shot for each subsequent shot. This provides precise motion parameters in matrix form for each transition.

     The final dataset, ShotWeaver40K, contains 40K high-quality videos with detailed shot transition annotations, serving as a foundation for training ShotDirector.

4.2.2. Camera Information Injection

To enable precise control over camera settings and focal centers during shot transitions, ShotDirector integrates parametric camera settings as a critical conditioning signal into the diffusion model. This is done via a dual-branch architecture that injects both Plücker embedding and direct camera extrinsic parameters.

Conventionally, camera pose is defined by intrinsic parameters ($K \in \mathbb{R}^{3 \times 3}$) and extrinsic parameters ($E = [R | t] \in \mathbb{R}^{3 \times 4}$). $R \in \mathbb{R}^{3 \times 3}$ is the rotation component, and $t \in \mathbb{R}^{3 \times 1}$ is the translation vector.

1. Extrinsic Branch:

   • This branch directly injects the camera extrinsic parameters ($E$) into the visual latents.
   • An MLP (Multi-Layer Perceptron) processes the flattened extrinsic matrix: $ C _ { \mathrm { e x t r i n s i c } } = M L P ( \mathrm { f l a t t e n } ( E ) ) $
   • Here, $\mathrm{flatten}(E)$ converts the $3 \times 4$ matrix $E$ into a 12-dimensional vector. The MLP then transforms this vector into an embedding $C_{\mathrm{extrinsic}}$, which captures camera orientation cues.

2. Plücker Branch:

   • This branch uses Plücker embedding to represent the spatial ray map for each pixel, providing more granular geometric information, including intrinsic camera properties.
   • For each pixel (u, v) in the image coordinate space, its Plücker representation is a 6-dimensional vector: $ p _ { u , v } = ( o \times d _ { u , v } , d _ { u , v } ) \in \mathbb { R } ^ { 6 } $
     • $o \in \mathbb{R}^3$ denotes the camera center in world coordinates.
     • $d_{u,v} \in \mathbb{R}^3$ is the viewing direction vector from the camera center to pixel (u, v).
     • $d_{u,v}$ is computed using the inverse of the intrinsic matrix $K^{-1}$ and the rotation matrix $R$: $ d _ { u , v } = R K ^ { - 1 } [ u , v , 1 ] ^ { T } $ The vector $R K^{-1} [u, v, 1]^T$ is then normalized to unit length.
   • The complete Plücker embedding for a frame, $P = [p_{u,v}] \in \mathbb{R}^{h \times w \times 6}$ (where $h \times w$ is the resolution), is processed by convolutional layers: $ C _ { \mathrm { Plü cker } } = C o n v ( P ) $
   • This $C_{\mathrm{Plücker}}$ provides a pixel-wise spatial ray map.

3. Integration into Visual Latents:

   • Finally, the embeddings from both branches for the $i$-th shot ($C_{\mathrm{extrinsic},i}$ and $C_{\mathrm{Plücker},i}$) are added to the visual tokens $z_i$ associated with that shot, prior to the self-attention mechanism in the diffusion model: $ z _ { i } ^ { \prime } = z _ { i } + C _ { \mathrm { e x t r i n s i c } , i } + C _ { \mathrm { Plü cker } , i } $
   • Here, $z_i$ represents the original visual latent representation for the $i$-th shot, and $z_i'$ is the camera-conditioned latent. This direct addition allows the diffusion model to implicitly learn the design intent encoded in camera settings, providing auxiliary cues for generating controlled multi-shot videos.

4.2.3. Shot-aware Mask Mechanism

Beyond parametric conditioning, ShotDirector employs a shot-aware mask mechanism to provide high-level semantic guidance and enable the diffusion model to capture professional shot transition patterns. This mechanism guides tokens to incorporate both global and local cues in both visual and textual domains.

1. Attention Layer Modification:

   • The visual latents $z_i'$ (which already incorporate camera information) are processed through the attention layers of the DiT architecture.
   • The shot-aware mask explicitly constrains the query ($q_{z_i'}$) from the current shot to interact only with its corresponding contextual information (keys and values): $ \mathrm { A t t n } _ { \mathrm { s h o t - a w a r e } } ( z _ { i } ^ { \prime } ) = \mathrm { A t t n } ( q _ { z _ { i } ^ { \prime } } , K ^ { * } , V ^ { * } ) $
     • $q_{z_i'}$ is the query derived from the visual latents of the $i$-th shot.
     • $K^* = [K_i^{\mathrm{global}}, K_i^{\mathrm{local}}]$ and $V^* = [V_i^{\mathrm{global}}, V_i^{\mathrm{local}}]$ represent the combined key and value matrices, respectively, consisting of both global and local information.

2. Visual Masking:

   • Local visual information: Refers to all visual tokens within the current shot.
   • Global visual information: Consists of tokens from the first frame of the entire video. This allows the model to maintain overall scene context and identity.
   • In the initial layers of denoising, all tokens are kept visible to promote sufficient global interaction, ensuring the model establishes overall coherence before focusing on shot-specific details.
   • This mechanism ensures that each shot maintains overall scene context while retaining its shot-specific visual details, aligning with the goal of multi-shot generation that requires both high-level contextual consistency and visual diversity.

3. Textual Masking:

   • Local textual information: Includes shot-specific descriptions and cinematographic cues (e.g., "tight close-up, shallow focus, dim light, cool tones, low angle" from the Shot Caption in Figure 8).

   • Global textual information: Covers shared subject attributes (e.g., "Subject: [Woman 1] a young woman with auburn hair..." from Figure 8), overall narrative, and transition semantics (e.g., "Transition Caption: Cut-out. From tight close-up to wider three-person shot." from Figure 8).

   • This design ensures a precise alignment between textual guidance and corresponding visual tokens.

   • The subject label helps maintain inter-shot consistency (e.g., the same character across shots).

   • Transition semantics (e.g., cut-in, cut-out) provide prior knowledge of professional editing patterns, leading to coherent and controllable transitions.

     By facilitating structured interaction between global and local contexts, the shot-aware mask enables a hierarchical and editing-aware prompting strategy, allowing each shot to be contextually consistent with the whole video while preserving its distinct appearance and directorial intent.

4.2.4. Implementation and Training Strategy

1. Base Model: The training is performed based on Wan2.1-T2V-1.3B [41], a large-scale text-to-video diffusion model, indicating that ShotDirector is built on a strong foundation for video generation.

2. Branch Initialization:

   • The extrinsic branch is initialized with the camera encoder from ReCamMaster [4]. A zero-initialized MLP transfer layer connects this encoder to the DiT framework, ensuring a smooth start to training without immediate large perturbations.
   • The Plücker branch is randomly initialized.

3. Warm-up Phase:

   • During an initial warm-up phase, only the dual-branch encoder (responsible for processing camera information) is trained.
   • The extrinsic branch is limited to training only its transfer layer during this phase. This allows the camera information injection mechanism to stabilize before the core generative model components are fully engaged.

4. Joint Optimization: After the warm-up, the self-attention parameters of the DiT model are unfrozen, and the entire model (including the camera branches and the generative core) undergoes joint optimization.

5. Two-Stage Training Scheme:

   • Recognizing that camera poses in real data (like ShotWeaver40K) can be less reliable than in synthetic datasets, a two-stage training scheme is adopted:

     • Stage 1: The model is initially trained exclusively on the ShotWeaver40K dataset. This stage allows the model to learn the fundamental transition behavior and editing patterns present in real cinematic footage.

     • Stage 2: The training data is augmented with SynCamVideo [5] (a synthetic multi-camera video dataset) at a 7:3 real-to-synthetic ratio. This second stage enhances the model's understanding of camera information as an auxiliary guidance, improving stability and controllability, especially concerning precise camera movements and viewpoints. SynCamVideo provides accurate, ground-truth camera parameters that help refine the camera control module.

       This comprehensive approach ensures that ShotDirector not only learns to generate high-quality video but also internalizes the complex interplay between camera dynamics and narrative-driven shot transitions.

5. Experimental Setup

5.1. Datasets

The experiments primarily utilize two datasets:

• ShotWeaver40K: This is a novel, high-quality multi-shot video dataset constructed by the authors to capture the priors of film-like editing patterns.

  • Source: Curated from 16K full-length films.

  • Scale: Consists of 40K multi-shot video clips. Each clip contains exactly two shots to focus on the transition between them.

  • Characteristics:

    • Duration: Average duration of 8.72 seconds.
    • Aesthetic Score: Average aesthetic score of 6.21 (on some scale, likely 1-10).
    • Inter-shot CLIP Feature Similarity: A mean similarity of 0.7817 between adjacent-shot frame pairs, indicating controlled visual and semantic coherence across transitions.
    • Annotations: Detailed hierarchical captions (covering subjects, overall narrative, shot-wise descriptions, cinematographic features, transition types, and descriptions) and camera poses (rotation and translation) are provided for each clip.
    • Domain: Real-world cinematic footage.

  • Purpose: ShotWeaver40K is crucial for training the model to understand and generate professional editing patterns, as it explicitly encodes both high-level semantic transition types and low-level camera parameters. The rigorous filtering ensures meaningful transitions.

  • Data Sample: An example of the hierarchical captioning output, which forms part of the dataset, is shown in Figure 8:

    Figure 8 (from the original paper): Prompt for hierarchical captioning using GPT-5-mini. This figure also implicitly provides a concrete example of a data sample's textual annotations for ShotWeaver40K, showing how subjects, general descriptions, shot-specific details, and transition types are structured.

    The figure provides a clear example for Cut-in, Shot/Reverse Shot, Cut-out, and Multi-Angle transition types. For instance, for Cut-in:

    • Subject: $<Man 1>$ an elderly Asian man... $<Man 2>$ a bespectacled man... $<Man 3>$ a large bearded man...

    • General Description: $<Man 1>$ speaks with $<Man 2>$ and $<Man 3>$ in a lush outdoor garden; the second frame cuts closer to $<Man 1>$ revealing hooded attendants standing behind him.

    • Shot 1: $<Man 1>$ holds a book facing $<Man 2>$ and $<Man 3>$ outdoors. Medium-wide shot, eye-level, natural daylight, moderate depth-of-field, vibrant costume colors, balanced composition.

    • Shot 2: $<Man 1>$ in a tighter close-up with hooded attendants blurred behind. Close-up, shallow focus on face, soft natural lighting, warm tones, intimate framing emphasizing expression.

    • Transition Caption: Cut-in. Cut from a medium-wide group shot including $<Man 2>$ and $<Man 3>$ to a closer, intimate portrait of $<Man 1>$, highlighting his expression and attendants.

      Statistical characteristics of ShotWeaver40K are summarized in Figure 9 (from the supplementary material):

      Figure 9 (from the original paper): Distribution of key video attributes in ShotWeaver40K. This figure shows histograms for duration (seconds), clip similarity, and aesthetic score, illustrating the statistical properties of the dataset.

• SynCamVideo [5]: A synthetic multi-camera video dataset.

  • Purpose: Used to augment ShotWeaver40K in the second stage of training (at a 7:3 real-to-synthetic ratio). Synthetic data provides highly accurate ground-truth camera parameters, which helps the model to better leverage camera information as auxiliary guidance, especially for precise camera control, compensating for potential inaccuracies in real-world camera pose estimation.

5.2. Evaluation Metrics

A comprehensive assessment protocol is designed, encompassing controllability, consistency, and visual quality. The evaluation dataset consists of 90 prompts featuring hierarchical text captions and camera poses.

For every evaluation metric, here's a detailed explanation:

5.2.1. Transition Control

• Transition Confidence Score (↑):

  • Conceptual Definition: This metric quantifies the clarity and reliability of detected shot transitions in generated videos. It measures how strongly a model indicates a shot boundary, distinguishing clear cuts from ambiguous changes. A higher score indicates a more pronounced and confident transition.
  • Mathematical Formula: $ \mathrm { T r a n s i t i o n C o n f i d e n c e Score = max } ( \sigma ( d ) ) $
  • Symbol Explanation:
    • $\mathrm{Transition Confidence Score}$: The final score for a generated video, indicating the maximum confidence of a transition within it.
    • $\mathrm{max}(\cdot)$: The maximum value function.
    • $\sigma(\cdot)$: The sigmoid activation function, which maps any real-valued number to a value between 0 and 1, representing a probability or confidence level.
    • $d$: A frame-wise transition likelihood feature vector output by TransNetV2 [38]. Each element of $d$ corresponds to the raw transition probability (before sigmoid) for a specific frame.
  • Method: TransNetV2 [38] is used to compute this score. TransNetV2 is a deep learning model specifically trained for shot transition detection.

• Transition Type Accuracy (↑):

  • Conceptual Definition: This metric assesses whether the generated transition type (e.g., cut-in, cut-out) aligns with the specified transition type in the input prompt. It measures the model's ability to correctly interpret and execute high-level editing pattern instructions.

  • Mathematical Formula: This is a classification accuracy metric. Let $N$ be the total number of evaluation prompts, and $I(\cdot)$ be the indicator function. $ \mathrm{Transition\ Type\ Accuracy} = \frac{1}{N} \sum_{j=1}^{N} I(\mathrm{predicted_type}_j = \mathrm{ground_truth_type}_j) $

  • Symbol Explanation:

    • $\mathrm{Transition\ Type\ Accuracy}$: The percentage of generated videos where the predicted transition type matches the ground truth.
    • $N$: Total number of evaluation prompts/videos.
    • $j$: Index for each generated video.
    • $I(\cdot)$: Indicator function, which equals 1 if the condition inside is true, and 0 otherwise.
    • $\mathrm{predicted\_type}_j$: The transition type classified for the $j$-th generated video.
    • $\mathrm{ground\_truth\_type}_j$: The target transition type specified in the prompt for the $j$-th generated video.

  • Method: Qwen [43], a vision-language model (VLM), is employed to classify the transition type of each generated video. The classification is then compared against the ground-truth transition type from the prompt. The distribution of transition types in the evaluation set is provided in Table 5, and the prompt used for VLM recognition is shown in Figure 11.

    The following are the results from Table 5 of the original paper:

    Transition Type	Count
    Cut-in	24
    Cut-out	26
    Shot/Reverse Shot	25
    Multi-Angle	15

    Figure 11 (from the original paper): Prompt for recognition of shot transition type using Qwen.

5.2.2. Overall Quality

• Aesthetic Score (↑):

  • Conceptual Definition: Measures the perceived artistic quality or visual appeal of the generated videos. It reflects how pleasing or well-composed the visuals are.
  • Method: An aesthetic predictor (likely a pre-trained model capable of scoring images/videos based on learned aesthetic preferences) is used.

• Imaging Quality (↑):

  • Conceptual Definition: Measures the technical quality of the generated videos, such as sharpness, clarity, and absence of artifacts. It assesses how photorealistic and clean the visuals are.
  • Method: An imaging quality model [24] is used for assessment. MUSIQ (Multi-scale Image Quality Transformer) [24] is a potential candidate, which learns to predict image quality using transformers.

• Overall Consistency (↑):

  • Conceptual Definition: Quantifies the alignment between the textual prompt and the generated video's content. It ensures that the video accurately reflects the semantic description provided.
  • Method: ViCLIP [44] feature similarity is used. ViCLIP is a video-text model that embeds videos and text into a common space, allowing their similarity to be measured. Higher similarity means better text-video alignment.

• Fréchet Video Distance (FVD) (↓):

  • Conceptual Definition: A metric used to evaluate the perceptual quality and diversity of generated videos by comparing the distribution of features from generated videos to those from real videos. A lower FVD score indicates that the generated videos are perceptually closer to real videos. It's an extension of the Fréchet Inception Distance (FID) used for images.
  • Mathematical Formula: $ \mathrm{FVD} = ||\mu_r - \mu_g||^2 + \mathrm{Tr}(\Sigma_r + \Sigma_g - 2(\Sigma_r \Sigma_g)^{1/2}) $
  • Symbol Explanation:
    • $\mu_r$: The mean feature vector of real videos.
    • $\mu_g$: The mean feature vector of generated videos.
    • $\Sigma_r$: The covariance matrix of features from real videos.
    • $\Sigma_g$: The covariance matrix of features from generated videos.
    • $||\cdot||^2$: The squared Euclidean distance.
    • $\mathrm{Tr}(\cdot)$: The trace of a matrix (sum of diagonal elements).
    • $(\cdot)^{1/2}$: Matrix square root.
  • Method: Features (typically from a pre-trained video classification model like I3D) are extracted from both real and generated videos. The means and covariance matrices of these feature distributions are then used in the formula.

5.2.3. Cross-shot Consistency

• Semantic Consistency (↑):

  • Conceptual Definition: Measures the semantic coherence of content (e.g., objects, actions, overall meaning) across different shots within the generated multi-shot video.
  • Method: ViCLIP [44] features are extracted from each shot. The similarity of these features across adjacent shots (or across all shots) is averaged to quantify semantic coherence.

• Visual Consistency (↑):

  • Conceptual Definition: Measures the visual coherence of specific elements, particularly subjects and backgrounds, across adjacent shots. It ensures that characters and environments remain visually consistent despite camera changes.
  • Method: Computed as the averaged similarity of subject features [31] and background features [11] between adjacent shots.
    • Subject similarity likely uses features from models like DINOv2 [31], which learns robust visual features without supervision and is good for object recognition.
    • Background similarity might use features from models like DreamSim [11], which learns human visual similarity.

5.3. Baselines

The paper compares ShotDirector against three categories of baseline models to demonstrate its effectiveness:

1. End-to-End Multi-Shot Video Generation: These models aim to generate multiple shots within a single generative process.

   • Mask2DiT [33]: A Diffusion Transformer model that uses dual masks for multi-scene long video generation.
   • CineTrans [46]: A masked diffusion model designed to generate videos with cinematic transitions.

2. Stitching-based / Shot-by-Shot Generation: These approaches generate individual shots and then combine them, focusing on consistency.

   • StoryDiffusion [53] with CogVideoXI2V [50]: StoryDiffusion generates a series of keyframes, which are then animated by a text-to-video (or image-to-video) model like CogVideoXI2V.
   • Phantom [28]: A reference-to-video method that uses reference images (generated by a text-to-image model) to compose multi-shot videos, aiming for subject consistency.

3. Pre-trained Video Diffusion Models: Large-scale models trained for general video generation, which might exhibit some multi-shot capabilities implicitly.

   • HunyuanVideo [26]: A systematic framework for large video generative models.
   • Wan2.2 [41]: An open and advanced large-scale video generative model.

4. Multi-View and Camera-Control Methods: Models specifically designed for camera control or multi-view synthesis.

   • SynCamMaster [5]: Synthesizes two-view videos to produce shot transitions, focusing on synchronized multi-camera generation.

   • ReCamMaster [4]: Performs camera-controlled video editing based on existing videos, designed for smoothly varying camera poses.

     These baselines represent the state-of-the-art in various related domains, allowing for a comprehensive evaluation of ShotDirector's performance across different aspects of multi-shot video generation and control.

6. Results & Analysis

6.1. Core Results Analysis

The experimental results demonstrate that ShotDirector significantly outperforms existing baselines in achieving controllable, film-like multi-shot video generation with coherent narrative flow.

6.1.1. Qualitative Results

Figure 4 (from the original paper) provides a visual comparison of ShotDirector against representative baselines, showcasing its ability to respond to specified shot transition types and exhibit professional editing patterns.

Figure 4 (from the original paper): Qualitative comparison of ShotDirector against other methods, showing its ability to produce consistent shot transitions with fine control over shot content.

The qualitative analysis highlights several key observations:

• Mask2DiT: Exhibits instability and often produces animation-like visuals, indicating lower photorealism and control over complex visual details.

• CineTrans: While capable of producing shot transitions, it lacks a clear understanding of shot transition types, resulting in transitions without professional cinematic characteristics.

• StoryDiffusion and Phantom: These stitching-based approaches maintain moderate subject and style consistency but struggle with coherent visual details and continuous narrative flow. StoryDiffusion often shows substantial disparities between shots. Phantom improves subject consistency but fails to maintain a unified scene or background continuity, breaking the narrative.

• HunyuanVideo and Wan2.2: As large-scale pre-trained models, they demonstrate certain awareness of professional editing patterns due to training on vast datasets. However, their outcomes are unstable, and they cannot ensure the occurrence of multi-shot structures or explicitly control transition types. HunyuanVideo performs better than Wan2.2 in multi-shot scenarios.

• SynCamMaster: Maintains camera positioning and scene consistency but has no concept of shot transition type and produces relatively low-quality visuals. Its consistency comes at the cost of visual fidelity, possibly due to reliance on synthetic training data.

• ReCamMaster: Designed for smoothly varying camera poses, it struggles with abrupt pose changes inherent in hard cuts, leading to distorted frames and failure to achieve true shot transitions.

  In stark contrast, ShotDirector effectively responds to specified shot transition types (e.g., cut-in, shot/reverse-shot), demonstrating film-like, professional editing patterns that convey coherent visual storytelling and semantic expression. The generated videos exhibit stable transitions while maintaining an understanding of professional cinematographic language.

6.1.2. Quantitative Results

Table 1 (from the original paper) summarizes the quantitative evaluation across the designed suite of metrics.

The following are the results from Table 1 of the original paper:

Key findings from the quantitative results:

• Transition Control: ShotDirector achieves the highest Transition Confidence Score (0.8956) and Transition Type Accuracy (0.6744), indicating its superior ability to generate clear transitions that align with specified editing patterns. Most baselines show weak control, with Mask2DiT, Wan2.2, and ReCamMaster having very low scores. Even CineTrans, which focuses on transitions, is significantly lower (0.7976 confidence, 0.3944 accuracy).
• Overall Quality: ShotDirector leads in Aesthetic (0.6374) and Imaging (0.6984) quality, and achieves the best FVD (68.45, lower is better), meaning its generated videos are perceptually closer to real film-edited videos. It also has the highest Overall Consistency (0.2394), reflecting strong text-video alignment.
• Cross-shot Consistency: While SynCamMaster achieves the highest Semantic (0.7949) and Visual (0.8418) consistency scores, the paper notes this is "at the cost of visual fidelity" (low aesthetic and imaging quality), implying an over-constraint. ShotDirector ranks second in both Semantic (0.7918) and Visual (0.8251) consistency, demonstrating it maintains strong coherence while also achieving superior overall quality. This highlights ShotDirector's ability to balance consistency with visual fidelity and controlled transitions.

6.1.3. Transition Type Distribution

Figure 5 (from the original paper) illustrates the distribution of generated shot transition types for different methods.

Figure 5 (from the original paper): The distribution of shot transition types in videos generated by different methods. Our model (ShotDirector) stably produces transitions and achieves a more balanced distribution across transition types compared to other methods.

• Many methods (ReCamMaster, HunyuanVideo, Mask2DiT, Wan2.2) produce a high proportion of "No-Transition" videos, indicating their inability to reliably generate cuts.
• Others (SynCamMaster, CineTrans) tend to default to "Multi-Angle" transitions, suggesting a lack of explicit control over diverse editing patterns.
• In contrast, ShotDirector shows a more balanced distribution across Cut-in, Cut-out, Shot/Reverse Shot, and Multi-Angle types, with a very low "No-Transition" rate. This demonstrates its robust and diverse transition capability, aligning with directorial intent.

6.2. Ablation Studies / Parameter Analysis

Ablation studies were conducted to evaluate the individual contribution of each component of the ShotDirector framework.

6.2.1. Camera Information Injection

Table 3 (from the original paper) presents the ablation results for camera information, using RotErr (Rotation Error) and TransErr (Translation Error) to measure camera pose control performance (lower is better).

The following are the results from Table 3 of the original paper:

• Removing all camera information (w/o Camera Info) leads to the highest RotErr and TransErr, confirming that camera information is crucial for accurate pose control.
• Both the Plücker Branch and Extrinsic Branch contribute positively. Removing either branch (w/o Plücker Branch or w/o Extrinsic Branch) results in higher errors compared to the full model.
• The Plücker Branch performing slightly better than the Extrinsic Branch individually (comparing w/o Extrinsic Branch vs w/o Plücker Branch for RotErr and TransErr). This is attributed to the Plücker representation including intrinsic parameters and spatial ray maps, which help the model interpret camera pose variations more comprehensively in shot transitions. The full ShotDirector with both branches achieves the lowest errors, demonstrating their complementary benefits.

6.2.2. Shot-Aware Mask Mechanism

Table 2 (from the original paper) summarizes the ablation results for the shot-aware mask.

The following are the results from Table 2 of the original paper:

• Impact of Visual Mask: Removing the Visual Mask (w/o Visual Mask) significantly lowers Transition Confidence (from 0.8956 to 0.8044) and Type Accuracy (from 0.6744 to 0.5583). This indicates that the visual mask is critical for enforcing distinct shot boundaries and preventing information leakage, thereby strengthening transition effects and diversity.
• Impact of Semantic Mask: Removing the Semantic Mask (w/o Semantic Mask) primarily affects Semantic Consistency (from 0.7918 to 0.6901) and Visual Consistency (from 0.8251 to 0.7761), along with a slight drop in Type Accuracy. This confirms that the semantic mask is crucial for aligning textual guidance with visual content and ensuring overall coherence across shots.
• The visual shot-aware mask has a stronger effect on transition control (Confidence, Type Acc), as globally visible visual tokens would cause information leakage and reduce shot diversity.
• The semantic shot-aware mask primarily influences consistency (Semantic, Visual), which aligns with the intuition that fine-grained semantic control helps balance consistency and diversity.

6.2.3. Training Strategy

Table 2 also includes ablation results for the two-stage training process.

• ShotDirector (w/o Training): This refers to the base model Wan2.1-T2V-1.3B without any fine-tuning using ShotWeaver40K. It shows very low Transition Confidence (0.1402) and Type Accuracy (0.2489), confirming that the base model lacks dedicated multi-shot transition capability. Interestingly, it has high Semantic (0.8419) and Visual (0.8256) consistency, but this is explained by its failure to generate large visual variations or distinct shots, effectively remaining "consistent" because it doesn't transition much.

• ShotDirector (w/o Stage II Training): This variant is trained only on ShotWeaver40K (Stage I). It performs well in Transition Confidence (0.8615) and Type Accuracy (0.6300) but is slightly inferior to the full model. This verifies that Stage II training with SynCamVideo augmentation further enhances transition controllability and overall visual quality (lower FVD, better Aesthetic/Imaging), especially by refining the camera control mechanism using accurate synthetic camera parameters.

  These ablations clearly validate the design choices of ShotDirector, showing that each component and the multi-stage training strategy contribute significantly to its overall superior performance in controllable multi-shot video generation.

6.3. Additional Capability

The paper highlights an additional capability of ShotDirector: its seamless integration with other functional modules trained on the base model. This indicates the generalizability and modularity of the framework.

• Reference-to-Video Synthesis: As an example, the authors demonstrate transferring the weights of a reference-to-video model ([22], likely a model like VACE for video creation and editing) directly to ShotDirector.

• This enables ShotDirector to generate multi-shot videos featuring specified subjects by taking reference images as additional inputs.

• Figure 6 (from the original paper) visually illustrates this capability.

  Figure 6 (from the original paper): The performance of our method after being transferred to the reference-to-video model. This figure shows ShotDirector's ability to generate multi-shot videos consistent with a reference image, while executing specific transition types like Cut-in and Multi-Angle.

This observation indicates that ShotDirector preserves the base model's understanding of video content, allowing it to interface with other specialized functional modules. Such adaptability further highlights the generalizability of their approach and its potential for more complex creative workflows.

7. Conclusion & Reflections

7.1. Conclusion Summary

This work introduces ShotDirector, a unified framework for controllable multi-shot video generation that focuses on the often-overlooked aspect of cinematographic transitions. By integrating parameter-level camera control (via a dual-branch injection of Plücker embeddings and extrinsic parameters) with hierarchical, editing-pattern-aware prompting (facilitated by a shot-aware mask mechanism), ShotDirector enables fine-grained control over shot transitions. The model can accurately generate various professional editing patterns while maintaining both semantic and visual consistency across shots. The development of the ShotWeaver40K dataset, specifically curated for film-like editing patterns, and a comprehensive evaluation protocol further solidifies this contribution. Extensive experiments demonstrate ShotDirector's effectiveness in producing film-like, coherent, and directorially controllable multi-shot videos, establishing a new perspective that emphasizes directorial control in video generation.

7.2. Limitations & Future Work

The authors acknowledge several limitations and suggest promising avenues for future exploration:

• Failure Case - Subject Mixing: In certain generated samples, the visual characteristics of multiple subjects can become mixed, especially when multiple subjects are present. This suggests an insufficient understanding of multi-subject scenarios within the current model.

  • Potential Improvement: The authors propose that providing more detailed bounding-box-level annotations in the dataset could enhance the model's ability to better understand and model complex multi-subject situations. An example of this failure case is shown in Figure 12 (from the supplementary material):

    Figure 12 (from the original paper): A representative failure case in which the visual characteristics of multiple subjects become unintentionally blended during generation.

• Integrating Camera-Control and Semantic Cues More Cohesively: The current approach employs two separate modules for high-level semantic information and parameter-level camera control. A future direction is to investigate how to unify these two forms of conditioning more seamlessly, leading to a more coherent and expressive transition modeling process. This could involve learning a joint representation or a more deeply integrated architectural design.

• Toward Longer Videos with More Shot Transitions: The current work primarily focuses on two-shot clips. Extending the framework to generate longer video sequences containing a richer and more diverse set of complex transition types is a valuable research direction. The authors believe this is feasible with additional data and fine-tuning on extended datasets, suggesting that their core methodology is scalable.

7.3. Personal Insights & Critique

This paper presents a highly relevant and innovative step forward in video generation. By explicitly focusing on cinematographic language and directorial control over shot transitions, ShotDirector moves beyond merely generating visually consistent video frames to creating narratively meaningful content. The core insight that a cut is a deliberate directorial choice, not just a visual break, is well-articulated and forms a strong foundation for the proposed method.

The dual approach of parameter-level camera control and hierarchical editing-pattern-aware prompting is particularly elegant. It addresses both the low-level geometric precision required for camera movements and the high-level semantic understanding of narrative flow. The ShotWeaver40K dataset is a crucial contribution, as high-quality, annotated data is often the bottleneck for such specialized tasks. Its rigorous curation process, including VLM-based filtering, demonstrates a thorough understanding of the challenges in data preparation for cinematic tasks.

Potential Issues/Unverified Assumptions:

1. Complexity of Control: While fine-grained control is a strength, the level of detail required in prompting (hierarchical captions, precise camera parameters) might be too demanding for casual users. Simplifying the input interface while retaining control could be an area for improvement.
2. Scalability to Long Narratives: The current focus on two-shot sequences, while necessary for foundational research, might not fully capture the complexities of a feature-length film's narrative structure and the accumulation of directorial choices over time. Scaling to truly long videos with multiple, interacting characters and evolving plots will introduce new challenges beyond just consecutive shot transitions (e.g., character consistency over long durations, scene continuity across complex environments).
3. Aesthetic Nuance: While quantitative aesthetic scores are provided, the subjective nature of "film-like" and "cinematographic expression" means there might still be a gap between model-generated and human-directed artistic choices. The current system provides powerful tools, but the artistic judgment often lies in subtle nuances that are hard to quantify or prompt.

Applicability and Future Value: The methods and conclusions of ShotDirector can be widely applied:

• Film Pre-visualization: Directors could rapidly prototype and visualize complex shot sequences and transitions.

• Content Creation: Democratizing access to sophisticated editing techniques for independent filmmakers or content creators.

• Virtual Production: Generating dynamic virtual camera paths and cuts for virtual environments.

• Education: Teaching film students about editing principles through interactive generation.

  This paper paves the way for generative AI to become a more integral directorial tool, enabling a future where AI can assist not just in generating visual frames, but in crafting coherent, emotionally resonant, and narratively compelling cinematic experiences. Future work could also explore incorporating sound design and musical scoring into this framework, further enhancing its narrative capabilities.