1. Bibliographic Information

1.1. Title

The title of the paper is: Compact and Wide-FOV True-3D VR Enabled by a Light Field Display Engine with a Telecentric Path. This title clearly indicates the central topic: a virtual reality (VR) display system that offers true 3D visuals, a compact design, and a wide field of view (FOV), achieved by integrating a light field display (LFD) engine with a telecentric optical path.

1.2. Authors

The authors of the paper are: Qimeng Wang, Yi Liu, Xinni Xie, Yaya Huang, Hao Huang, Hanlin Hou, and Zong Qin. Their affiliation is the School of Electronics and Information Technology, Sun Yat-Sen University, Guangzhou, 510006 China. The corresponding author's email is qinzong@mail.sysu.edu.cn.

1.3. Journal/Conference

The paper does not explicitly state the journal or conference where it was published, but the format and content suggest it is likely a publication in a peer-reviewed conference proceedings or a journal focusing on optics, displays, or virtual reality. Given the technical nature and specific domain, it would likely be a reputable venue within these fields.

1.4. Publication Year

The publication year for this paper is not explicitly stated within the provided text. However, a reference [10] indicates SID Symp. Dig. Tech. 55(1), 1271-1274 (2024), which suggests recent work.

1.5. Abstract

This paper introduces a true-3D VR display system that leverages a light field display (LFD) engine. This engine is designed to generate intermediate images that incorporate computational focus cues. To achieve high resolution, the system employs a field-sequential-color (FSC) micro-LCD. A significant challenge in LFDs, the aberration-induced FOV reduction, is addressed and mitigated through the implementation of a telecentric optical path. Experimental results demonstrate the successful generation of clear 3D images with a field of view (FOV) exceeding 60 degrees.

1.6. Original Source Link

The original source link provided is: /files/papers/693e2cf4a078743fa50a04b5/paper.pdf. This indicates that the paper is available as a PDF document. Based on the context, it appears to be an officially published paper.

2. Executive Summary

2.1. Background & Motivation

The core problem this paper aims to solve is the vergence-accommodation conflict (VAC) in current Virtual Reality (VR) displays, particularly those utilizing Pancake optics. Pancake optics are a popular solution for VR headsets due to their compact and lightweight design, achieved through folded optical paths, and their ability to provide a large Field of View (FOV) without significantly increasing system volume. However, most Pancake headsets only support a fixed virtual image distance, which causes VAC. VAC occurs when the eyes' vergence (angle at which the eyes converge on an object) and accommodation (focusing of the lens) cues provide conflicting depth information, leading to visual discomfort, eye strain, and a reduced sense of realism in VR experiences.

This problem is important because true-3D perception is crucial for immersive VR experiences, impacting applications in education, gaming, and healthcare. The existing challenges include:

• Mechanical solutions for depth-variable Pancake VR: These are often complicated with limited response speed and cannot support multiple focal planes within a single scene.
• Varifocal elements (e.g., LC lenses): While capable of diopter adjustment, they typically cannot present true 3D scenes and add complexity due to dynamic components.
• Other VAC-free technologies:

  • Maxwellian view displays offer always-in-focus retinal images but are limited by a fixed pupil position and a restricted eyebox.

  • Holographic displays reproduce wavefronts for phase information but require coherent sources and complex optical systems, challenging compactness for near-eye displays. Although recent advancements integrate AI-driven digital holography into waveguides, a more affordable VAC-free VR solution is still needed.

  • Light Field Displays (LFDs) using microlens arrays (MLAs) can generate computational focus cues, but when directly used as near-eye displays, they suffer from visual resolution drops due to pixel magnification and severe FOV limitations caused by MLA aberration.

    The paper's innovative idea is to integrate an LFD as a picture engine with Pancake optics to create a VAC-free Pancake VR headset. This approach aims to leverage the LFD's ability to generate variable depth cues while benefiting from the Pancake optics' compactness and large FOV. The key innovation lies in addressing the FOV limitation of LFDs by using the telecentric optical path inherent in Pancake optics.

2.2. Main Contributions / Findings

The primary contributions and key findings of this paper are:

• Proposed VAC-free Pancake VR architecture: The paper introduces a novel design that combines a Light Field Display (LFD) engine with Pancake optics to achieve true-3D VR experiences by providing computational focus cues, thereby mitigating the vergence-accommodation conflict (VAC).

• High-resolution display engine: It incorporates a field-sequential-color (FSC) micro-LCD with a 2.3K-by-2.3K resolution and a mini-LED RGB backlight. This choice significantly enhances resolution by eliminating color filter arrays and improves optical efficiency, which is crucial for Pancake optics that typically have low efficiency.

• Expanded Field of View (FOV) through telecentric path: The paper demonstrates that the aberration-induced FOV reduction commonly found in direct LFD near-eye implementations can be effectively mitigated by utilizing the object-space telecentric optical path of Pancake optics. This ensures that lenslets in the microlens array (MLA) work with near-paraxial rays, leading to low aberrations over a large FOV.

• Image quality matching strategy: A detailed analysis and strategy are presented for matching the image quality variations between the LFD engine and the Pancake module across different virtual image distances. This involves intentionally configuring the LFD engine's Central Depth Plane (CDP) to a relatively worse object plane of the Pancake to achieve a balanced image quality.

• Experimental validation of true-3D and wide FOV: A prototype was built using a 1500-ppi FSC micro-LCD, an MLA, and a commercial Pancake module.

  • It successfully demonstrated computationally adjustable virtual image distances, showcasing the true-3D feature by clearly focusing on objects at different depth planes.
  • The measured FOV was 68.6 degrees, which is significantly larger than what an LFD engine alone would achieve and close to the native FOV of the Pancake module.

• Compact design: The integration resulted in an acceptable additional optical track of 2.1 cm, maintaining a relatively compact form factor suitable for near-eye displays.

  These findings solve the critical problem of VAC in VR headsets while simultaneously addressing resolution and FOV limitations that often plague LFD-based approaches, offering a practical pathway toward more immersive and comfortable VR experiences.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To understand this paper, a beginner needs to grasp several core concepts in optics, display technology, and virtual reality.

3.1.1. Virtual Reality (VR) Displays

Virtual Reality (VR) displays are head-mounted devices that immerse users in a simulated environment by providing visual and sometimes auditory and haptic feedback. They typically consist of a display panel and an optical system that magnifies the image and presents it to the user's eyes. Key performance indicators for VR displays include Field of View (FOV), resolution, refresh rate, and the ability to present true-3D images.

3.1.2. Pancake Optics

Pancake optics are a type of optical system commonly used in VR headsets to achieve a compact and lightweight design with a large Field of View (FOV). Their primary mechanism involves a folded optical path created by a combination of lenses, quarter-wave plates (QWPs), half-mirrors, and reflective polarizers. Light from the display panel passes through a QWP to become circularly polarized, then enters the lens module, is reflected multiple times within the cavity, and finally exits to the user's eye. This folded path reduces the overall optical track (the distance light travels) required, making the headset thinner.

As shown in Figure 1 from the original paper, the Pancake system works as follows:

1. Light from the display panel (e.g., micro-LCD) is emitted.

2. It passes through a quarter-wave plate (QWP), which converts linearly polarized light into circularly polarized light.

3. This circularly polarized light then enters the front lens through a half-mirror.

4. The light undergoes multiple reflections within the cavity (between the front lens and other optical elements like a reflective polarizer and the half-mirror). Each reflection changes the polarization state.

5. After the reflections, the light eventually passes through the half-mirror and exits the lens module towards the observer's eye.

   该图像是一个示意图，展示了宽视场真3D VR显示的工作原理。它包括了显示器、四分之一波片（QWP）、前透镜、半透镜和反射偏振器，最终将图像投射到观察者的眼睛中。

Fig. 1. Working principle of the Pancake.

3.1.3. Vergence-Accommodation Conflict (VAC)

Vergence-Accommodation Conflict (VAC) is a fundamental problem in many 3D displays, including VR headsets. In the real world, when you look at an object, your eyes automatically perform two actions simultaneously:

• Vergence: Your eyes rotate inward (converge) to point at the object. The closer the object, the more your eyes converge.

• Accommodation: The lens in your eye changes shape to focus the image of the object sharply on your retina. The closer the object, the more your lens accommodates (becomes fatter).

  In traditional VR displays, the image is typically rendered at a fixed virtual distance (e.g., 2 meters). This means:

• Vergence will change as you focus on virtual objects at different simulated distances within the VR scene.

• However, your eyes' accommodation remains fixed at the display's virtual image distance, as the light rays entering your eyes are effectively coming from a single plane.

  This mismatch between the vergence cue (which suggests varying depth) and the accommodation cue (which suggests fixed depth) causes VAC. Symptoms include eye strain, fatigue, headaches, and a reduced sense of realism. True-3D displays aim to resolve VAC by providing accurate accommodation cues that match vergence cues.

3.1.4. Light Field Display (LFD)

A Light Field Display (LFD) is a technology capable of rendering true-3D images by reconstructing the light field (the direction and intensity of light rays at every point in space). Unlike stereoscopic 3D displays which only provide two viewpoints (one for each eye), an LFD generates multiple viewpoints, allowing the viewer's eyes to naturally focus (accommodate) on objects at different depths.

The core components of an LFD typically include a microdisplay and a microlens array (MLA). As illustrated in Figure 2:

1. An Elemental Image Array (EIA) is displayed on the microdisplay. The EIA is a composite image made up of many small images (elemental images), each captured from a slightly different perspective of the 3D scene.

2. The MLA, positioned in front of the microdisplay, consists of many tiny lenses (lenslets). Each lenslet corresponds to an elemental image on the microdisplay.

3. Each lenslet projects its corresponding elemental image into space, manipulating the light rays. By encoding parallaxes (the apparent displacement of an object when viewed from different positions) in the EIA, the MLA reconstructs the light rays from the original 3D scene.

4. This reconstruction creates computational focus cues, meaning that light rays originating from different virtual depths converge at different physical distances, allowing the human eye to naturally accommodate and perceive true 3D.

   该图像是一个示意图，展示了光场显示技术的工作原理。左侧为元素图像阵列，右侧展示通过透镜阵列重建的三维图像。该技术通过控制每个体素的光线，从而实现真实三维效果。

Fig. 2. Working principle of the light field display.

3.1.5. Microlens Array (MLA)

A Microlens Array (MLA) is a sheet containing a periodic arrangement of many small lenses (lenslets). In LFDs, the MLA is placed in front of a microdisplay to direct light from individual pixels or elemental images into specific angular directions. This creates the different viewpoints necessary for light field reproduction. The characteristics of the MLA, such as lenslet pitch (distance between centers of adjacent lenslets) and focal length, are crucial for the performance of the LFD. A short-focal MLA can lead to significant pixel magnification when used directly near the eye, which reduces visual resolution. MLA aberration (optical imperfections) can also severely limit the Field of View (FOV).

3.1.6. Telecentric Optical Path

A telecentric optical path is an optical design where the chief rays (rays passing through the center of the aperture stop) are parallel to the optical axis in either object space, image space, or both.

• Object-space telecentric: Chief rays from the object are parallel to the optical axis when entering the lens. This means the magnification does not change with the object's distance from the lens, making it useful for precise measurements.

• Image-space telecentric: Chief rays exiting the lens are parallel to the optical axis.

• Bi-telecentric: Telecentric in both object and image space.

  In the context of this paper, an object-space telecentric path means that chief rays originating from the microdisplay (which acts as an object for the Pancake system) are nearly perpendicular to the object plane (the display surface) when they enter the optical system. This is achieved by placing the aperture stop (which for the human eye in a VR system is effectively the eye pupil) at the image-space focal point of the lens module. The key benefit of a telecentric path in this LFD-Pancake integration is to ensure that all lenslets in the MLA (which is part of the LFD engine feeding the Pancake) work with near-paraxial rays (rays close to the optical axis). This minimizes aberrations that would otherwise occur with oblique rays (rays far from the optical axis), especially in large FOV scenarios.

3.1.7. Field-Sequential-Color (FSC) Micro-LCD

A Field-Sequential-Color (FSC) micro-LCD is a type of liquid crystal display that achieves full color without using a traditional color filter array (CFA) (red, green, blue subpixels arranged in a pattern). Instead, FSC-LCDs display red, green, and blue images sequentially in rapid succession (field by field). A mini-LED RGB backlight cycles through these colors very quickly. The human eye's visual persistence (the phenomenon where an image lingers on the retina for a brief period after it disappears) then merges these rapidly changing colored images into a single full-color perception.

Advantages of FSC-LCDs:

• Higher Resolution: By removing subpixels and color filters, each pixel on the display can directly represent full-color information, effectively tripling the perceived spatial resolution compared to a subpixel-based display of the same physical pixel count.

• Increased Optical Efficiency: Color filters absorb a significant portion of light. Eliminating them means more light passes through, leading to higher optical efficiency. This is particularly beneficial for Pancake optics which inherently have low optical efficiency due to multiple reflections and polarization losses.

• Compactness: Mini-LED backlights are typically very thin, contributing to the overall compactness of the display engine.

  A potential drawback of FSC-LCDs is color breakup (also known as the "rainbow effect"), which can occur if the refresh rate is not high enough or if the user makes rapid eye movements. The paper mentions previous work on suppressing color breakup using deep learning [11].

Figure 4 from the original paper illustrates the difference between traditional subpixel-based LCDs and FSC-LCDs.

• Figure 4(b) shows a subpixel-based LCD, where each physical pixel is divided into red, green, and blue subpixels to form a full color.

• Figure 4(c) shows an FSC-LCD, where a mini-LED RGB backlight cycles through red, green, and blue light, and the entire pixel (not just subpixels) displays the corresponding color component sequentially.

  该图像是图表，展示了2.1英寸FSC微型LCD的细节（图4(a)）以及子像素LCD（图4(b)）和FSC-LCD（图4(c)）的工作原理，分析了其在空间和时间上对人眼信息处理的影响。

Fig.4. (a) The 2.1-inch FSC micro-LCD. (b) Subpixelbased LCD and (c) FSC-LCD.

3.1.8. Modulation Transfer Function (MTF)

The Modulation Transfer Function (MTF) is a key metric used in optics to quantify the image quality and resolution performance of an optical system. It describes how well an optical system can transfer contrast from the object to the image at different spatial frequencies.

• Spatial Frequency: Refers to the number of line pairs per millimeter (lp/mm) or cycles per degree (cpd) in an image. High spatial frequencies correspond to fine details.

• Contrast (Modulation): For a sinusoidal pattern, contrast is defined as $(I_{max} - I_{min}) / (I_{max} + I_{min})$, where $I_{max}$ is the maximum intensity and $I_{min}$ is the minimum intensity.

• MTF value: An MTF value ranges from 0 to 1 (or 0% to 100%). A value of 1 means perfect contrast transfer (the image has the same contrast as the object). A value of 0 means no contrast is transferred (fine details are completely blurred).

  A higher MTF value at a given spatial frequency indicates better image quality and sharper details. In optical system design, MTF curves (plots of MTF vs. spatial frequency) are used to evaluate and compare the performance of lenses and imaging systems. The paper uses MTF to evaluate the Pancake module's performance at different virtual image distances and to match its image quality with the LFD engine.

3.2. Previous Works

The paper references several prior works to contextualize its contributions and highlight existing limitations.

3.2.1. Pancake Optics and VAC

• Standard Pancake Optics (Li et al. [1], Meta Platforms [2]): Current Pancake optics provide compactness and large FOV through folded optical paths. However, as mentioned in the introduction, they typically support only a fixed virtual image distance, leading to VAC. Reference [1] discusses broadband cholesteric liquid crystal lenses for chromatic aberration correction in Pancake optics, while [2] is a patent for a Pancake lens assembly. These works establish the baseline performance and form factor benefits of Pancake optics.
• Depth-variable Pancake VR: The paper notes that mechanically moving lenses is complicated and slow, and varifocal elements like LC lenses (which only support diopter adjustment) don't present 3D scenes and add complexity [2]. This highlights the limitations of existing approaches to address VAC in Pancake systems.

3.2.2. VAC-free Technologies

• Maxwellian View Display (Lin et al. [3]): This technology projects images directly onto the retina, ensuring always-in-focus retinal images regardless of accommodation. However, its primary drawback is a significantly restricted eyebox due to its dependence on a fixed pupil position. This limits user freedom of head and eye movement.
• Holographic Display (Gopakumar et al. [4]): Holographic displays record and reproduce wavefronts, retrieving phase information to create true-3D. The challenge lies in the coherence source requirements and complexity for near-eye displays. Reference [4] describes a significant breakthrough in integrating AI-driven digital holography into a compact waveguide with metasurface coupling gratings for AR glasses, but notes that an affordable source is still needed for VR.
• Light Field Display (LFD) (Javidi et al. [5]): LFDs use microlens arrays (MLAs) and microdisplays to generate computational focus cues by encoding parallaxes. Reference [5] provides a roadmap on 3D integral imaging (a form of LFD). While LFDs offer feasible hardware and minimized volume, direct application as near-eye displays faces challenges:
  • Resolution drop (Ding et al. [6]): Short-focal MLAs significantly magnify pixels, sharply reducing visual resolution. Reference [6] proposed an optical super-resolution method using incoherent synthetic apertures, but its full effect is limited to specific image depths.
  • FOV limitation (Wen et al. [7]): MLA aberration severely limits the FOV. Reference [7] explored large viewing angle integral imaging using a symmetrical compound lens array.
  • FOV expansion attempts (Huang and Hua [8]): Some approaches, like combining LFD with freeform prisms and tunable lenses [8], have expanded FOV for AR displays. However, freeform prism-based VR architectures tend to be bulkier than Pancake solutions, which contradicts the compactness goal.

3.2.3. LFD Resolution and Rendering

• Resolution enhancement (Yang et al. [9], Qin et al. [10]): LFDs inherently sacrifice resolution because display pixels encode both angular and spatial information. Mechanically dithering the microdisplay or MLA [9] can enhance resolution but may slow response times. The authors' previous work [10] details the FSC micro-LCD used in this paper, highlighting its high resolution capabilities (90.4 or 2.3Kx2.3K).
• Color Breakup Suppression (Wang et al. [11]): FSC-LCDs are prone to color breakup. The authors' prior research [11] addresses this with deep learning-based real-time driving to suppress color breakup and maintain high fidelity.
• LFD Modeling (Qin et al. [12]): Previous work by the authors [12] on image formation modeling and analysis of near-eye light field displays provides a foundation for understanding the aberration issues and FOV limitations of LFDs. This background informs the current paper's approach to mitigating FOV reduction.
• Image Rendering (Qin et al. [13]): Viewpoint-based projection is a typical method for EIA rendering, where each lenslet acts as a virtual camera. The authors have also reported an accelerated rendering method [13] for real-time computer-generated integral imaging light field displays.

3.2.4. Differentiation Analysis

Compared to existing Pancake VR solutions, this paper's core innovation is its ability to provide true-3D (addressing VAC) while retaining the compactness and wide FOV of Pancake optics. Traditional Pancake systems lack accommodation cues, leading to VAC. Mechanical or varifocal solutions for Pancake systems are either too slow, complex, or cannot generate multiple focal planes within a single scene.

Compared to other VAC-free technologies:

• Unlike Maxwellian view displays, this system aims for a wide eyebox by leveraging the large exit pupil of Pancake optics.

• Unlike holographic displays, it uses LFD technology, which is generally more affordable and less demanding regarding coherent light sources, making it more practical for VR.

  Compared to existing LFD approaches:

• The key differentiator is the novel integration with Pancake optics and, critically, the exploitation of the Pancake's telecentric optical path. Previous LFD implementations, when directly used as near-eye displays, suffer from severe FOV limitations due to MLA aberrations caused by oblique rays at large angles. This paper directly tackles this FOV issue by making the MLA operate with near-paraxial rays within the telecentric environment of the Pancake system.

• The use of a Field-Sequential-Color (FSC) micro-LCD further distinguishes it by providing high resolution and optical efficiency, overcoming common LFD resolution challenges without relying solely on complex optical super-resolution techniques or mechanical dithering.

  In essence, this paper integrates the strengths of LFD (true 3D) and Pancake optics (compactness, wide FOV) while systematically addressing their individual weaknesses (LFD's FOV limitation, Pancake's VAC).

3.3. Technological Evolution

The field of VR displays has evolved from simple stereoscopic displays (presenting two slightly different 2D images to each eye, causing VAC) towards true-3D displays that aim to mimic natural vision.

1. Early VR (Stereoscopic): Focused on basic immersion but suffered from VAC, leading to discomfort. Pancake optics emerged as a solution for compactness and FOV but inherited the VAC issue.

2. Addressing VAC with Dynamic Optics: Attempts included varifocal displays (mechanically moving lenses or using LC lenses), but these were often limited to single focal planes or slow response times.

3. Advanced True-3D Technologies:

   • Maxwellian displays offered focus cues but were hampered by restricted eyeboxes.
   • Holographic displays promised ultimate realism but faced challenges in compactness, efficiency, and cost for near-eye applications. Recent progress with waveguides and metasurfaces [4] shows promise for AR but VR still seeks more affordable solutions.
   • Light Field Displays (LFDs) emerged as a promising VAC-free alternative due to their ability to provide computational focus cues with relatively feasible hardware.

4. Challenges of LFDs in VR: When LFDs were directly applied to near-eye VR, new issues arose: resolution degradation (due to pixel magnification by MLAs) and severe FOV limitations (due to MLA aberrations from oblique rays). Efforts were made to enhance LFD resolution (e.g., optical super-resolution [6], dithering [9]) and FOV (e.g., freeform optics [8]), but often at the cost of bulkiness or complexity.

5. This Paper's Position: This work represents a significant step in the evolution of true-3D VR. It synergistically combines the best features of LFDs (VAC-free) and Pancake optics (compact, wide FOV). Crucially, it tackles the Achilles' heel of LFDs (FOV limitation) by ingeniously exploiting the telecentric optical path of Pancake optics. Furthermore, the integration of FSC micro-LCDs addresses the resolution and efficiency challenges inherent in LFDs and Pancake systems, respectively. This paper positions itself at the forefront of practical, high-performance true-3D VR display development by offering a compact, high-resolution, wide-FOV, and VAC-free solution.

4. Methodology

4.1. Principles

The core idea of this paper's method is to combine the true-3D capability of a Light Field Display (LFD) engine with the compactness and wide Field of View (FOV) of Pancake optics in a Virtual Reality (VR) headset. The theoretical basis rests on two main principles:

1. LFD for True-3D: The LFD engine generates intermediate images with computational focus cues. This means it can produce light rays that converge or diverge as if they were coming from real objects at different distances, thus providing the necessary accommodation cues to resolve the vergence-accommodation conflict (VAC). This is achieved by encoding parallaxes in an Elemental Image Array (EIA) displayed on a microdisplay and then projecting these through a microlens array (MLA).
2. Pancake Optics for Form Factor and Telecentricity: The Pancake module serves two critical functions:

   • Relaying Intermediate Images: It takes the intermediate images generated by the LFD engine and relays them to the user's eye, magnifying them and presenting them over a wide FOV.

   • Mitigating LFD Aberration: Crucially, Pancake optics inherently provides an object-space telecentric optical path. By replacing the Pancake's native microdisplay with the LFD engine's intermediate image, the MLA within the LFD engine operates with near-paraxial rays (rays close to the optical axis). This minimizes aberrations that typically limit the FOV of standalone LFDs when oblique rays pass through the MLA at large angles.

     Additionally, to address the inherent resolution sacrifice in LFDs, a field-sequential-color (FSC) micro-LCD is employed. This type of display removes color filter arrays, effectively tripling the spatial resolution and increasing optical efficiency, which is beneficial for the low-efficiency Pancake optics.

4.2. Core Methodology In-depth (Layer by Layer)

The proposed system integrates an LFD engine with a Pancake module. The LFD engine provides the true-3D capability, while the Pancake module handles the FOV expansion, compactness, and aberration mitigation.

4.2.1. Overall System Architecture

The overall system architecture is shown in Figure 3. The LFD engine (composed of a microdisplay and a microlens array, MLA) is placed before the Pancake module. The LFD engine generates intermediate images that possess computational depth cues. These intermediate images then act as the object for the Pancake module, which relays them to the observer's eye.

该图像是图示，展示了采用光场显示引擎的VAC-free Pancake模块的结构。图中显示了光场3D引擎与Pancake模块之间的关系，以及如何通过中间图像实现立体视觉效果。

Fig. 3. Proposed VAC-free Pancake using an LFD engine.

4.2.2. Microdisplay Panel for High Resolution

To address the inherent resolution sacrifice in LFDs (where pixels encode both angular and spatial information), the system adopts a 2.1-inch field-sequential-color (FSC) micro-LCD with a 2.3K-by-2.3K resolution [10].

• Principle of FSC-LCD: Unlike traditional subpixel-based LCDs (Figure 4(b)) which use a color filter array with red, green, blue subpixels to form full color, the FSC-LCD (Figure 4(c)) removes the color filter array. Instead, a mini-LED RGB backlight rapidly cycles through red, green, blue illumination. Due to the visual persistence of the human eye, these rapidly displayed chromatic subframes are fused, creating a full-color image.
• Benefits:

  • Tripled Resolution: The removal of subpixels means each physical pixel can display sequential full-color information, effectively tripling the perceived spatial resolution.

  • Multiplied Optical Efficiency: Color filters in traditional LCDs absorb a significant amount of light. Their elimination in FSC-LCDs leads to much higher optical efficiency, which is particularly advantageous for Pancake optics known for their low light throughput.

  • Color Breakup Suppression: The authors acknowledge color breakup as a potential issue but refer to their previous work [11] which used deep learning to suppress it, ensuring high fidelity.

    该图像是图表，展示了2.1英寸FSC微型LCD的细节（图4(a)）以及子像素LCD（图4(b)）和FSC-LCD（图4(c)）的工作原理，分析了其在空间和时间上对人眼信息处理的影响。

Fig.4. (a) The 2.1-inch FSC micro-LCD. (b) Subpixelbased LCD and (c) FSC-LCD.

4.2.3. Expanded FOV through Telecentric Path

A major challenge for LFDs directly used as near-eye displays is the aberration-induced FOV reduction. As shown in the simulation model of a directly near-eye LFD (Figure 5(a)), oblique beams passing through the MLA at large field angles produce severe aberrations, leading to a rapid degradation of the retinal Point Spread Function (PSF) (Figure 5(c)) and a sharp drop in visual resolution (Figure 5(b)). For instance, the FOV can be limited to less than 10 degrees (unilateral) where no image can be formed.

该图像是图表，展示了直接近眼光场显示（LFD）的仿真模型（a），不同视场随视觉分辨率变化的曲线（b），以及不同视场下的点扩散函数（PSF）示意图（c）。该数据显示，随着视场的增大，视觉分辨率显著降低，12°时分辨率降至0.7 PPD。

Fig. 5. (a) Simulation model of a directly near-eye LFD; (b) visual resolution decreased with field to demonstrate the FOV limited by aberration; (c) PSFs of different fields.

The proposed solution leverages the object-space telecentric optical path of Pancake optics to mitigate these aberrations.

• Telecentric Path in Pancake Optics: As illustrated in Figure 6, a typical Pancake model in Zemax simulation shows that the telecentric path is achieved by locating the aperture stop (which corresponds to the eye pupil in a near-eye system) at the image-space focal point of the Pancake lens module. This configuration ensures that chief rays from the object plane (where the LFD's intermediate image is formed) are parallel to the optical axis when entering the Pancake system.

• Benefit for LFD Engine: When the microdisplay of the Pancake is replaced by the intermediate image produced by the LFD engine, all lenslets within the MLA effectively operate with near-paraxial rays. This means the light rays passing through the MLA are close to the optical axis of each lenslet, regardless of the overall field angle. This significantly suppresses the aberrations that would typically arise from oblique rays in a non-telecentric LFD system, thereby enabling a large FOV with low aberrations.

  该图像是示意图，展示了FSC-LCD的物体空间传递路径及其如何通过Pancake光学组件生成中间图像，图中标示了不同颜色光线的传播轨迹。该设计旨在抑制由MLA引起的像差。

Fig. 6. The object-space telecentric path of Pancake and its benefit in suppressing the aberrations induced by oblique rays through MLA in the LFD engine.

4.2.4. Matching between Pancake and the LFD Engine

The Pancake module is usually optimized for a specific virtual image distance. When the LFD engine adjusts the virtual image distance (by changing the position of its intermediate image), residual aberrations may occur within the Pancake module. To ensure balanced image quality across different depth planes, a matching strategy is crucial.

• Pancake's MTF Variation: The Modulation Transfer Function (MTF) of the Pancake varies non-negligibly with the virtual image distance (image depth), as simulated using Zemax (Figure 7(a)). The MTFs are acquired by placing the microdisplay at different positions relative to the Pancake's native object plane.

• LFD's Image Quality Variation: The LFD engine itself has varying image quality. The highest resolution is achieved at the MLA's native image plane, known as the Central Depth Plane (CDP). As the Reconstructed Depth Plane (RDP) moves away from the CDP, the MLA's defocus reduces image quality. Additionally, transverse magnification affects the voxel size on the RDP.

  The LFD-determined MTF is given by Equation (1): $$M T F = \Big \{ \tilde { P } ( s , t ) \otimes \tilde { P } ( s , t ) \Big \} \cdot \mathrm { s i n c } \Bigg ( \frac { g } { p \cdot l _ { R D P } } \Bigg )$$ where $$\tilde { P } ( s , t ) = P ( s , t ) \mathrm { e x p } \Bigg [ i k \Bigg ( \frac { 1 } { l _ { C D P } } - \frac { 1 } { l _ { R D P } } \Bigg ) \frac { s ^ { 2 } + t ^ { 2 } } { 2 } \Bigg ]$$ Here:

• MTF: The Modulation Transfer Function of the LFD engine.

• $\tilde{P}(s, t)$: The pupil function of the MLA with an additional phase term accounting for defocus.

• s, t: Pupil coordinates on the MLA.

• $\otimes$: Denotes convolution.

• $g$: Represents the defocus amount or the distance between the MLA and the microdisplay. (Note: The diagram Figure 7(b) seems to indicate $g$ as the distance from the MLA to the reconstructed depth plane, $l_{RDP}$, or the CDP, $l_{CDP}$, but the formula implies it relates to the physical separation. In LFD contexts, $g$ often refers to the gap between MLA and microdisplay.)

• $p$: The pixel pitch of the microdisplay.

• $l_{CDP}$: The distance from the MLA to the Central Depth Plane (native image plane of the MLA).

• $l_{RDP}$: The distance from the MLA to the Reconstructed Depth Plane (where the 3D image is rendered).

• $i$: The imaginary unit.

• $k$: The wave number, $k = 2\pi/\lambda$, where $\lambda$ is the wavelength of light.

• $\mathrm{sinc}(x) = \sin(\pi x) / (\pi x)$: The sinc function, which arises from the diffraction limit and pixel sampling effects in LFDs.

• The first term $\Big \{ \tilde { P } ( s , t ) \otimes \tilde { P } ( s , t ) \Big \}$ represents the optical transfer function (OTF) for the MLA, derived from the autocorrelation of the pupil function. The exponential term within $\tilde{P}(s, t)$ accounts for the wavefront curvature due to defocus when the RDP is not at the CDP.

• The sinc term $\mathrm { s i n c } \Bigg ( \frac { g } { p \cdot l _ { R D P } } \Bigg )$ accounts for the magnification and sampling effects of the pixels at the RDP.

  Figure 7(b) illustrates the image quality matching concept. The blue solid and dashed lines represent the MTF of the Pancake at different object planes. The red line represents the MTF of the LFD engine, which is highest at its CDP and decreases as the RDP moves away. The compromised configuration shown suggests that the LFD engine's CDP is intentionally positioned at a Pancake object plane that might not be the Pancake's absolute optimal point, but rather a point that allows for balanced image quality across the range of depths produced by the LFD engine.

  该图像是图表，展示了不同虚拟图像距离下调制传递函数（MTF）与空间频率的关系（图7(a)），以及在各种条件下的分辨率变化情况（图7(b)）。图中展示的MTF曲线反映了从0.1m到2m等距离下的性能。

Fig. 7. (a) MTF varying with the virtual image distance

4.2.5. Image Rendering for the LFD Engine

The depth of the Reconstructed Depth Plane (RDP) is adjusted by appropriately rendering the Elemental Image Array (EIA).

• Viewpoint-based Projection: A typical rendering approach is viewpoint-based projection. In this method, each lenslet in the MLA is conceptually treated as a virtual camera. These virtual cameras capture the target 3D scene from slightly different perspectives, and their outputs form the individual elemental images that constitute the EIA.
• Light Ray Manipulation: When this EIA is displayed on the microdisplay, the MLA then manipulates the directions of the light rays such that they inversely project the elemental images to reconstruct the 3D scene at a specific depth plane (RDP). By altering how the EIA is rendered (i.e., changing the perspective or scale of the elemental images), the effective RDP can be shifted, thus providing computational focus cues for different depths.
• Accelerated Rendering: The authors mention their previous work [13] on an accelerated rendering method, which is important for real-time performance in VR applications.

5. Experimental Setup

5.1. Datasets

The paper does not use traditional datasets in the machine learning sense. Instead, it involves physical optical experiments using a prototype VR headset. The "data" in this context refers to 3D scenes rendered for the light field display engine and the optical measurements obtained from the prototype.

• Sample Scene: For the experimental demonstration, a sample scene containing two objects located at two different depths was rendered as an Elemental Image Array (EIA). This scene allows for verification of the true-3D capability by showing that the display can correctly focus on objects at distinct depths.
  • One object is intended to be reconstructed in the foreground on the Central Depth Plane (CDP) of the LFD.
  • The second object is intended to be reconstructed in the background. The EIA for this sample scene is shown in Figure 8(b).

5.2. Evaluation Metrics

The paper evaluates the system's performance using qualitative and quantitative metrics, primarily focusing on image quality, true-3D perception, and Field of View (FOV).

5.2.1. True-3D Capability / Focus Cues

• Conceptual Definition: This metric qualitatively assesses whether the system can correctly present accommodation cues for objects at different depths, thereby resolving the vergence-accommodation conflict (VAC). It is evaluated by observing if objects at different virtual distances can be brought into sharp focus by a camera (mimicking the human eye's accommodation) without altering the display system itself.
• Mathematical Formula: No explicit mathematical formula is provided as it is a qualitative assessment based on observation of focal planes.
• Symbol Explanation: Not applicable for this qualitative metric.

5.2.2. Image Quality (Sharpness / Blur)

• Conceptual Definition: This metric qualitatively assesses the sharpness of the reconstructed 3D images at different depth planes. It involves focusing a camera on specific objects within the rendered scene and observing which objects appear sharp and which appear blurred. A sharp image indicates good image quality at that specific Reconstructed Depth Plane (RDP).
• Mathematical Formula: No explicit mathematical formula is provided, as it's primarily a visual assessment. However, the theoretical basis for image quality is quantified by the Modulation Transfer Function (MTF) discussed in the methodology section, which measures the system's ability to transfer contrast at different spatial frequencies.
  • As discussed in Section 4.2.4, the MTF is given by: M T F = \Big \{ \tilde { P } ( s , t ) \otimes \tilde { P } ( s , t ) \Big \} \cdot \mathrm { s i n c } \Bigg ( \frac { g } { p \cdot l _ { R D P } } } \Bigg ) where $$\tilde { P } ( s , t ) = P ( s , t ) \mathrm { e x p } \Bigg [ i k \Bigg ( \frac { 1 } { l _ { C D P } } - \frac { 1 } { l _ { R D P } } \Bigg ) \frac { s ^ { 2 } + t ^ { 2 } } { 2 } \Bigg ]$$
    • MTF: Modulation Transfer Function.
    • $\tilde{P}(s, t)$: Pupil function with defocus term.
    • s, t: Pupil coordinates on the MLA.
    • $\otimes$: Convolution.
    • $g$: Distance between MLA and microdisplay.
    • $p$: Pixel pitch.
    • $l_{CDP}$: Distance from MLA to Central Depth Plane.
    • $l_{RDP}$: Distance from MLA to Reconstructed Depth Plane.
    • $i$: Imaginary unit.
    • $k$: Wave number.
    • $\mathrm{sinc}(x)$: Sinc function.
• Symbol Explanation: See Section 4.2.4 for detailed explanation of symbols in the MTF formula.

5.2.3. Field of View (FOV)

• Conceptual Definition: The Field of View (FOV) is the angular extent of the observable world at any given moment. In VR displays, a larger FOV contributes to a more immersive experience. It is measured in degrees.
• Mathematical Formula: While not explicitly provided in the paper, FOV is typically calculated using the display's dimensions, the focal length of the optics, and the eye relief. For a VR system, it can be estimated using the dimensions of the projected image and the effective viewing distance (e.g., the focal length of the capturing camera). In this paper, FOV is measured using the camera's specifications and the picture size on the image sensor. For a camera lens, the FOV can be approximated by: $ \mathrm{FOV} = 2 \cdot \arctan \left( \frac{D}{2 \cdot f} \right) $
  • $\mathrm{FOV}$: The Field of View in degrees.
  • $D$: The dimension (e.g., width or height) of the image sensor or the captured picture size on the sensor.
  • $f$: The focal length of the camera lens. The angular FOV can be calculated for horizontal, vertical, or diagonal dimensions.
• Symbol Explanation:
  • $\mathrm{FOV}$: Field of View.
  • $D$: Dimension of the image sensor or captured image.
  • $f$: Focal length of the camera.
  • $\arctan$: Arctangent function.

5.2.4. Optical Track

• Conceptual Definition: The optical track refers to the physical length or depth that the optical components occupy in the system. A shorter optical track is desirable for compact and lightweight VR headsets. The paper measures the additional optical track introduced by the LFD engine.
• Mathematical Formula: Not applicable, as it's a direct physical measurement.
• Symbol Explanation: Not applicable.

5.3. Baselines

The paper implicitly compares its proposed method against several existing or theoretical baselines:

• Conventional Pancake VR Headsets (fixed virtual image distance): This is the primary baseline for compactness and wide FOV. The paper's system aims to maintain these benefits while overcoming the VAC inherent in these headsets. The limitation of these systems is the VAC itself, which the paper aims to solve.
• Direct Near-Eye LFDs (LFD engine used alone): This serves as a baseline for true-3D capability. However, direct LFDs suffer from significantly limited FOV (e.g., less than 10 degrees unilateral as shown in Figure 5) and resolution degradation due to MLA aberrations and pixel magnification. The paper's method explicitly addresses and overcomes these limitations by integrating with Pancake optics.
• Other VAC-free Technologies:

  • Maxwellian view displays: Offer true-3D but have a restricted eyebox. The paper aims for a wide eyebox compatible with VR.

  • Holographic displays: Offer true-3D but are complex, expensive (requiring coherent sources), and challenging for compactness in VR. The paper's LFD-based approach is presented as a more affordable and practical alternative.

  • LFDs with freeform optics [8]: These can achieve expanded FOV but often result in a bulkier volume compared to Pancake solutions, which the paper aims to avoid.

    The paper's success is demonstrated by combining the advantages of these baselines (true-3D from LFD, compactness/FOV from Pancake) while mitigating their respective drawbacks.

6. Results & Analysis

6.1. Core Results Analysis

The paper details the experimental results of the prototype, focusing on its true-3D capability, image quality at different depths, and Field of View (FOV).

6.1.1. Experimental Setup and Optical Track

The prototype was constructed using:

• A 1500-ppi FSC micro-LCD based on a mini-LED backlight.

• A microlens array (MLA) with a 1-mm lens pitch.

• A commercial Pancake module.

  As shown in Figure 8(a), the experimental setup positions the LFD engine and the Pancake module. The designed object plane of the Pancake module was placed 6 mm from the LFD's Central Depth Plane (CDP) to achieve optimal image quality (this specific distance is a result of the image quality matching strategy discussed in Section 4.2.4). The LFD engine introduced an additional optical track of 2.1 cm. This is considered acceptable for near-eye displays, indicating that the integration maintains a relatively compact form factor.

该图像是示意图，展示了实验设置（a）、样本场景的ElA（b），以及在两个深度平面上的重建图像（c和d），并标明测得的视场（FOV）为68.6°。

Fig. 8. (a) Experimental setup; (b) ElA of the sample scene; (c) and (d) reconstructed images on two depth planes and the measured FOV.

6.1.2. True-3D Capability Demonstration

To demonstrate the true-3D feature and computationally adjustable virtual image distances, a sample scene was used (Figure 8(b)). This Elemental Image Array (EIA) contained two objects located at two distinct depths.

• Object 1 (Foreground): This object was rendered to be reconstructed in the foreground, coinciding with the LFD's Central Depth Plane (CDP). The first intermediate image plane was 9.7 mm from the MLA.

• Object 2 (Background): This object was rendered to be reconstructed in the background. The second image plane was positioned 16 mm from the MLA.

  A smartphone camera (with a focal length of 5.5 mm) was used to capture virtual images through the Pancake module, mimicking how a human eye would accommodate.

• Focusing on Foreground Object (Figure 8(c)): When the camera was focused on the object intended for the foreground (reconstructed on the CDP), Figure 8(c) shows that this object exhibited sharp details. Conversely, the object intended for the background appeared blurred, and its subviews were visible (a characteristic blur for out-of-focus elements in light field displays). This clearly demonstrates that the system provides focus cues for the foreground.

• Focusing on Background Object (Figure 8(d)): When the camera's focus was adjusted to the object intended for the background, Figure 8(d) shows that this object became sharper. Simultaneously, the out-of-focus object in the foreground became blurred. The paper notes that even though the background object was reconstructed with slightly out-of-focus beams from the LFD perspective (as it's not on the CDP), this intermediate RDP was intentionally placed on a Pancake object plane that had a better MTF according to the image quality matching strategy (Section 4.2.4). This optimized placement ensured good image quality for both depths.

  These results verify computationally adjustable virtual image distances, successfully demonstrating the true-3D feature where accommodation cues are provided, resolving the VAC.

6.1.3. Field of View (FOV) Measurement

The Field of View (FOV) was measured using the camera's specifications and the picture size on the image sensor.

• The measured FOV was 68.6 degrees.
• This FOV is described as close to the Pancake module's original FOV. This indicates that the LFD engine integration did not significantly degrade the native FOV capability of the Pancake optics.
• Crucially, this 68.6 degrees FOV is significantly larger than the LFD engine used alone. As noted in the methodology (Figure 5), a standalone LFD could be limited to under 10 degrees (unilateral) due to MLA aberrations. This result strongly validates the effectiveness of using the Pancake's telecentric optical path to mitigate LFD aberrations and expand the FOV.

6.2. Data Presentation (Tables)

The paper primarily presents its findings through images demonstrating the optical output and quantitative measurements discussed in the text, rather than through structured data tables. There are no tables in the paper to transcribe.

6.3. Ablation Studies / Parameter Analysis

The paper does not explicitly present ablation studies or detailed parameter analysis in the form of separate experiments. However, elements of such analysis are implicitly part of the methodology:

• Image Quality Matching (Section 4.2.4): The discussion about MTF varying with virtual image distance for the Pancake and the LFD engine, and the decision to find a compromised configuration where the LFD engine's CDP is placed at a relatively worse object plane of the Pancake, serves as a form of parameter optimization. This analysis ensures balanced image quality across multiple depth planes rather than optimizing for a single, perfect depth. This demonstrates an understanding of how different components' characteristics (Pancake's optimal focus vs. LFD's CDP) interact and how parameters (like the distance between LFD CDP and Pancake object plane) are tuned.

• FOV Mitigation: The comparison of the achieved 68.6 degrees FOV with the theoretical FOV limitation of a standalone LFD (less than 10 degrees unilateral, as depicted in Figure 5) acts as an implicit ablation study. It shows the telecentric path of the Pancake is the crucial component enabling the wide FOV for the LFD engine, demonstrating its effectiveness in mitigating MLA aberrations.

• Microdisplay Choice: The selection of the FSC micro-LCD and its benefits (tripled resolution, multiplied optical efficiency) is a design choice backed by prior research [10, 11]. While not an ablation study in this paper, it implies that other microdisplay types (e.g., traditional subpixel LCDs) would yield inferior results regarding resolution and efficiency.

  These elements, while not framed as formal ablation studies, highlight how specific design choices and parameter tuning were critical to achieving the overall performance of the proposed system.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper successfully demonstrates a novel true-3D VR headset by integrating a Light Field Display (LFD) engine with Pancake optics. The core achievement is overcoming the vergence-accommodation conflict (VAC) through the LFD's computational focus cues, while simultaneously retaining the compactness and wide Field of View (FOV) offered by Pancake optics. Key to this integration is the exploitation of the Pancake's object-space telecentric optical path, which effectively mitigates the aberration-induced FOV reduction typically found in LFDs. Furthermore, the use of a field-sequential-color (FSC) micro-LCD ensures high resolution and optical efficiency. The prototype demonstrates sharp images at different depth planes with an impressive FOV of 68.6 degrees, sacrificing only an acceptable additional optical track of 2.1 cm.

7.2. Limitations & Future Work

The paper implicitly and explicitly mentions a few limitations and areas for improvement, which can be seen as directions for future work:

• Additional Optical Track: The system introduces an additional optical track of 2.1 cm. While deemed acceptable, further efforts could aim to reduce this to achieve even greater compactness.
• Image Quality Matching Complexity: The image quality matching strategy between the LFD and Pancake involves compromises to balance quality across multiple depth planes. Optimizing this matching to achieve consistently high image quality over a broader range of depths remains a challenge. The paper notes that the LFD's CDP is intentionally placed on a relatively worse object plane of the Pancake to achieve balance, implying there's still a trade-off.
• Accurate Modeling of Commercial Pancake: The paper mentions the difficulty in accurately modeling the commercial Pancake. More precise modeling could lead to better optimization of the LFD-Pancake interface and potentially further improve overall image quality and performance.
• Color Breakup in FSC-LCDs: Although the authors refer to prior work [11] on suppressing color breakup using deep learning, this remains an inherent challenge for FSC-LCDs and may require continuous refinement to ensure a flawless visual experience.
• Resolution and Efficiency Trade-offs: While the FSC micro-LCD significantly boosts resolution and efficiency, LFDs generally still have inherent resolution sacrifice due to angular information encoding. Future work could explore more advanced super-resolution techniques or display technologies that further enhance spatial resolution without compromising depth cues.
• Dynamic Response and Rendering Speed: The paper mentions an accelerated rendering method [13] for the EIA. For truly seamless VR experiences, maintaining real-time rendering and display response speeds as 3D scenes become more complex is crucial.

7.3. Personal Insights & Critique

This paper presents a highly practical and well-engineered solution to a fundamental problem in VR displays. The core idea of combining an LFD engine with Pancake optics is elegant, as it leverages the strengths of both technologies while using the Pancake's telecentric path to specifically address the FOV limitation of LFDs. This "two birds with one stone" approach is a significant contribution.

My personal insights are:

• Synergistic Design: The paper excels in its synergistic design. Instead of trying to fix LFDs in isolation or Pancake optics in isolation, it identifies how the inherent properties of one (Pancake's telecentricity) can naturally mitigate a major drawback of the other (LFD's FOV aberration). This holistic approach to system design is often more effective than incremental improvements to individual components.
• Practicality for VR: The focus on compactness (acceptable 2.1 cm additional optical track), wide FOV (68.6 degrees), and VAC-free true-3D makes this a highly relevant solution for next-generation VR headsets. The choice of FSC micro-LCD is also a smart move, addressing both resolution and efficiency which are critical for Pancake systems.
• Potential for Mass Adoption: Unlike holographic displays which are still far from mass market due to complexity and cost, this LFD-Pancake hybrid seems more amenable to practical implementation, potentially lowering the barrier for widespread true-3D VR experiences.
• Unverified Assumptions/Areas for Improvement:

  • Eyebox Size: While FOV is measured, the paper doesn't explicitly discuss the eyebox size. A wide FOV is good, but if the eyebox (the region where the user's eye can be placed while still seeing the full image) is small, it can still lead to user discomfort. Pancake optics typically have a decent eyebox, but the LFD integration might introduce new constraints that warrant further investigation.

  • Light Field Rendering Fidelity: The paper mentions EIA rendering and accelerated methods. The fidelity of the reconstructed light field (e.g., depth resolution, smoothness of focus cues) and the computational cost of real-time rendering for complex scenes are crucial. More detailed analysis of these aspects would strengthen the paper.

  • Chromatic Aberration: While FSC-LCDs avoid spatial chromatic aberration from subpixels, chromatic aberration can still arise from the lenses themselves. Reference [1] discusses chromatic aberration correction in Pancake optics. It would be beneficial to explicitly discuss how chromatic aberrations are managed in this combined system, especially with the FSC backlight.

  • Perceived Resolution (PPD): While 2.3Kx2.3K on 2.1-inch FSC micro-LCD is high, the final perceived resolution in pixels per degree (PPD) is a critical VR metric. Providing this value would offer a more complete picture of the visual quality.

    The methods and conclusions of this paper could potentially be applied to augmented reality (AR) systems, particularly those aiming for true-3D overlays without VAC. The telecentric principle for LFD aberration mitigation is broadly applicable wherever LFDs are used with relay optics. This work paves the way for more comfortable and immersive VR experiences, pushing the boundaries of near-eye display technology.