Multi-illumination-interfered Neural Holography with Expanded Eyebox

Xinxing Xia1*, Pengfei Mi1, Yiqing Tao1, Xiangyu Meng2, Wenbin Zhou2, Yingjie Yu1, Yifan (Evan) Peng2*
1Shanghai University ,2The University of Hong Kong
IEEE Transactions on Visualization and Computer Graphics (ISMAR), 2025
*Corresponding author
Paper Supplementary Code - Coming soon...
teaser

Top-left: Conceptual holographic display. Fiber laser outputs pass through BS, LP, and CL to illuminate the SLM. Top-right: Eyebox expansion schematic. Multi-angle illumination creates adjacent viewpoints on the eyepiece’s focal plane. Pupil shifts align viewpoints at center (blue), edge (yellow), or both simultaneously (purple). Bottom: Experimental single vs. dual illumination, with/without CITL. First row: Pupil centered at (0 mm, 0 mm). Dual illumination achieves clarity even at single-source spectrum edges. Second row: Single illumination shows limited quality. Dual maintains fidelity within combined spectral range, expanding eyebox. (Dual-light configuration simplified in experiments.)

Abstract

Holography has immense potential for near-eye displays in the realms of virtual and augmented reality (VR/AR), offering natural 3D depth cues through wavefront reconstruction. However, a critical challenge is balancing the field of view (FOV) with the eyebox, fundamentally constrained by the étendue limitation of current hardware. Additionally, holographic image quality is often compromised due to discrepancies between actual wave propagation behavior and simulation models. This study addresses these challenges by expanding the eyebox through the use of multi-angle illumination, and by enhancing image quality with end-to-end pupil-aware hologram optimization. Further, energy efficiency is significantly improved by incorporating higher-order diffractions and pupil constraints into the simulation. We explore a Pupil-HOGD optimization algorithm for multi-angle illumination and validate it with a dual-angle holographic display prototype. Integrated with camera calibration and tracked eye position, the developed Pupil-HOGD algorithm can improve image quality and expand the eyebox by 50% in the horizontal direction. We envision the proposed approach paves the way for extending the space-bandwidth product (SBP) of holographic displays, enabling broader applications in immersive, high-quality visual computing environments.

Method

We use a viewpoint spacing smaller than the pupil size to create a compact, continuous eyebox to ensure consistent, aligned holograms in near-eye displays. To address image aliasing introduced by the reduced spacing, a multi-angle illumination propagation framework is developed. This framework explicitly incorporates the dynamic pupil position and size to maintain consistent image quality across different viewpoints.

MY ALT TEXT
Left: Schematic of propagation under multi-angle light illumination. The SLM is illuminated by the superposition of plane waves incident from four distinct angles. Four light sources hit the SLM plane on P0, and to the target plane on P1, P2, P3, and P4. The parallel light waves, corresponding to their respective angles of inclination, superimpose at the same overlapping position on the target plane to reconstruct the image. Right: Without algorithmic optimization, under horizontal dual-sources, the pupil position is moved along X, Y, and Z axes for simulations. When positioned at the origin, image overlap occurs. With X-axis pupil displacement, the result is similar to single-light-source imaging. Displacement along the Y-axis significantly degrades image quality due to a lack of vertical light source expansion. Z-axis displacement creates image overlap with arc-shaped artifacts due to aperture effects.
MY ALT TEXT
Pipeline of Pupil-HOGD. Each wavefront is depicted with its amplitude (black) and phase (blue). Each dimension's spatial/frequency domain range is indicated, where N is the number of SLM pixels along the dimension, and p is the pixel pitch. The Fourier transform of the SLM wavefront is repeated in the frequency domain to consider high-order diffraction effects. This is then multiplied by a propagation kernel with attenuation and a pupil mask to generate the target wavefront. Finally, the loss function, comparing the target and the simulated images, is computed, and the phase pattern is updated through back-propagation using gradient descent.


Results

We conducted simulations to evaluate image quality under varying parameters, including different tilt angles and numbers of light sources. High image quality is preserved with two distinct light source angles, which significantly enlarges the eyebox. We built an experimental prototype using laser light sources from two distinct angles to validate proposed method.

MY ALT TEXT
Left: Display prototype. Two fiber laser (FL) outputs pass through a beam splitter (BS), linear polarizer (LP), and collimating lens (CL) to illuminate the SLM. The target image is magnified by the eyepiece and captured by an industrial camera, simulating the human eye. The camera is mounted on a 3D moving stage to measure the eyebox. Right: Schematic of the Pupil-HOGD with CITL calibration process. The light field in the target plane is formed by the superposition of multi-angle illumination. A camera with a lens simulates the human eye, with its aperture representing the pupil size, and captures the reconstructed image as the feedback. The loss between the amplitude of the captured image and the target image is computed, and the phase is iteratively updated through back-propagation until convergence. Pupil size is incorporated in the back-propagation as a filter M(fx,fy) in the frequency domain.
MY ALT TEXT
Comparison of hologram optimization algorithms across various pupil positions. Each row corresponds to pupil positions at (-1.5 mm, 0 mm), (0 mm, 0 mm), and (1.5 mm, 0 mm). The first column visualizes the pupil position in the frequency domain, while columns two to four display the results of the SGD, HOGD, and Pupil-HOGD algorithms, respectively. The fifth column showcases the optimized hologram using the Pupil-HOGD algorithm. The SGD algorithm yields the lowest PSNR due to neglecting higher-order diffraction. In contrast, the HOGD algorithm slightly improves but still ignores the pupil’s low-pass filtering effect. The Pupil-HOGD algorithm outperforms the others by accounting for the constraining effect of pupil position on light source utilization at various angles.
MY ALT TEXT
Experimental results of HOGD, Pupil-HOGD, and Pupil-HOGD with CITL algorithm at various pupil positions under single-angle illumination. The pupil positions range from -2~mm to 2~mm in 1~mm intervals. The first column shows the frequency-domain filtering corresponding to different pupil positions. The second column displays the HOGD result, where pupil constraints are not incorporated into the algorithm. The third column presents the Pupil-HOGD results, which include the pupil's constraints in the optimization process. The fourth column illustrates the Pupil-HOGD-CITL result, where camera calibration further enhances Pupil-HOGD performance

BibTeX


    @article{MultiangleHolography2025,
    author={Xia, Xinxing and Mi, Pengfei and Tao, Yiqing and Meng, Xiangyu and Zhou, Wenbin and Yu, Yingjie and Peng, Yifan},
    journal={IEEE Transactions on Visualization and Computer Graphics},
    title={Multi-illumination-interfered Neural Holography with Expanded Eyebox},
    year={2025},
    volume={},
    number={},
    pages={1-10},
    keywords={Lighting;Pupils;Optimization;Image quality;Diffraction;Light fields;Three-dimensional displays;Laser beams;Holography;Light sources;Holographic display;Multi-illumination;Eyebox;Camera-in-the-loop calibration},
    doi={10.1109/TVCG.2025.3616793}
    }