TransLight: Image-Guided Customized Lighting Control with Generative Decoupling

Zongming Li1 2, Lianghui Zhu1, Haochen Sheng2, Longjin Ran2, Wenyu Liu1, Xinggang Wang1,
1School of Electronic Information and Communications, Huazhong University of Science and Technology, 2vivo Mobile Communication Co., Ltd,
Paper arXiv

Transfer light effect from a reference image to a user input image.

Abstract

Most existing illumination-editing methods struggle to jointly offer customized lighting control and preserve content integrity, limiting their effectiveness especially in transferring complex light effects from a reference to a target image in portrait photography. To address this problem, we propose TransLight, a novel framework that enables high-fidelity and high-freedom transfer of geometric-structured light effects such as Tyndall beams and specular highlights. Extracting light effects from the reference image is the most critical and challenging step, as real-world lighting contains complex geometric structures tightly coupled with image content. To achieve this, we propose Generative Decoupling, using two fine-tuned diffusion models to accurately separate image content and lighting, and create a new million-scale dataset of image–content–light triplets. We then adopt IC-Light as the generative model, training it on these triplets with the reference lighting image as an additional conditioning signal. The resulting model enables customized and natural transfer of diverse light effects. Notably, by fully disentangling light effects from reference images, our generative decoupling strategy gives TransLight highly flexible illumination control. Experiments show that TransLight successfully transfers geometric-structured lighting effects across diverse images in portrait photography, offering more customized control than existing methods and charting new directions in illumination harmonization and editing.

Generative Decoupling

We use two diffusion models to decouple the image content and light effects, which we refer to as generative decoupling.

Interpolation end reference image.

Visualizations of Generative Decoupling

Interpolation end reference image.
Interpolation end reference image.




TransLight

Overall framwork. (a) We fine-tune two diffusion models initialized with the weights of IC-Light. During training, the input synthesized image IS is the direct addition result of no light image I and light material image L. (b) This pipeline involves selecting relevant data using a vision language model, decoupling image content and lighting via light removal and extraction models, and filtering out poor generation results. (c) Our TransLight training consists of two stages: LoRA fine-tuning for content-preserving lighting editing and ControlNet training to inject light effects.

Interpolation end reference image.




Triplets

We obtain 1 million triplets using our data construction pipeline. We categorize all triplets in our dataset into nine light-effect types: 1.Light Beam in Tyndall Effect; 2.Dappled Light; 3.Backlighting Effect; 4.Iridescent Halo; 5.Lens Flare; 6.Bright Light Source; 7.Gleaming Water Reflection; 8.Projected Light Patch; 9.Volumetric Light Rays.

Interpolation end reference image.
Interpolation end reference image.




Visualizations of TransLight

Our TransLight tackles this challenging task through generative decoupling. We extract the light effect from the reference image using our fine-tuned model and composite it onto the target image, enabling flexible control over position and direction for realistic, high-freedom results.

Interpolation end reference image.
Interpolation end reference image.
Interpolation end reference image.

BibTeX

@article{translight,
  author    = {Li, Zongming and Zhu, Lianghui and Sheng, Haochen and Ran, Longjin and Liu, Wenyu and Wang, Xinggang},
  title     = {TransLight: Image-Guided Customized Lighting Control with Generative Decoupling},
  journal   = {ICML},
  year      = {2026},
}