Abstract
The disclosed technology employs an automated pipeline to dynamically insert context-aware digital advertisements into video streams. A vision-language model (VLM) processes decoded video frames to perform spatiotemporal affordance and obstacle detection, identifying semantic regions suitable for ad placement. If an existing object obstructs the placement area, a physics-aware preparation layer performs 3D semantic object translation. This object translation directly morphs an existing unbranded scene object into a branded alternative while preserving the original scene physics and lighting. The system generates a joint multimodal embedding vector representing the immediate semantic and audio context of the video. This vector guides an image generation model to dynamically synthesize a multimodal ad asset, such as a logo and/or dynamic, contextually generated data overlays, tailored to the context. Finally, a generative video in-painting and neural rendering architecture synthesizes new pixels to incorporate the ad, maintaining temporal stability, relighting, and deformation for a natural appearance within the scene.
Keywords: vision language model (VLM), image generation large language model (LLM), video diffusion architecture, multimodal embedding vector, spatiotemporal affordance, obstacle detection, physics-aware removal, dynamic ad asset synthesis, semantic context analysis, neural rendering, ad asset, ad placement
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Kakumani, Lakshmi Venkatesh, "Automated Context-Aware In-Video Advertising Through Multimodal Generative Models", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/10792