Abstract
Processing continuous video streams can be computationally expensive and may lead to information loss with fixed-rate sampling techniques. A content-aware filtering system for adaptive video sampling may utilize a set of parallel analysis components to examine a video stream in real-time for factors such as scene changes, motion, semantic content, and latent feature similarity. Based on the aggregated output of these components, the system can select information-rich keyframes for downstream processing. A selected keyframe may be paired with a temporal timestamp. This approach may reduce data volume and computational load compared to fixed-rate sampling, while seeking to preserve temporal details that can be beneficial for certain applications, for example, those that use live video input for conversational multimodality models.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Lai, Larry; Fan, Ruixi; Wang, Lan; and Tang, Youbao, "Adaptive Video Sampling Using Multi-Factor Analysis for Multimodal AI", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/10435