Abstract
Users capture screenshots of their devices while running particular applications for various purposes such as bookmarking a specific interaction or state of the application. However, currently, finding the correct screenshot at a later time is challenging since screenshots are not amenable to searching. Further, the non-interactive nature of screenshots makes them inadequate for accessing underlying content such as embedded links or multimedia resources. This disclosure describes techniques to automatically enhance screenshots by capturing relevant contextual metadata that can enable semantic indexing of screenshots. The indexing facilitates search and retrieval and can support resuming the underlying application in the same state as at the time of screenshot capture. Semantic indexing can be performed using natural language understanding, image processing, optical character recognition, etc. The semantic index is usable to retrieve screenshots that match a user query and to automatically invoke the relevant application and resume it from the state captured in the screenshot.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Tran, Duc-Hieu, "Screenshots Enriched with Metadata to Enable Bookmarking and Semantic Retrieval", Technical Disclosure Commons, (March 22, 2023)
https://www.tdcommons.org/dpubs_series/5750