Abstract
Regression tests are conducted on new editions of an app to ensure backward compatibility with previous editions. Due to testing complexity, live, end-to-end regression tests are relatively rare. Therefore, regression tests may not truly reflect software behavior from a user perspective. This disclosure describes the use of statistical significance techniques to regression-test new editions of an app. Two editions are given similar inputs and configurations, and the user-facing outputs, e.g., screenshots of viewports of the user interface, are subjected to statistical significance tests that output a likelihood that the two outputs arise from the same source. Insubstantial differences are not presented as errors, while genuine differences between the two editions are flagged as potential backward-compatibility issues. Live production modules are tested such that testing is end-to-end and reflects user experience. The statistical, live, and end-to-end properties of the described testing techniques ease testing: adding a test is as simple as logging and subjecting the test metrics to the described statistical testing.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Kennell, Grant; Wang, Fei; Burns, Christopher Charles; Li, Zening; Lee, Hsin-Pei; Hua, Jessica; Cunningham, Sam; Cheng, Shu-Wei; Krasichkov, Eugene; and Williams, Lee, "Statistical Techniques of Regression Testing to Validate the Correctness of New App Editions", Technical Disclosure Commons, (June 26, 2024)
https://www.tdcommons.org/dpubs_series/7136