Abstract
A generative framework with supporting code library evaluates and optimizes disparate geo-experimentation methodologies on a finite sample of historical data. The evaluation process utilizes a multivariate moving block bootstrapping algorithm that generates synthetic histories while preserving cross-sectional dependencies to augment data in order to run hundreds of simulations prior to experiment launch. By simulating experiments across various generated datasets, the framework estimates the statistical validity, bias, and sensitivity of different experimental designs allowing users to compare outcomes from analysis methods (e.g. time-based-regression vs. synthetic controls) and runtime parameters. This approach provides a consistent basis for optimizing experimental designs across a search space of multiple methodologies on a single dataset, accommodating complex constraints and multi-cell test configurations. Keywords: geo-experimentation, block bootstrapping, synthetic controls, time-based regression, minimum detectable effect, automated design exploration.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Bailey, Sam and Liu, Wincy, "A Generative Framework for Evaluating and Optimizing Geographic Experimentation Methodologies", Technical Disclosure Commons, (May 20, 2026)
https://www.tdcommons.org/dpubs_series/10185