Paper: Robust Causal Inference for Incremental Return on Ad Spend with Randomized Geo Experiments

Evaluating the incremental return on ad spend (iROAS) of a prospective online marketing strategy—that is, the ratio of the strategy’s causal effect on some response metric of interest relative to its causal effect on the ad spend—has become progressively more important as advertisers increasingly seek to better understand the impact of their marketing decisions. Although randomized ‘geo experiments’ are frequently employed for this evaluation, obtaining reliable estimates of the iROAS can be challenging as oftentimes only a small number of highly heterogeneous units are used. In this paper, we formulate a novel causal framework for inferring the iROAS of online advertising in a randomized geo experiment design, and we develop a robust model-free estimator ‘Trimmed Match’ which adaptively trims poorly matched pairs. Using simulations and case studies, we show that Trimmed Match can be more efficient than some alternatives, and we investigate the sensitivity of the estimator to some violations of its assumptions. Consistency and asymptotic normality are also established for a fixed trim rate.


Paper: Measurable Counterfactual Local Explanations for Any Classifier

We propose a novel method for explaining the predictions of any classifier. In our approach, local explanations are expected to explain both the outcome of a prediction and how that prediction would change if ‘things had been different’. Furthermore, we argue that satisfactory explanations cannot be dissociated from a notion and measure of fidelity, as advocated in the early days of neural networks’ knowledge extraction. We introduce a definition of fidelity to the underlying classifier for local explanation models which is based on distances to a target decision boundary. A system called CLEAR: Counterfactual Local Explanations via Regression, is introduced and evaluated. CLEAR generates w-counterfactual explanations that state minimum changes necessary to flip a prediction’s classification. CLEAR then builds local regression models, using the w-counterfactuals to measure and improve the fidelity of its regressions. By contrast, the popular LIME method, which also uses regression to generate local explanations, neither measures its own fidelity nor generates counterfactuals. CLEAR’s regressions are found to have significantly higher fidelity than LIME’s, averaging over 45% higher in this paper’s four case studies.


Paper: Climate extreme event attribution using multivariate peaks-over-thresholds modeling and counterfactual theory

Numerical climate models are complex and combine a large number of physical processes. They are key tools to quantify the relative contribution of potential anthropogenic causes (e.g., the current increase in greenhouse gases) on high impact atmospheric variables like heavy rainfall. These so-called climate extreme event attribution problems are particularly challenging in a multivariate context, that is, when the atmospheric variables are measured on a possibly high-dimensional grid. In this paper, we leverage two statistical theories to assess causality in the context of multivariate extreme event attribution. As we consider an event to be extreme when at least one of the components of the vector of interest is large, extreme-value theory justifies, in an asymptotical sense, a multivariate generalized Pareto distribution to model joint extremes. Under this class of distributions, we derive and study probabilities of necessary and sufficient causation as defined by the counterfactual theory of Pearl. To increase causal evidence, we propose a dimension reduction strategy based on the optimal linear projection that maximizes such causation probabilities. Our approach is tested on simulated examples and applied to weekly winter maxima precipitation outputs of the French CNRM from the recent CMIP6 experiment.


Paper: Minimax Crossover Designs

In crossover experiments, two broad types of treatment effects are typically considered: direct effects that capture the immediate impact of the treatment, and carryover effects that capture the lagged impact of past treatments. Existing approaches to optimal crossover design usually minimize a criterion that relies on an outcome model, and on assumptions that carryover effects are limited, say, to one or two time periods. The former assumption relies on correct specification, which is usually untenable, whereas the latter assumption is problematic when long-range carryover effects are expected, and are of primary interest. In this paper, we derive minimax optimal designs to estimate both direct and carryover effects simultaneously. In contrast to prior work, our minimax designs do not require specifying a model for the outcomes, relying instead on invariance assumptions. This allows us to address problems with arbitrary carryover lag, such as those encountered in digital experimentation.


Article: Measurable Counterfactual Local Explanations for Any Classifier

We propose a novel method for explaining the predictions of any classifier. In our approach, local explanations are expected to explain both the outcome of a prediction and how that prediction would change if ‘things had been different’. Furthermore, we argue that satisfactory explanations cannot be dissociated from a notion and measure of fidelity, as advocated in the early days of neural networks’ knowledge extraction. We introduce a definition of fidelity to the underlying classifier for local explanation models which is based on distances to a target decision boundary. A system called CLEAR: Counterfactual Local Explanations via Regression, is introduced and evaluated. CLEAR generates w-counterfactual explanations that state minimum changes necessary to flip a prediction’s classification. CLEAR then builds local regression models, using the w-counterfactuals to measure and improve the fidelity of its regressions. By contrast, the popular LIME method [15], which also uses regression to generate local explanations, neither measures its own fidelity nor generates counterfactuals. CLEAR’s regressions are found to have significantly higher fidelity than LIME’s, averaging over 45% higher in this paper’s four case studies.


Article: Structure Learning from Time Series with False Discovery Control

We consider the Granger causal structure learning problem from time series data. Granger causal algorithms predict a ‘Granger causal effect’ between two variables by testing if prediction error of one decreases significantly in the absence of the other variable among the predictor covariates. Almost all existing Granger causal algorithms condition on a large number of variables (all but two variables) to test for effects between a pair of variables. We propose a new structure learning algorithm called MMPC-p inspired by the well known MMHC algorithm for non-time series data. We show that under some assumptions, the algorithm provides false discovery rate control. The algorithm is sound and complete when given access to perfect directed information testing oracles. We also outline a novel tester for the linear Gaussian case. We show through our extensive experiments that the MMPC-p algorithm scales to larger problems and has improved statistical power compared to existing state of the art for large sparse graphs. We also apply our algorithm on a global development dataset and validate our findings with subject matter experts.
Advertisements