Paper: How and Why to Use Experimental Data to Evaluate Methods for Observational Causal Inference

Rectangular matrix-vector products are used extensively throughout machine learning and are fundamental to neural networks such as multi-layer perceptrons, but are notably absent as normalizing flow layers. This Methods that infer causal dependence from observational data are central to many areas of science, including medicine, economics, and the social sciences. We describe and analyze observational sampling from randomized controlled trials (OSRCT). This method is used to create observational data sets with corresponding unbiased estimates of treatment effect, increasing the number of data sets available for evaluating causal inference methods. We show that, OSRCT creates data sets that are equivalent to those produced by randomly sampling from empirical data sets in which all potential outcomes are available. We then perform a large-scale evaluation and find notable performance differences when comparing across data from different sources, demonstrating the importance of using data from a variety of sources when evaluating any causal inference method.