Using Machine Learning to Catch Bogus Firms

We investigate the use of a machine learning algorithm to identify non-existent (fraudulent) firms that are used for tax evasion. Using a rich dataset of tax returns in an Indian state over several years, we train a machine learning-based model to predict fraudulent firms. We then use the model predictions to carry out field inspections of firms identified as suspicious by the machine learning tool. We find that the machine learning model is accurate in both simulated and field settings in identifying non-existent firms. Withholding a randomly selected group of firms from inspection, we estimate the causal impact of machine learning-driven inspections. Despite the strong predictive performance, our model-driven inspections do not yield a significant increase in enforcement, as shown by the cancellation of fraudulent firm registrations and tax recovery. We provide two reasons for this discrepancy, based on a close analysis of the tax department’s operating protocols – selection bias, and institutional friction in integrating the model into existing administrative systems. Our study serves as a cautionary tale for the application of machine learning in public policy contexts, and relying solely on test set performance as an effectiveness indicator. Field evaluations are critical in assessing the real-world impact of predictive models.