Estimating the extent of inflated significance in economics

Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as being non-significant. We estimate the extent of inflated significance to range between 56.2% and 71.3% of the significant p-values. Subsample analysis suggests that the extent of inflated significance is reduced in research fields that use experimental designs, analyze microeconomics research questions and have at least some adequately powered studies.