Austerity, now with decimals

In 2010, Reinhart and Rogoff (2010) reported a striking threshold: countries with public debt above roughly 90% of GDP appeared to experience sharply lower average growth. Austerity had arrived dressed as arithmetic and certified by “science.” The result traveled from a spreadsheet into speeches, newspapers, and public policy with the authority of an apparently neutral fact; an empirical association was converted into a policy prescription.

Then graduate student Thomas Herndon tried to reproduce it. Herndon and his co-authors found “selective exclusion of available data, coding errors and inappropriate weighting of summary statistics” (Herndon, Ash, and Pollin, 2014, p. 257). Their corrections changed the reported average growth rate for high-debt countries from $-0.1\%$ to $2.2\%$ and destroyed the dramatic threshold. The data had not spoken for themselves. Researchers had made consequential choices, the spreadsheet contained errors, and policymakers had listened to a politically convenient number.

This is the credibility crisis in miniature. This crisis feeds anti-scientific discourse and scientific denialism, but the answer cannot be to protect scientific authority from criticism. Science is a human enterprise and therefore necessarily involves judgment. Those choices do not make science arbitrary; concealing them behind mechanical procedures does.

Statistical significance and $p$-value fetishism: the answer you did not calculate

Herndon’s exercise exposes a deeper problem: data do not interpret themselves. To see why, imagine a disease affecting one person in a thousand. A test correctly identifies 99% of sick patients but also produces a positive result for 1% of healthy patients. A positive result sounds nearly conclusive. But which probability do you actually need: the probability of testing positive when sick, or the probability of being sick after testing positive?

You need the second because the evidence you have is the positive result. Bayes’ rule tells us that the probability is only about 9%.¹ The apparently impressive sensitivity of 99% does not settle the question because the disease has a base rate of only 0.1%.

The same reversal of conditioning appears in statistical significance, typically assessed using $p$-values. Empirical papers decorate estimates with magical stars indicating whether results are “statistically significant.” Those stars can give political, ideological, and interest-driven claims a veneer of scientism: mathematical notation appears to elevate them above the realm of contestable “opinions.”

A $p$-value does not tell us the probability that the null hypothesis is true after seeing the data; instead, it describes how the test statistic would behave across hypothetical samples generated under $H_0$ and $\mathcal{M}$. For a test statistic $T$ whose larger values count against $H_0$, the $p$-value is a tail probability calculated under $H_0$ and the specified statistical model $\mathcal{M}$:

$$\underbrace{p}_{p\text{-value}} = \underbrace{\Pr\!\left(T(X)\geq T(D)\mid H_0,\mathcal{M}\right)}_{\text{tail probability under }H_0\text{ and }\mathcal{M}} \neq \underbrace{\Pr\!\left(H_0\mid D,\mathcal{M}\right)}_{\text{posterior probability of }H_0}$$

Here, $X$ represents hypothetical data generated under $H_0$ and $\mathcal{M}$, whereas $D$ represents the data actually observed (Wasserstein and Lazar, 2016). Researchers nevertheless routinely read a small $p$-value as if it were the expression on the right. But the left-hand side assumes $H_0$ and asks how unusual results at least as extreme as the observed one would be; the right-hand side asks how plausible $H_0$ is after observing the data. Treating the first as if it answered the second reverses the conditioning. This is Cohen’s inverse-probability error (Cohen, 1994). Cohen exposed it in 1994; the ASA had to repeat the warning in 2016. The ritual survived.

The misunderstanding becomes institutionally powerful through a familiar rule: when the $p$-value falls below a predetermined significance level $\alpha$, reject the null hypothesis. Although a test may concern one parameter or several restrictions, standard practice often defines the null as no effect and the alternative as some nonzero effect.² Rejecting $H_0$ does not prove the alternative theory motivating $H_1$; failing to reject $H_0$ does not prove that there is no effect. Nevertheless, a continuous tail probability is routinely converted into a binary scientific verdict.

The ASA’s six principles amount to a rejection of this ritual. A $p$-value indicates how incompatible the data are with a specified model, but it does not measure the probability that a hypothesis is true, the probability that chance alone produced the result, the magnitude or importance of an effect, or, by itself, the strength of the evidence. It therefore cannot serve as a mechanical decision rule detached from scientific context, transparency, and full reporting (Wasserstein and Lazar, 2016).

Replacing $p$-values with confidence intervals does not abolish the ritual; it can merely repackage it. For a two-sided test at the 5% significance level, the corresponding 95% confidence interval ordinarily produces the same binary decision: if it excludes the value specified by $H_0$, the null is rejected. Confidence intervals provide more information by displaying a range of parameter values compatible with the data and model, but researchers often reduce them to the same question: does the interval contain zero? The stars disappear, but the ritual survives.

Confidence intervals are also routinely misread. A 95% confidence interval does not mean that there is a 95% probability that the “true” parameter lies between the observed bounds. In frequentist inference, the parameter is fixed and the interval varies across hypothetical samples: the 95% refers to the long-run proportion of intervals produced by the same procedure that would contain the “true” parameter. A Bayesian credible interval can make the probability statement readers naturally want, conditional on the prior, likelihood, model, and observed data (Lambert, 2018).

Null-hypothesis significance testing has its own holy trinity: the null model is the Father, creating the hypothetical world in which evidence is judged; the $p$-value is the Son, its visible numerical incarnation in the regression table; and $\alpha=.05$ is the Holy Spirit, the invisible convention that transforms a tail probability into a scientific verdict. The confidence interval, when reduced to whether it contains zero, is merely the same creed in different vestments.

The problem is therefore larger than one abused statistic. It is a statistical culture that transforms conditional statements into certainties and continuous uncertainty into binary verdicts. Why has that culture survived? Because it offers a seductive fiction: feed neutral data into a black box and receive a scientific verdict. Yet no analysis escapes background commitments. Judgments about model form, measurement, sampling, variable selection, specifications, and plausible effects enter before any $p$-value is calculated.

This concealment is not politically innocent. As dos Santos (2025) argues, conventional statistical practice frequently presents conditional inferences as statements of fact rather than statements of rational belief given prior information. A Bayesian objection begins from the opposite premise: refusing to represent prior plausibility does not purify inference; it merely leaves consequential assumptions in the background. Mechanical objectivity does not eliminate judgment. It pushes judgment backstage, where readers are less able to inspect and contest it.

Like every fetish, statistical significance conceals a social relation behind the apparent authority of a thing: the judgments and incentives of researchers, journals, and policymakers return as if they were properties of the number itself.

When the threshold becomes the target

Once $p<\alpha$ becomes the price of admission, researchers acquire powerful incentives to produce it. With many researcher degrees of freedom, outcomes, controls, samples, exclusions, functional forms, and stopping rules can all be varied. Sometimes these data-dependent choices create a garden of forking paths without conscious manipulation (Gelman and Loken, 2013); sometimes they become $p$-hacking, a deliberate search across analyses until a favorable result emerges. Plümper and Neumayer call the intentional selection and reporting of research designs and model specifications that produce a desired result tweaking (Plümper and Neumayer, 2026, p. 8). Beyond these questionable research practices lie fabrication and falsification. The acts differ sharply in intention and gravity, but all exploit the distance between the published result and the hidden process that produced it. Once publication, fame, and career advancement reward the validation ritual, gaming the system can become individually rational even as it corrodes collective scientific credibility (Plümper and Neumayer, 2026, p. ix).

Economics is not exempt. Brodeur, Cook, and Heyes (2020) found that evidence of $p$-hacking and publication bias varied across the causal-identification methods promoted by the credibility revolution, with instrumental variables and, to a lesser extent, difference-in-differences appearing particularly problematic. After correcting for rounding, Kranz and Pütz (2022) found that the evidence around the 5% threshold weakened substantially for difference-in-differences but remained for instrumental variables; Brodeur, Cook, and Heyes (2022) replied by acknowledging the weaker result. That dispute is healthy science: methods are exposed, claims are attacked, and conclusions are revised—not accepted because readers worship a regression table.

Replication, credibility, and the authoritarian opening

The bill for this statistical culture is visible in the replication literature. Clayton (2021, Table 6.5) reveals a scientific record far less secure than the authority attached to published and statistically significant results would suggest.

Source: Adapted from Clayton (2021, Table 6.5). Note: The underlying projects differ in design and in their definitions of replication success; the rates are therefore not directly comparable across fields.

A failed replication does not automatically prove that the original result was fraudulent or even false. Taken together, however, these results destroy the comforting belief that publication and statistical significance reliably separate knowledge from noise.

The credibility crisis does not mechanically produce authoritarianism, but it creates an opening that authoritarian and anti-scientific movements readily exploit: genuine cases of error, manipulation, and institutional concealment are generalized into the claim that science is nothing but vested interest, experts are political enemies, and truth is whatever power declares it to be.

The Ariely–Wertenbroch case shows why aggregate replication rates are only the beginning. A close replication found negligible effects and failed to replicate the original performance claim (Hyndman and Bisin, 2026). Data Colada subsequently reported duplicated observations and other anomalies in the historical Study 2 file, as well as apparent alterations to thirteen grades in the Study 1 file (posts 138 and 139). On September 2, 2026, Psychological Science retracted the original article, citing the replication, its lack of confidence in the underlying datasets, and Data Colada’s investigation; both authors agreed to the retraction. The replication failure did not itself prove fabrication: it opened the door to a separate forensic examination.

No more worshiping statistical gods: show us the prior

The alternative is not inference without judgment, because no such inference exists. As Foley and Scharfenaker (2025) emphasizes, social scientists generally possess two forms of information: observations and theoretical knowledge about the processes and constraints generating them.

Bayesian inference makes their different roles visible. Let $\theta$ denote the unknown parameter or collection of parameters, $D$ the observed data, and $I$ the relevant background information and modeling assumptions. The prior $p(\theta\mid I)$ represents what this information implies about $\theta$ before conditioning on $D$; the likelihood $p(D\mid\theta,I)$ describes how the model says the observed data could have been generated; and the posterior $p(\theta\mid D,I)$ expresses what follows when they are combined:

$$p(\theta\mid D,I) = \frac{p(D\mid\theta,I)\,p(\theta\mid I)} {p(D\mid I)} \propto p(\theta\mid I)\,p(D\mid\theta,I)$$

Here, $p(D\mid I)$ is the marginal likelihood, or evidence, that normalizes the posterior.

The prior is therefore not an embarrassing confession of bias. It is an auditable statement of the information and commitments brought to the analysis before conditioning on the current data. Priors can be challenged, replaced, and subjected to sensitivity analysis; the implications of the full model can be confronted with observed data through prior and posterior predictive checks. Bayesianism does not remove disagreement from science. It forces disagreement into the open, where it belongs.

When theoretical information takes the form of constraints rather than a conventional probability distribution, maximum-entropy reasoning can identify the least committal distribution relative to the stated constraints and reference measure (Jaynes, 1957); (Foley and Scharfenaker, 2026). Theory disciplines the prior, data update it through the likelihood, and neither is permitted to masquerade as assumption-free fact.

Bayesian coherence cannot repair fabricated data, undisclosed exclusions, or code that nobody is permitted to inspect. A Bayesian can choose convenient priors, misspecify a likelihood, hide unsuccessful analyses, or selectively report posterior results. Replacing $p<.05$ with $BF>10$ would merely elevate another statistic into an object of worship. But this does not make Bayesian and frequentist inference equivalent. Bayesianism supplies the inferential language that the scientific question requires; open data, code, provenance, sensitivity analysis, reproduction, and replication provide the institutional discipline that credible inference requires (Simonsohn, Simmons, and Nelson, 2020); (Steegen et al., 2016); (Christensen and Miguel, 2018).

Scientific objectivity does not require researchers to become dispassionate machines. It requires commitments visible enough to be challenged and conclusions revisable in light of evidence. The choice is between Bayesian inference that makes its assumptions available for contestation and the dominant frequentist ritual that protects judgment behind the appearance of mechanical objectivity.³ This is a more honest conception of science. Reproduction reconnects scientists to one another; replication reconnects scientific claims to the world and gives reality a veto.