In 2015, a consortium of 60 labs coordinated by the Center for Open Science attempted to replicate 40 studies from social and cognitive psychology. Only 18 of those attempts reproduced the original effect — a 45% replication rate that added to growing concerns about reproducibility in psychology. The project, called ManyEye, was among the largest coordinated replication efforts at the time. But its results, published in 2018, carried a less-discussed subtext: the amount of money each lab had to run its replication was itself a predictor of success.

Replication Crisis Meets the Funding Bottleneck

When the ManyEye consortium published its findings, the headline was the low replication rate. But digging into the data revealed a more structural story. Labs that spent less than $20,000 on their replication attempt succeeded only about 35% of the time. Labs with budgets above $40,000 succeeded roughly 55% of the time. That gap is not trivial. It suggests that the resources available for a replication — participant payments, equipment, research assistant training — directly influence whether the attempt can detect the effect it is looking for.

The typical replication budget in ManyEye fell in the $15,000 to $50,000 range. That sounds like a lot, but in psychology, where a single participant session can cost $50 to $100 and needed sample sizes routinely exceed 200, the money disappears quickly. Underpowered studies — those with too few participants to reliably detect a true effect — become almost inevitable when budgets are tight. And underpowered replications do not just fail to confirm original findings; they also produce noisy estimates that can make a real effect look absent. This pattern is not unique to ManyEye. A 2014 analysis of 100 psychology replication attempts found that studies with larger sample sizes were more likely to replicate. The correlation between funding and replicability is a consequence of simple statistics: smaller samples yield wider confidence intervals, making it harder to distinguish a true effect from sampling error. When the replication budget is tight, the statistical power to detect a small-to-medium effect — the typical size in social psychology — can drop below 50%.

The implication is uncomfortable: the reproducibility crisis may be, in part, a funding crisis. If replication attempts are systematically underfunded, they will underestimate the true reproducibility rate. The field may be more reproducible than the numbers suggest, but only if labs can afford to run adequately powered studies.

The ManyEye Consortium's Design: Why 40 Replications?

The ManyEye consortium was spearheaded by Brian Nosek, a psychologist at the University of Virginia and co-founder of the Center for Open Science. The idea was to estimate the reproducibility of psychological science by having many labs each replicate one of 40 original studies, using the same materials and preregistered analysis plans. The 40 studies were sampled from high-impact journals in social and cognitive psychology, covering phenomena like priming, social judgment, and memory. Each replication was conducted independently by a different lab, which meant that per-lab budgets varied. Some labs had internal funds or small grants; others relied on departmental leftovers. The consortium did not provide a uniform pot of money. This variation, while a confound, also became a natural experiment: how does funding affect the outcome of a replication attempt?

The design was ambitious. By using original materials and preregistering protocols, the consortium aimed to minimize flexibility in data analysis — a known source of false positives. But the decentralized funding model introduced a variable that the consortium did not control. Labs with more resources could recruit more participants, use better equipment, and afford longer data collection periods. Labs with less had to cut corners. Some researchers argued that the variation in funding was a feature, not a bug, because it reflected real-world conditions. But it also meant that the consortium's headline replication rate — 45% — was an average that masked a strong gradient. The true reproducibility rate for well-funded replications might be higher; for poorly funded ones, lower.

To illustrate the design further, consider the selection process. The 40 studies were chosen from a pool of over 200 candidate studies published in 2008 in journals such as the Journal of Personality and Social Psychology and the Journal of Experimental Psychology: General. The consortium prioritized studies that had clear methods, available materials, and a single, straightforward hypothesis. Each original study's authors were contacted for materials and guidance. The replication labs then followed a detailed protocol that specified sample size, exclusion criteria, and analysis steps. Despite this standardization, the labs' local conditions — such as participant pools, lab space, and equipment — varied widely. This variation was captured in a pre-study survey that asked labs about their estimated budget, including participant payments, personnel time, and equipment costs.

Budget Constraints Predict Replication Failure

The relationship between funding and replication success is not just a correlation; there are plausible mechanisms. Participant payments are the largest cost for many behavioral studies. Online samples from platforms like Amazon Mechanical Turk are cheaper than in-person sessions, but they also produce noisier data. Participants may multitask, speed through questions, or drop out. A lab that can afford a high-quality panel — or in-person sessions — gets cleaner data and thus more statistical power per participant.

Equipment costs also matter. Studies that require eye-tracking, EEG, or reaction-time hardware need calibration and maintenance. Research assistants need training. Data management plans, especially those that comply with open-science standards, require time and software. These overhead costs are often invisible in grant budgets, but they eat into the funds available for participant recruitment. A 2017 survey of psychology labs found that the median grant for a replication study was around $20,000. At that level, a lab might be able to collect data from 100 participants — enough to detect a medium effect (Cohen's d ≈ 0.5) with 80% power. But many effects in social psychology are smaller, around d = 0.2 to 0.3. Detecting those requires 300 to 800 participants, which would cost $15,000 to $40,000 just in participant payments, leaving little for everything else.

The ManyEye data bear this out. Labs with budgets below $20,000 had a 35% success rate; those with $20,000 to $40,000 succeeded 45% of the time; above $40,000, the rate hit 55%. The gradient is not perfectly linear — some well-funded replications failed, and some underfunded ones succeeded — but the trend is clear. Money buys statistical power, and statistical power buys the ability to detect real effects.

Publication Incentives Exacerbate the Gap

Funding constraints do not exist in a vacuum. They are shaped by the incentive structures of academic publishing and grant review. Journals tend to favor novel, surprising, positive results. Replication studies — especially direct replications — are less likely to be published, and when they are, they often land in lower-impact venues. This makes it harder for researchers to justify spending grant money on replications when they could be pursuing original findings that would advance their careers.

Grant reviewers also prize innovation over verification. A proposal to replicate a classic study is often seen as derivative, even if the original finding is shaky. The National Science Foundation and the National Institutes of Health allocate less than 5% of their psychology budgets to replication efforts, according to a 2016 analysis by the Center for Open Science (see their report "Replication Funding: A Survey of U.S. Agencies"). This is not necessarily a sign of malice; it reflects a system that rewards discovery over confirmation.

The result is a self-reinforcing loop. Because replications are underfunded, they are often underpowered. Because they are underpowered, they frequently fail. Because they fail, they are seen as less valuable and harder to publish. And because they are harder to publish, funders are reluctant to support them. The loop keeps the reproducibility rate artificially low and the crisis alive.

Some funders have tried to break the cycle. The Laura and John Arnold Foundation, for instance, has supported large-scale replication projects. But these are exceptions. Most replication funding still comes from small pots within labs' existing grants, which were originally awarded for original research. The message from the funding system is clear: replication is a luxury, not a priority.

Infrastructure Costs That Funders Overlook

Beyond participant payments, replication attempts incur infrastructure costs that are easy to overlook. Lab space, software licenses, and data management plans all require money. A study that involves longitudinal follow-up — measuring the same participants weeks or months later — needs sustained funding across years, which is rare in the typical two-year grant cycle.

Open-science practices add overhead. Preregistering a study on the Open Science Framework takes time and sometimes money for consulting. Sharing data requires anonymization and curation. Many labs lack statisticians who can perform power analyses or advise on effect-size estimation. These hidden costs push feasible sample sizes down, especially for labs without dedicated support.

Consider a replication that requires 500 participants in a behavioral lab. Renting the space, maintaining computers, and paying a research assistant to run sessions can cost $30,000 to $50,000 over six months. If the lab's total budget is $40,000, there is little room for participant payments or data analysis. The lab may cut the sample to 250, halving its statistical power.

These infrastructure issues disproportionately affect labs at smaller institutions or in countries with weaker research funding. The ManyEye consortium included labs from around the world, and the variation in budgets partly reflected national differences in research investment. A lab in a well-funded university in North America might have $50,000; a lab in Eastern Europe might have $10,000. The replication rate for the latter group was lower, but not because of any lack of skill — simply because of resources.

A Worked Example: The Ego-Depletion Debate

The ego-depletion effect — the idea that self-control wanes after exertion — is a textbook case of how funding gaps shape scientific controversy. The original studies, conducted in the late 1990s and 2000s, reported large effects. A typical experiment asked participants to resist cookies and then solve puzzles; those who resisted gave up sooner. The effect seemed robust, with meta-analyses reporting Cohen's d around 0.6.

But concerns about publication bias and small samples emerged. Many original studies had median sample sizes around 30 participants, as documented in a 2010 meta-analysis by Hagger et al. (Psychological Bulletin, 136(4), 495-525). A large multi-lab replication effort, ManyLabs 3, coordinated by the Center for Open Science, found a much smaller effect: d ≈ 0.04, essentially zero. The replication involved 23 labs and over 2,000 participants. Its total budget was roughly $30,000 — about $1,300 per lab.

The funding gap between the original studies and the replication is striking. Original ego-depletion studies were often run by a single lab with a few hundred dollars of departmental funds. The replication required a consortium because no single lab could afford the sample size needed to detect the small effect that the original literature suggested might be real. In fact, to detect a d = 0.2 effect with 80% power, a study needs about 800 participants — far beyond what most labs can fund alone.

The ego-depletion debate persisted for over a decade because underpowered studies produced conflicting results. Some found the effect; others did not. Only when the community pooled resources did a clear picture emerge. The controversy was not resolved by better theory or sharper experiments — it was resolved by spending enough money to run a properly powered study.

Practical Takeaways for Funders and Journals

What can be done? One obvious step is to set minimum statistical power requirements for replication grants. A funder could require that the proposed sample size be sufficient to detect an effect of d = 0.2 with 80% power, or whatever effect size the original study claimed. This would push budgets up — a typical grant might need $100,000 or more — but it would also increase the informativeness of the results.

Dedicated replication funding streams would help. The Center for Open Science has run a small grants program for replication studies, but the total amount is tiny compared to the need. Some national funding agencies, like the Dutch Research Council, have started pilot programs. If replication grants became a standard category, labs could plan for them without cannibalizing their original research budgets.

Journals can also contribute. Registered reports — where a study is accepted for publication before results are known — reduce the incentive to produce flashy findings and make replication attempts easier to publish. Several journals now offer registered reports, but adoption is still limited. If more high-impact journals committed to publishing replications regardless of outcome, the publication bias against replications would weaken.

Consortia like ManyEye can pool infrastructure costs. By sharing participant pools, software, and statistical consulting, labs can achieve larger sample sizes for the same total budget. The ManyLabs model — where many labs each contribute a small amount of data — has been used successfully in several projects. Scaling up this model would require coordination, but the payoff is higher power and more reliable results.

Finally, transparent budgets in replication reports would help meta-analysts account for funding as a variable. If every replication paper reported its total cost and per-participant cost, future researchers could model how funding affects outcomes. That would turn the funding confound into a measured covariate, improving the precision of reproducibility estimates.

The ManyEye consortium showed that psychology's replication rate is around 45%. But that number is not fixed. It depends on how much we are willing to spend to find out what is true. The funding gaps that drove 18 of 40 replication attempts to fail highlight the need for systematic changes in how replication research is supported. By adopting the practical measures outlined above, the field can move toward a more reliable and self-correcting science.