The Surprising Math Behind Truly Impactful Business Research

The Math That Makes Research Matter (And Why Most Papers Get It Wrong)

A few years ago, a group of researchers at the University of Granada and Tilburg University noticed something unsettling. They were reading through hundreds of papers in information systems research, and they kept seeing the same pattern: brilliant hypotheses, rigorous data collection, and then a statistical analysis that turned everything into noise.

The problem wasn't that the math was wrong. The problem was that the math was asking the wrong questions.

In their 2019 paper in Information & Management, Jose Benitez, Jörg Henseler, Ana Castillo, and Florian Schuberth laid out a new framework for what they called "impactful analysis" using a technique called partial least squares structural equation modeling, or PLS-SEM. The paper has since accumulated over 1,650 citations, not because it invented a new method, but because it revealed why so many perfectly competent researchers fail to produce research that anyone actually uses.

The answer is both simpler and stranger than you might expect.

What Most Researchers Get Wrong About "Impact"

Here is the uncomfortable truth that Benitez and his coauthors confronted directly: most business research is statistically correct but practically useless.

The standard approach works like this. A researcher collects data, runs a model, and reports whether their hypotheses were supported. If the p values are low enough, the paper gets published. The problem is that this process tells you almost nothing about whether the findings matter in the real world.

Benitez et al. (2019) argued that the real test of impactful research is not statistical significance. It is something they called "explanatory power." This is a measure of how much of the variation in your outcome variable your model actually accounts for. A relationship can be statistically significant while explaining almost nothing about what is actually happening in a business context.

Think of it this way. You can prove that wearing red shoes is statistically associated with higher sales performance. If you run enough tests, something random will appear significant. But if the effect size is tiny, it tells you nothing useful. The authors formalized this by insisting that researchers report not just whether a relationship exists, but how much it matters.

The Three Questions That Separate Good Research From Great Research

Benitez and his colleagues proposed that every impactful analysis must answer three distinct questions. Most researchers only answer the first one.

Question 1: Does the relationship exist?

This is the standard hypothesis test. You check whether your path coefficient is statistically different from zero. The authors call this "significance assessment."

Question 2: How strong is the relationship?

Here is where things get interesting. The authors argued that researchers must report the size of the effect, not just its existence. In PLS-SEM, this means reporting the path coefficients, the R squared values for endogenous constructs, and the effect sizes (f squared). A path coefficient of 0.03 might be statistically significant with a large enough sample, but it explains almost nothing.

Question 3: Can the model predict new cases?

This is the killer question that almost nobody asks. Benitez et al. (2019) insisted that researchers must test whether their model can predict outcomes in new data, not just fit the data they already have. They recommended using blindfolding procedures to generate cross validated redundancy measures, specifically the Stone Geisser Q squared value.

If your model cannot predict, it is not explanatory. It is just description dressed up as science.

The Methodology That Made This Possible

The authors did not just complain about bad practices. They provided a specific protocol for doing it right.

The technique they focused on, PLS-SEM, is particularly suited for business research because it handles complex models with many constructs, works well with smaller samples, and does not require the data to follow a normal distribution. Most business data is messy. PLS-SEM was designed for that mess.

But the authors were brutally specific about when to use it. PLS-SEM is appropriate when the research goal is prediction and explanation of complex relationships. It is not appropriate when the goal is theory testing with well established measures and large samples. In those cases, covariance based SEM is better.

They also provided a detailed reporting checklist. Every paper should include:

• ▸The sample size and how it was determined
• ▸The number of indicators per construct
• ▸The type of measurement model (reflective or formative)
• ▸The path coefficients with confidence intervals
• ▸The R squared values for all endogenous constructs
• ▸The effect sizes for all relationships
• ▸The predictive relevance using Q squared

This is not glamorous work. But it is the difference between a paper that gets cited and a paper that changes how people think.

What This Research Does Not Prove

This is the part that matters most and gets ignored most often.

The framework that Benitez et al. (2019) proposed does not prove that PLS-SEM is always better than other methods. It does not prove that statistical significance is meaningless. And it does not prove that predictive power is the only thing that matters.

What it proves is that the standard way of reporting results is incomplete. The problem is not that researchers use the wrong math. The problem is that they stop too early. They check significance, declare victory, and move on. They never ask whether their model actually explains anything worth explaining.

The open question is whether this approach will survive the replication crisis that has hit psychology, medicine, and economics. Benitez and his coauthors were writing specifically for information systems research, but their logic applies broadly. Any field that relies on complex models with latent variables could benefit from this kind of rigor.

The authors also acknowledged a limitation that many readers miss. PLS-SEM is excellent for prediction, but it can produce biased parameter estimates if the model is misspecified. The technique is not a magic bullet. It is a tool that requires careful use.

The Real Reason Most Business Research Fails

Here is the honest answer that Benitez et al. (2019) danced around but never quite stated directly.

Most business research fails because it is designed to get published, not to be useful. Researchers optimize for statistical significance because that is what journals reward. They optimize for novelty because that is what gets attention. They do not optimize for explanatory power because nobody asks them to.

The authors' framework is a direct challenge to this incentive structure. By requiring researchers to report effect sizes, predictive relevance, and model fit, they force a conversation about whether the research actually matters. A paper with a significant p value but an R squared of 0.02 is not impactful. It is noise.

This is uncomfortable for a field that has built its reputation on finding statistically significant relationships. But it is also liberating. It means that the next time you read a business paper, you can look past the asterisks and ask the real question: does this model tell me something I can use?

What This Actually Means

Here is what changes if you take Benitez, Henseler, Castillo, and Schuberth seriously.

• ▸If you are a researcher, stop reporting only significance. Report the path coefficients, the R squared values, and the Q squared values. If your model cannot predict, it is not explanatory.

• ▸If you are a reviewer, reject papers that do not report effect sizes. A significant relationship with a tiny effect is not a finding. It is a distraction.

• ▸If you are a practitioner reading academic research, ignore any paper that does not tell you how much of the variation it explains. If the R squared is below 0.10, the model is not useful for decision making.

• ▸If you are designing a study, use PLS-SEM when your goal is prediction and explanation of complex relationships. Use covariance based SEM when your goal is theory testing with established measures.

• ▸If you are teaching research methods, teach students to ask the three questions before they ever run a model. Does the relationship exist? How strong is it? Can it predict new cases?

The math behind impactful research is not complicated. It just requires asking the right questions and refusing to stop at the easy answers.