Evaluating Economic Evidence — Evaluate the strength of evidence for an economic argument (intro)

Every day you encounter economic claims—on social media, in the news, or even in political debates. Someone might say, "Raising the minimum wage always destroys jobs," or "Tax cuts pay for themselves." But how do you know whether these claims are backed by solid evidence or are just opinions dressed up as facts? The skill of evaluating economic evidence has been developing for centuries, evolving alongside the discipline of economics itself.

In the early days of economic thought, arguments were largely philosophical. Thinkers like Adam Smith and David Ricardo relied on logic, observation, and narrative examples to make their cases. It was not until the twentieth century that economists began to insist on empirical evidence—data collected from the real world—as a standard for evaluating claims. This shift transformed economics from a purely theoretical field into one that uses statistics, experiments, and careful measurement to test ideas.

This history reveals a key question that still matters today: How can we tell the difference between a well-supported economic argument and one that only sounds convincing? Answering that question is the central goal of this lesson.

Before you can judge whether economic evidence is strong or weak, you need a framework—a set of guiding ideas that help you ask the right questions. Think of these principles as a toolkit you carry with you whenever you read an article, hear a speech, or discuss policy. Each principle highlights a different dimension of evidence quality.

The diagram below illustrates a step-by-step process for evaluating any economic claim. Start at the top with the claim itself, then move through four evaluation filters. Each filter asks a critical question, and the answers help you decide whether the overall evidence is strong, moderate, or weak.

Notice that the framework is sequential: you work through each filter in order. An argument that fails at Filter 1—for example, an anonymous source with a clear financial interest—should already raise red flags, even if the data itself looks impressive. Conversely, evidence from a credible source that still confuses correlation with causation (Filter 3) has a significant weakness. The more filters the evidence passes, the more confidence you can place in the argument.

While evaluating economic evidence is not primarily a math exercise, it helps to understand a few basic concepts that economists use to measure the strength of evidence. Even at an introductory level, knowing what these terms mean will sharpen your ability to ask the right questions.

Understanding Sample Size

When economists collect data to support a claim, the sample size (often represented as n) refers to the number of observations or data points they used. A larger sample generally produces more reliable results. If someone claims that a new trade policy boosted exports based on data from only 5 companies, that's a very small sample. If the study examined 5,000 companies, the evidence is far stronger.

The Correlation vs. Causation Test

One of the most common errors in economic arguments is treating correlation (two things moving together) as if it were causation (one thing directly causing the other). To test for causation, economists look for a plausible mechanism—a logical explanation for why A would cause B—and they try to control for confounding variables, which are outside factors that could explain both A and B. Strong evidence accounts for confounders; weak evidence ignores them.

Checking for Cherry-Picking

Cherry-picking occurs when someone selects only the data points that support their argument while ignoring data that contradicts it. For example, if a company reports its stock price on only the three best days of the year, it paints a misleading picture. Evaluating evidence means asking: Is all the relevant data being presented, or just the convenient data? Strong arguments acknowledge contradictory evidence and explain why their conclusion still holds.

Not all evidence is created equal. Economists rely on several types of evidence, and each type carries different strengths and weaknesses. Understanding this hierarchy helps you quickly assess how much weight to give to a particular piece of evidence. The diagram below arranges the most common types of economic evidence from strongest to weakest.

Let's apply the evidence evaluation framework to a real-world-style economic claim. Suppose you read the following argument in a news article:

As you practice evaluating economic evidence, you will start to notice recurring patterns—some arguments consistently display signs of strong evidence, while others fall into predictable traps. The table below summarizes the most common strengths and pitfalls you should watch for.

The evidence evaluation skills you are learning in this introductory lesson lay the groundwork for more sophisticated analysis you might encounter in college economics, AP courses, or real-world policy work. As you advance, the questions become more complex, but the core principles remain the same. The table below shows how introductory concepts connect to their advanced counterparts.

The good news is that by mastering the basics now, you are building a foundation that will serve you whether you pursue a business degree, enter the workforce, or simply want to be a more informed citizen. Every time you read an economic claim in the news, you are practicing the same skills that professional economists and policy analysts use every day—just at a different scale.

Test your understanding of evidence evaluation with the following five problems. They increase in difficulty from basic recall to critical analysis.