hypothesis testing

Hypothesis testing is a formal statistical method for deciding whether sample data provides enough evidence to reject a default assumption about a larger population. It frames a question as two competing claims and measures how well the observed data fits the default one.

That default claim is the null hypothesis, usually a statement of no effect or no difference, such as a redesign making no difference to sign-ups. The alternative hypothesis is the claim a researcher suspects instead.

The test reduces the sample to a single test statistic and asks how surprising that value would be if the null hypothesis were true. That surprise is captured by a p-value, the probability of a result at least as extreme as the observed one, assuming the null hypothesis holds.

A p-value below a chosen threshold, the significance level (often 0.05), leads to rejecting the null hypothesis. Because the decision rests on probability, it can go wrong in two ways:

  • A Type I error rejects a true null hypothesis, a false positive.
  • A Type II error fails to reject a false null hypothesis, a false negative.

The interactive figure below shows how the two errors trade off as the decision threshold moves:

Interactive diagram — enable JavaScript to view.

Named procedures such as the t-test, chi-squared test, and analysis of variance (ANOVA) apply this framework to different kinds of data. In software, hypothesis testing underlies A/B testing, performance benchmarking, and the statistical evaluation of machine learning models.

Python Statistics Fundamentals: How to Describe Your Data

Tutorial

Python Statistics Fundamentals: How to Describe Your Data

In this step-by-step tutorial, you'll learn the fundamentals of descriptive statistics and how to calculate them in Python. You'll find out how to describe, summarize, and represent your data visually using NumPy, SciPy, pandas, Matplotlib, and the built-in Python statistics library.

intermediate data-science numpy

For additional information on related topics, take a look at the following resources:

By Martin Breuss • Updated Aug. 4, 2026
Computer Science Glossary Share Feedback