What a $p$-value means and what it does not mean

A $p$-value answers a conditional question: if the null hypothesis were true, how surprising would this statistic, or something at least this extreme, be?

That means the $p$-value is about the compatibility of the observed data with the null model. It is not a direct probability assigned to the null hypothesis itself.

A small $p$-value does not prove the alternative hypothesis, and a large $p$-value does not prove the null. It only reports how extreme the observed result looks under one model.

It also does not tell you whether the effect size matters in practice. Statistical significance and practical significance are separate questions.

The test statistic measures how far the observed summary is from the null value after scaling by the expected variability. The calculator then turns that statistic into a $p$-value using the assumed null model.

The decision line, such as 'reject' or 'fail to reject,' is a reporting convenience built from comparing the $p$-value with the significance level $\alpha$. It is not an extra theorem beyond that comparison.

The language around tests sounds causal even when the procedure is conditional. That is why people slide from 'surprising under the null' to 'unlikely that the null is true,' even though those are different statements.

A good habit is to say the result in full: 'if the null were true, this outcome would be this surprising.' That wording keeps the logic of the test intact.