What does p-value of 0.3 mean? (2023)

Table of contents

• What does p-value of 0.3 mean?
• What does less p-value mean?
• How do you reject a null hypothesis?
• Why do we use 0.05 level of significance?
• Related questions

Share this Post:

The P-value approach involves determining "likely" or "unlikely" by determining the probability — assuming the null hypothesis were true — of observing a more extreme test statistic in the direction of the alternative hypothesis than the one observed. If the P-value is small, say less than (or equal to) P-value is large, say more than

If the P-value is less than (or equal to) P-value is greater than

Specifically, the four steps involved in using the P-value approach to conducting any hypothesis test are:

• Specify the null and alternative hypotheses.
• Using the sample data and assuming the null hypothesis is true, calculate the value of the test statistic. Again, to conduct the hypothesis test for the population meanμ, we use thet-statisticwhich follows at-distribution withn- 1 degrees of freedom.
• Using the known distribution of the test statistic, calculate theP-value: "If the null hypothesis is true, what is the probability that we'd observe a more extreme test statistic in the direction of the alternative hypothesis than we did?" (Note how this question is equivalent to the question answered in criminal trials: "If the defendant is innocent, what is the chance that we'd observe such extreme criminal evidence?")
• Set the significance level,, the probability of making a Type I error to be small — 0.01, 0.05, or 0.10. Compare theP-value to. If theP-value is less than (or equal to), reject the null hypothesis in favor of the alternative hypothesis. If theP-value is greater than, do not reject the null hypothesis.

The significance level determines how far out from the null hypothesis value we'll draw that line on the graph. To graph a significance level of 0.05, we need to shade the 5% of the distribution that is furthest away from the null hypothesis.

In the graph above, the two shaded areas are equidistant from the null hypothesis value and each area has a probability of 0.025, for a total of 0.05. In statistics, we call these shaded areas the critical region for a two-tailed test. If the population mean is 260, we’d expect to obtain a sample mean that falls in the critical region 5% of the time. The critical region defines how far away our sample statistic must be from the null hypothesis value before we can say it is unusual enough to reject the null hypothesis.

Our sample mean (330.6) falls within the critical region, which indicates it is statistically significant at the 0.05 level.

We can also see if it is statistically significant using the other common significance level of 0.01.

But on the other hand, p-values are dependent on sample size. They are not an absolute measure. Thus we cannot simply say 0.001593 is more significant than 0.0439. Yet this what would be implied in Fisher's framework: we would be more surprised to such an extreme value. There's even discussion about the term highly significant being a misnomer: Is it wrong to refer to results as being "highly significant"?

I've heard that p-values in some fields of science are only considered important when they are smaller than 0.0001, whereas in other fields values around 0.01 are already considered highly significant.

Related questions:

• Is the "hybrid" between Fisher and Neyman-Pearson approaches to statistical testing really an "incoherent mishmash"?
• When to use Fisher and Neyman-Pearson framework?
• Is the exact value of a 'p-value' meaningless?
• Frequentist properties of p-values in relation to type I error
• Confidence intervals vs P-values for two means
• Why are lower p-values not more evidence against the null? Arguments from Johansson 2011 (as provided by @amoeba)

How small is small enough? The most common threshold is p < 0.05; that is, when you would expect to find a test statistic as extreme as the one calculated by your test only 5% of the time. But the threshold depends on your field of study – some fields prefer thresholds of 0.01, or even 0.001.

The threshold value for determining statistical significance is also known as the alpha value.

Reporting p-values

P-values of statistical tests are usually reported in the results section of a research paper, along with the key information needed for readers to put the p-values in context – for example, correlation coefficient in a linear regression, or the average difference between treatment groups in a t-test.

Caution when using p-values

P-values are often interpreted as your risk of rejecting the null hypothesis of your test when the null hypothesis is actually true.

In reality, the risk of rejecting the null hypothesis is often higher than the p-value, especially when looking at a single study or when using small sample sizes. This is because the smaller your frame of reference, the greater the chance that you stumble across a statistically significant pattern completely by accident.

Suppose that you do a hypothesis test. Remember that the decision to reject the null hypothesis (H0) or fail to reject it can be based on the p-value and your chosen significance level (also called α). If the p-value is less than or equal to α, you reject H0; if it is greater than α, you fail to reject H0.

Your decision can also be based on the confidence interval (or bound) calculated using the same α. For example, the decision for a test at the 0.05 level of significance can be based on the 95% confidence interval:

• If the reference value specified in H0 lies outside the interval (that is, is less than the lower bound or greater than the upper bound), you can reject H0.
• If the reference value specified in H0 lies within the interval (that is, is not less than the lower bound or greater than the upper bound), you fail to reject H0.

• P > 0.05 is the probability that the null hypothesis is true.
• 1 minus the P value is the probability that the alternative hypothesis is true.
• A statistically significant test result (P ≤ 0.05) means that the test hypothesis is false or should be rejected.
• A P value greater than 0.05 means that no effect was observed.

If you answered “none of the above,” you may understand this slippery concept better than many researchers. The ASA panel defined the P value as “the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.”

Why is the exact definition so important? Many authors use statistical software that presumably is based on the correct definition. “It’s very easy for researchers to get papers published and survive based on knowledge of what statistical packages are out there but not necessarily how to avoid the problems that statistical packages can create for you if you don’t understand their appropriate use,” said Barnett S. Kramer, M.D., M.P.H., JNCI’s former editor in chief and now director of the National Cancer Institute’s Division of Cancer Prevention. (Kramer was not on the ASA panel.)

In the majority of analyses, an alpha of 0.05 is used as the cutoff for significance. If the p-value is less than 0.05, we reject the null hypothesis that there's no difference between the means and conclude that a significant difference does exist. If the p-value is larger than 0.05, we cannot conclude that a significant difference exists.

That's pretty straightforward, right? Below 0.05, significant. Over 0.05, not significant.

"Missed It By That Much!"

In the example above, the result is clear: a p-value of 0.7 is so much higher than 0.05 that you can't apply any wishful thinking to the results. But what if your p-value is really, really close to 0.05?

Like, what if you had a p-value of 0.06?

That's not significant.

Oh. Okay, what about 0.055?

Not significant.

How about 0.051?

It's still not statistically significant, and data analysts should not try to pretend otherwise. A p-value is not a negotiation: if p > 0.05, the results are not significant. Period.

• The p -value is conditional upon the null hypothesis being true is unrelated to the truth or falsity of the research hypothesis. A p -value higher than 0.05 (> 0.05) is not statistically significant and indicates weak evidence against the null hypothesis.

• P > 0.05 is the probability that the null hypothesis is true. 1 minus the P value is the probability that the alternative hypothesis is true. A statistically significant test result (P ≤ 0.05) means that the test hypothesis is false or should be rejected.

• To your point, the p value could be less than 0.05 and we could still have the test statistic be less than the critical value. This would mean our chosen α was smaller than 0.05, and would mean we would fail to reject the null. Thanks for contributing an answer to Cross Validated!

• If the p-value is not less than .05, then we fail to reject the null hypothesis and conclude that we do not have sufficient evidence to say that the alternative hypothesis is true. The following examples explain how to interpret a p-value less than .05 and how to interpret a p-value greater than .05 in practice.