Is a p-value needed?

Is a p-value needed?

Much has been written on the need for statistics in genome-scale molecular biology.  Very clever analytical approaches were devised, taking the form of carefully crafted and freely downloadable software packages.  But still, every month or so, I meet with students and researchers facing a similar dilemma:  they need to decide whether to report the strength of an effect (eg. gene X is over-expressed by 4.5-fold in condition A vs. B) or the significance of such effect (eg. gene X is overexpressed in condition A vs. B with a p-value of 0.0012).

A p-value is always linked to a statistical test which is nothing more than a mathematical formulation of a very precise question on the data.  To derive a p-value, this question takes the form of an hypothesis (eg. gene X is not differentially expressed), called $H_0$ (null hypothesis) in the statistical jargon.  A low p-value suggests that the hypothesis doesn’t hold (eg. the data does not support that gene X is not differentially expressed…  notice the double negation!).  In practice, this nuance bites in two cases: tiny, irrelevant differences in large datasets or solid differences accompanied with unexpected noise.  Here are two simulated datasets (imagine qPCR quantification of a gene for samples under two conditions).


Rplot02

In the first scenario, the statistical test comes out significant even if the actual difference in expression is probably irrelevant (3% increase) and probably the result of some unwanted bias.  In the second scenario, one of the 3 samples under condition ‘B’ misbehaved, resulting in a large difference becoming non-significant.  For both cases, the average difference in expression would give a much better indication of what is going on.  In most projects I’ve been involved in (microarrays, RNA-Seq, HTS, proteomics, qPCR, etc.), my first suggestion is always to start analyzing the results using strength of effect (differences, log-fold-change, %inhibition, etc.).

Then, what is the point of applying statistical tests? In what cases does it actually serve the underlying scientific question (does conditions A or B impact expression of gene X)?  I’ll come back to this in a later post…  but feel free to share your opinion in the meantime!

By | 2016-11-08T09:30:17+00:00 September 12, 2014|Categories: Statistics|2 Comments

About the Author:

Enjoys turning raw (big, noisy and convoluted!) biological data into knowledge… using any tools that informatics has to offer!

2 Comments

  1. Louis-Eric Trudeau September 9, 2015 at 11:49 - Reply

    Bonjour,
    Excellente discussion d’une question importante.
    A quand la suite, pour répondre à la question du deuxième paragraphe?

    • Sébastien Lemieux September 9, 2015 at 12:33 - Reply

      Thanks Louis-Éric! [I’m taking the liberty to answer in English as you commented on this version of the text.] The follow-up post eventually morphed into http://bioinfo.iric.ca/tweaking-fishers-exact-test-for-biology/ . But, as I wrote this, I left out the idea that in most situations a confidence interval would best serve the purpose of assessing “biological significance” (in light of the variability, is this of interest) while acknowledging the continuous nature of the measure.

Leave A Comment