Introduction to Linear Regression

A data scientist's first goal is to find underlying relations within the variables of a dataset. Several statistical and machine learning methods can be used to discover such relations. Once uncovered, this information can be applied to everyday problems. For example, in clinical medicine, a predictive model based on clinical data can help clinicians guide a patient's treatment by offering insights that might not have otherwise been taken into account. Simple linear regression One of the most basic methods available to [...]

A javascript implementation of the non-central version of Fisher’s exact test

In a previous post, I presented a case for choosing a non-central version of Fisher's exact test for most of bioinformatics' uses of this test. I will now present an implementation of this test in javascript that could easily be embedded in web interfaces. Although javascript is probably the least likely language to implement statistical methods, I hope this article will fill in as many details as possible to make it trivial to port it to other languages if the need arises. At [...]

By | 2017-01-13T11:01:46+00:00 January 13, 2017|Categories: Javascript, Statistics, Test|0 Comments

Fastest method to compute an AUC

Context: AUC is an acronym for "Area Under the (ROC) Curve". If you are not familiar with the ROC curve and AUC, I suggest reading this blog post before to continuing further. For several projects, I needed to compute a large number of AUC. It started with 25,000, increased to 230,000 and now I need to compute 1,500,000 AUC. With so many AUC, the time to compute each one becomes critical. On the web, I don't find much information about this specific [...]

By | 2016-11-08T09:30:03+00:00 August 18, 2016|Categories: Data Analysis, Performance, Python, R, Statistics|0 Comments

Standard deviation on a correlation scatter plot

I was recently asked by a colleague to provide visualization of differential gene expression computed using RPKM values (two samples, no replicates) and highlight genes that were outside the distribution by 2 standard deviations or more. As a first draft, I quickly obliged by calculating the fold change distribution, computing standard deviation and drawing lines on either side of the diagonal to obtain: This turns out to be equivalent to computing the standard deviation of the residual of a linear [...]

By | 2016-11-08T09:30:06+00:00 April 5, 2016|Categories: Data Analysis, Data Visualization, R, Statistics|0 Comments


Say we have the two following groups : g1 <- c(55, 65, 58) g2 <- c(12, 18, 32) We want to see if the two groups belong to the same distribution or can be considered as different groups. We might be tempted to try a Student’s t-test. t.test(g1, g2) ## Welch Two Sample t-test ## ## data: g1 and g2 ## t = 5.8366, df = 2.9412, p-value = 0.01059 ## alternative hypothesis: true difference in means is not equal [...]

By | 2016-11-08T09:30:09+00:00 October 14, 2015|Categories: Data Analysis, R, Statistics|0 Comments