Patrick

About Patrick

Former physicist turned structural biologist and software developer, he now manages a team of talented bioinformaticians at the platform. Racing god and master barista aside, he expertly handles the next-gen sequencing analysis service and IT infrastructure. Legend has it that he was roaming the future site of IRIC prior to its foundation. He also wrote this entire bio himself in the third person.

Fast network transfers?

Recently, everyone and their mother started using various tools in order to optimize large data transfer to, from and between supercomputers. Historically, we have seen tools like FDT, BBCP that tried to exceed the performance obtained from other transfer methods, like scp, rsync, ftp, etc. One tool in particular is now gaining traction and is being deployed on most supercomputers: GridFTP and its front-end Globus. The Globus frontend interface. Before jumping into the bandwagon, I thought it would [...]

By Patrick|2017-04-29T17:04:17+00:00October 13, 2016|Categories: Computer science, Performance|Tags: file manipulation, test|0 Comments

Speed up random disk access

When working with a software that accesses data from disk in a random fashion, it is common knowledge that best performance will be reached using SSD hard drives, with SAS disks being less efficient and SATA disks being the worst. However, high capacity SSD drives are still relatively expensive and thus, when working with large datasets, one typically ends up working with data stored on larger, and more common SATA drives. I recently experimented with the Jellyfish software to analyze [...]

By Patrick|2017-04-29T17:05:04+00:00August 4, 2016|Categories: Computer science, Performance|0 Comments

Standard deviation on a correlation scatter plot

I was recently asked by a colleague to provide visualization of differential gene expression computed using RPKM values (two samples, no replicates) and highlight genes that were outside the distribution by 2 standard deviations or more. As a first draft, I quickly obliged by calculating the fold change distribution, computing standard deviation and drawing lines on either side of the diagonal to obtain: This turns out to be equivalent to computing the standard deviation of the residual of a linear [...]

By Patrick|2017-04-29T17:05:35+00:00April 5, 2016|Categories: Data Visualization, R, Statistics|Tags: data analysis|3 Comments

Working with large files

When dealing with Next Generation Sequencing data, I am routinely asked by clients how to open sequence files. The answer is that given their huge size (often many million lines) and the consequent requirement in memory, they should probably not be opened in any way, they should only be processed. Most software designed to work with NGS data will then process these files in a sequential fashion or stream, loading just the required amount of data from disk, processing it [...]

By Patrick|2022-06-09T12:35:12+00:00October 1, 2015|Categories: Data Analysis, Shell scripting|Tags: big data, file manipulation|1 Comment

Identifying a point in ggplot2

So you have spent much time converting your simple R plot to a full-fledged ggplot2 graph with all its bells and whistles just to find that you are unable to identify a point on this graph to further investigate it. Indeed, the typical identify method is not applicable to ggplot2 graphs. Fortunately, there is a solution, which involves performing all the work yourself by going under the hood of ggplot2 to access the low-level graphics system on which it is [...]

By Patrick|2017-05-01T10:13:03+00:00March 11, 2015|Categories: Data Visualization, R|Tags: ggplot2|0 Comments