Thursday, 21 July 2011

10 rules-of-thumb in genomics

In my semi-ongoing series of "things I wished someone had told me..." I wanted to share my sense of common "gotchas" in genomics. Here's my top 10.

  1. No large scale dataset is perfect - and many are at some distance from perfection. This is true of everything from assemblies through to gene sets to transcription factor binding sites to phenotypes generated cohorts. For people who come from a more focused, single area of biology where in a single experiment you can have mastery of every component if desired this ends up being a bit weird - whenever you dig into something you'll find some miscalls, errors or general weirdness. Welcome to large scale biology.
  2. When you do predictions, favour specificity over sensitivity. Most problems in genomics are bounded by the genome/gene set/protein products, so the totally wide "capture everything" experiment (usually called "a genome-wide approach") has become routine. It is rarely the case (though not never) that one wants a high sensitivity set which is not genome-wide. This means for prediction methods you want to focus on specificity (ie, driving down your error rate) as long as one is generating a reasonable number of predictions (>1000 say) and, of course, cross validating your method.
  3. When you compare experiments you are comparing the combination of the experiment and the processing. If the processing was done in separate groups, in particular with complex scripting or filtering, expect many differences to be solely due to the processing.
  4. Interesting biology is confounded with Artefact (1). Interesting biology is inevitably about things which are outliers or form a separate cluster in some analysis. So are artefacts in the experimental process or bioinformatics process - everything from biases towards reference genome, to correlations of signals actually being driven by a small set of sites.
  5. Interesting biology is confounded with Artefacts (2). There is a subset to the above which is so common to be worth noting separately. When you have an error rate - and everything has an error rate due to point 1 - the errors are either correlated with biology classification (see point 2) or uniform. Even when they are uniform, you still get mislead because often you want to look at things which are rare - for example, homozygous stop codons in a whole genome sequencing run, or lack of orthologs between species. The fact that the biological phenomena you are looking for is rare means that you enrich for errors.
  6. Interesting biology is usually hard to model as a formal data structure and one has to make some compromises just to make things systematic. Should one classify the Ig locus as one gene, or many genes, or something else? Should one try to handle the creation of new selenocystine amber stop codon by a SNP as a non synonymous variant? To what extent should you model the difference between two epiptopes for a chip-seq pull down of the same factor when done in different laboratories? Trying to handle all of this "correctly" becomes such a struggle to be both systematic and precise one has to compromise at the some point, and just write down/reference papers a discussion in plain old English. Much of bioinformatics databases is trying to push the boundary between systematic knowledge and written down knowledge further; but you will always have to compromise. Biology is too diverse.
  7. The corollary of 1, 2 and 4 is that when most of your problems in making a large scale dataset is about modelling biological exceptions, your error rate is low enough. Until you are agonising over biological weirdness, you've still got to work on error rate.
  8. Evolution has a requirement that things work, not that it's an elegant engineering solution. Expect jury rigged systems which can be bewildering in their complexity. My current favourite is the Platypus X chromosomal system which is just clearly a bonkers solution to hetreogametic sex. Have fun reading about it (here's one paper to get you started)!
  9. Everyone should learn the basics of evolution (Trees, orthologs vs paralogs. And please, could everyone use these terms correctly!) and population genetics (Hardy Weinberg equilibrium, the impact of selection on allele frequency, and coalesence, in particular wrt to human populations). For the latter case people often need to be reminded that the fact a site is polymorphic does not mean it is not under negative selection.
  10. Chromosomes have been arbitrarily orientated p to q. This means that the forward and reverse strand have no special status on a reference genome. If any method gives a difference between forward and reverse strands on a genome wide scale - it's probably a bug somewhere :)

I am sure other people have other comments or suggestions ;)

Wednesday, 13 July 2011

Biology is a 1st class quantitative science

Last week I was at a "Town Hall" meeting for the UK High Performance Computing community (or at least part of it) which has a lot of computer people rubbing shoulder with what used to be called "hard" science scientists; high energy physics, astronomers, modellers (particularly climate change/oceanography) and the odd molecular modeller/chemist. High beard/non beard ratio, and almost 1:1 Y to X chromosome ratio. From their perspective I was a straight up biologist (though from the biologist's perspective, I am a computer scientist. The curse of being interdisciplinary is that you are often seen as on the outside of all the disciplines you span. Hey ho.).

At that meeting one person almost inadvertently stated the standard science hierarchy - something like (this is not a direct quote): we are here to serve the hard, mathematical sciences of physics and climate change, and bring in new quantitative sciences of molecular biology and even reach out to Arts and Humanities based research, such as Dance research.

At one level this is great - we (biology) are both fully acknowledged, and we're no longer the newest in town! It's the Arts and Humanities who are the newest.

But I do find it a bit frustrating, and when it came to my talk I ended a slide with a bullet point

"Biology is a 1st class quantitative science"

I really believe this. You can't really move in biology without having to do at least some number crunching, and the fully quantitative end of biology - be it genomics, image analysis or systems biology - has easily enough quantitative/mathematical/statistical meat on it to be totally at home with astronomy/physical chemistry or oceanography/climate science. And the extent to which biology is not quantitative I think is far more due to the 20 years of us getting away with "cartoon based" explanations of our data (which worked out fine for quite a long time realistically) and I dont think is inherent.

In short, in 50 years time I think biology will be (or should be) reunited to its quantitative roots where it started at the turn of the 20th Century, in particular the way that frequentist statistics developed mainly to tackle biological questions.

But this idea of Biology as a "soft" or "semi-soft" science is pervasive - in the expectations we have of students, in the way we train graduates, and in the way we deal with peer review on grants and papers. Even the Royal Society has an "A" section (old school "hard" science) and a "B" section (old school "soft" science). So we've got almost a generational change to execute here (which also concerns me; I am probably both interdisciplinary and intergenerational in this aspect. sigh).

There is a subtle but deep cultural change needed to place mathematics and statistics in a central position in biology; this has to go across every part of the science. I dont mean we all have to understand Fields, Tensors and high dimensional mathematical topology (and nor do most chemists/material scientists/climate modellers....), but just as a practicing physicist of any sort - from material science through to experimental physics must be mathematically conversant at some level - ditto chemist, ditto climate modeller, so every biologist must have some fundamental mathematical skills - mainly associated with statistics. It's just part of the science we do.

So - say it proudly:

"Biology is a 1st class quantitative science"