About the author
It has been mentioned a couple of times that chemistry and baking can be quite alike. Just as in baking, doing decent chemistry requires recipes from which you can eventually develop even more awesome recipes to make the best cake! And if you do…give me a call, I would volunteer to try it.
But now imagine this: you try to bake a cake according to this fantastic recipe you found online – and it doesn’t work. It just doesn’t work! No matter how hard you try and how closely you stick to the recipe. Your cake is always very, very different to what the image shows and it doesn’t taste like something you want to share with your friends.
Unfortunately, this does happen in chemistry from time to time. It is very frustrating because you would expect those things to be picked up in the peer-review process (probably most of them are). ).
It’s also really problematic, because all researchers completely rely on published procedures. If I want to make a molecule, I cannot restart chemistry from scratch – I rely on what other people have published.
After this happened to someone from our team recently, we had a discussion in our lab as to why there are so many procedures that have flaws or are difficult to reproduce even for experienced, highly skilled chemists. A 3rd year PhD student will have made more than 100 cakes, so it should be absolutely possible to make another one.
So what’s the problem?
1.) Being too general: this is probably the most common problem. Imagine you are baking a quite challenging cake and the difference between perfect taste and completely burned lies in just 5 minutes of heating in the oven. If the description isn’t really precise (leave 25 minutes at 207 °C) but more general (cake was baked in the oven) then you will struggle. Chemistry procedures tend to be too general (especially for the purification after the reaction).
2.) Missing details in the procedure: sometimes chemists do things intuitively. We all have our own ways of working. That might be good for that person, but it is really bad for everybody else if it is not stated in the paper. My industrial supervisor told me of a story where he couldn’t reproduce a reaction because he was working in standard conditions for that type of reaction (exclusion of sunlight) but in fact the reaction required sunlight to kick off. It just wasn’t described in the procedure!
3.) Cheating: pressure is a problem in almost any research and work field. That does also apply to chemistry. Your chances of securing a good job rely partly on you publishing some work during your PhD. Professors depend on publications from their students to secure further funding to pursue their research, therefore they tend to put quite a bit of pressure on their students. But what if experiments just don’t work and people just don’t have (positive) results…It is no surprise that a fair number of faked studies come from the most competitive institutes.
Cheating in baking: I would buy my cake in the bakery
Unfortunately, flawed studies have a negative effect on many different levels. They make work for all other chemists in universities and industry much harder, but they also have a massive influence on the way people perceive science and on decision-making in politics (e.g. research funds). We need to get our science right so that other chemists, the public and decision-makers on all levels can rely on our work.
The effects of bad science can be devastating, as Ben Goldacre demonstrates in the talk below:
So how do we get things right? In our discussion we came up with a couple of points, but I really believe this point has to be subject to discussion coming from every chemist or scientist out there:
be more detailed and expect more details in experimental procedures. It happens really too often that procedures leave you in the dark about crucial steps in an experiment. “Mixture was purified via chromatography” is really not sufficient and provides as much information as “flour, eggs and milk were used for this cake”. Be precise!
make small videos of major steps in your reaction. This could be especially useful for the small things you do intuitively in a lab. Give some useful comments about the most challenging aspects of the reaction; tell us what can go wrong!
when you read a recipe for a cake online, you usually check the rating and comments by other users first. This will tell you straight away how well the description is written and how high your chances of success are. I believe all scientific papers should have such a comment section in which people describe their experiences with a method. This could give us such a massive boost of information and help researchers find the most effective protocols.