We are using cookies to give you the best experience on our site. To find out more see our Cookies Policy. By continuing to use our website without changing the settings, you are agreeing to our use of cookies.

How You Turn a Flash of Insight into a Groundbreaking Idea

Matthew Syed, author of Black Box Thinking, on harnessing failure’s hidden power.

Perhaps the simplest route into Black Box Thinking is through the idea of marginal gains. This is the method through which British cycling has become so successful. The idea is that if you can break the problem of winning a bike race into all of its component parts and improve each of them by as little as one per cent, the accumulation can be extraordinary.

The general manager of Team Sky, Sir Dave Brailsford, who is perhaps most associated with this approach, improved bike design, diet, the aerodynamics of the skin suits, and even started taking mattresses from stage to stage during the Tour de France to improve sleep quality. These little changes, and dozens more, have had a powerful effect. British Team Sky riders have won the Tour de France four times in the last five years.

Successful teams are not afraid to adapt. They are not fearful of finding their own weaknesses, for that is how to address them. Psychologically, however, it is difficult to engage with our failings, because we like to think of ourselves as smart and competent. One has to flip this perspective to unleash the curiosity that is the hallmark of the greatest individuals and institutions.

I was inspired to write the book not just because of sport, but also because of a long study of the aviation industry, which learns systematically from the data of near-miss events and accidents recorded by the famous black boxes. Indeed, many of the most seminal improvements to system safety have happened via accident investigations. As with Team Sky, it is by finding the weaknesses, by probing the lessons of failure, that we achieve success.

As with Team Sky in cycling, it is by finding the weaknesses, by probing the lessons of failure, that we achieve success

A key moment in the formation of the book was an informal dinner with a surgeon, a friend of my wife. I asked what I thought was a friendly question: “How do you improve over time? What sort of data do you use to address your weak points?” To my surprise, he was somewhat offended by the question. The implication was that, as a consultant and senior leader, he didn’t have any weak points. Indeed, he was so defensive that I looked deeper.

It was then that I was shocked to learn about how the healthcare industry responds to errors and failures, particularly those that lead to patient harm. Instead of learning, as aviation would, they tend to cover them up. Many senior doctors struggle to own up to them, because it is too big a threat to their self-image, and so they use euphemisms to distract attention from failure: “it was just one of those things”, “we did everything we could”, etc.

There is also a problem of high blame. If clinicians anticipate being unfairly penalised, or even litigated, for honest mistakes, they will not volunteer the information. The overall effect is that failure is supressed, destroying the adaptive process. This is one reason why medical error is one of the biggest killers in the western world. According to one paper, 400,000 people die in America each year because of avoidable mistakes in hospitals. That’s like two jumbo jets crashing every day. The data in the UK is also shocking. Unless the culture changes, it will prove difficult to reduce these numbers.

Naturally, I tried to apply black box thinking to the writing of Black Box Thinking. I didn’t put a formal pitch to a publisher, but instead wrote the early chapters before meeting with potential editors. Once I had a publisher on board, I asked colleagues and friends to look at early drafts, to comment on the weaknesses, so they could be rewritten. I just wish I had got into the adaptive process earlier, and iterated the book rather more. I would doubtless have found many more marginal gains.

It is noteworthy, in this respect, that Pixar, the animation company which created Toy Story, Finding Nemo and Monsters Inc, has a powerful adaptive process at the heart of everything it does. As the animation gets into operation, each frame is subject to testing and feedback. All told, it takes around 12,000 storyboard drawings to make one 90-minute feature film, and because of the iterative process, story teams often create more than 125,000 storyboards by the time the film is delivered.

The key thing with an “ideas book” is, I think, to fuse evidence, ideally underpinned by randomised control trials, with stories that convey these ideas. It involves searching the academic journals (which I love doing), and then weaving the findings into a narrative. And this is why writing a non-fiction book is, above all, a learning experience. No book is ever really finished.

Matthew Syed is a columnist for The Times, a two-time Team GB Olympic table-tennis player and the author of Bounce: The Myth of Talent and the Power of Practice and Black Box Thinking: The Surprising Truth About Success.