If at first you don’t succeed, try, try again. Words of wisdom heralded throughout our upbringing, to be sure. But is there any scientific proof that successive failure is positive and propels innovation forwards?

Tom Pohlmann, head of strategy and value at Mu Sigma, a decision science and data analytics firm, says there is, having studied in depth the idea of successive failure. “The rapid change of pace in business puts companies under pressure to innovate constantly, new technologies are making it possible to meet this challenge through on-going experimentation,” he said. Mu Sigma’s report shows failing fast and often is the best approach and the key to success in many areas of business.

The Mu Sigma researchers define this in their analysis as “the power of extreme experimentation”, claiming that science can demonstrate that failure drives forward innovation. This approach is echoed by engineers working in the pharmaceuticals, material sciences and automotive industries. “Those at the forefront of technology have to fly into mountains,” explained Ray Gibbs, CEO of Haydale, a material-science company based in the UK, US and South Korea that works with graphene to improve the properties of everyday materials such as inks and coatings. Gibbs believes that in order to develop any successful product you must try lots of different ideas to get to that end result, learning from the failures.

Failing isn’t bad for business. It just leads to something else happening, which, if carried out correctly builds upon that failure, adds Pohlmann. Even the mighty Apple had its share of failures, such as its early hand-held computer, The Newton, which Steve Jobs swiftly killed off.

Rethinking your approach

“A lot of people still think of failure as a sign of personal incompetence and try to avoid it at all cost,” said Andrew Filev, CEO and founder of Wrike, a software firm in Mountain View, California. “But when you view building a business as a series of experiments, you start to see failure as an inevitable step in the process.”

Filev’s firm had to learn from its early failure. “We started off providing project management services to other companies. However, as we were managing projects for our customers, it became clear that we ourselves needed a tool to work together better. We built Wrike to do just that, and soon realised that this was a problem for many other businesses, too.” The result: Filev launched an entirely new company.

I don't know anyone who has built a business that hasn't gone through a lot of challenges and had to make changes accordingly. It would be boring if it was easy.

Similarly, Lopo Champalimaud, CEO of Treatwell, also had to reshape the entire business. The firm, which started off as “daily deals” site Wahanda, transformed into an online hair beauty-treatment booking platform. “I don't know anyone who has built a business that hasn't gone through a lot of challenges and had to make changes accordingly,” Champalimaud said. “It would be boring if it was easy."

Of course, constantly refining an innovation is nothing new. Every design from the steam train to the domestic cooker has gone through several iterations. “Any innovation has involved risk. There’s always a danger in trying something out that it won’t work” explained Stitian Westlake, executive director of research at NESTA, an independent organisation that works to increase business innovation in the UK.

Changing tides

For many firms the financial crash in 2007 meant this creative process came screeching to a halt. It made companies more hesitant to take risks, more afraid of competition and time and cost pressures.

But almost a decade later technology companies and digital start-ups are putting experimentation back on the map. A strategy helped by advances in technology — from Big Data, to cloud computing, reducing the cost of experimentation dramatically. The growth of the digital economy means “knowledge is becoming faster than ever, forcing companies to think on their feet,” Pohlmann said.

Any innovation has involved risk. There’s always a danger in trying something out that it won’t work. 

One of the most important areas this applies to is in software development. Unlike, for example, building a cooker or a freezer, software is an industry where it is easy to try something out and then send updates to customers to “iron out any creases”. At OpenStack, a firm that creates open-source software for data centres, the “fail fast and often” philosophy has shaped innovation from the outset. The company defines itself by experimentation with new technologies, with the underlying view that real breakthroughs don’t appear on traditional long-term roadmaps but come from taking risks.

“Think about an app on your smartphone” said Westlake from NESTA. Because the cost of experimentation is lower, it is easier to take the “fail fast and often” approach in this industry. This is something which couldn’t be done for more old-fashioned hardware products. “When an industry is being changed by technology, there may be more benefits to explore and take risk,” he added.

Obsolete before you launch

Take the car industry; “Luxury cars go out of date even by the time they are shipped [to salesrooms]. But then you look at car companies such as Elon Musk’s Tesla. Soon updates to the onboard operating systems can just be beamed out,” Westlake said. On-board operating systems in vehicles, such as sat navs, used to become out of date fast. Now just like your iPhone this software can simply be updated remotely.

The retail, music and pharmaceutical sectors are also jumping on the innovate-as-you-go approach. “This can feel unsettling but it’s actually a good thing, and a necessary thing,” said Pohlmann. “It forces companies to shed predetermined concepts and switch to a mode of constant experimentation and learning.”

For Pohlmann, innovation is often like throwing darts at a moving dartboard: you can either try to throw darts more accurately, or you have to throw more of them to increase the probability of hitting the bull’s eye. 

To comment on this story or anything else you have seen on BBC Capital, please head over to our Facebook page or message us on Twitter.