To be successful or fulfilled, we all know you need to have specific goals. 

To achieve them, you should visualise, plan your steps there and attach deadlines and incentives. Work hard, even if you hate the work. And never stray from the path. 

Goals can get you to work harder, focus more and perform better. But they also can make you more likely to cheat, kill your creativity, and make you less likely to thrive 

But that outlook, say a growing number of academic researchers, career coaches and thought leaders, isn’t only flawed; it may also, ironically, be keeping us from success.

“We get so emotionally attached to a goal that we’re setting ourselves up for failure and disappointment,” says business advisor, author and speaker Stephen Shapiro. “The key for success is, if you have somewhere you’d like to be in five years, don’t be so attached to it that it drives everything you do.” 

If that makes you say Huh?, you’re not alone. It bucks the self-help industry’s staple advice, not to mention what most of us were told by family and peers growing up. 

It’s true that decades of research show that goals can get you to work harder, focus more and perform better. But they also can kill your creativity, make you more likely to cheat, and less likely to thrive. 

“Goals in themselves aren’t bad,” says Lisa Ordonez, vice dean at University of Arizona’s Eller College of Management. “It’s how we treat them.” 

One of the first problems is the targets people choose, experts say. Many aren’t necessarily our own ambitions, but what we think we should do. 

That disconnect shows up even in the most basic example of what people want versus what they strive for. Most people say their main goal in life is to be happy. But while research has shown that happiness results from simple things like expressing gratitude rather than buying a bigger house, or by prioritising family over career, our ambitions often focus on the latter. Worse, we may sacrifice personal relationships to get there – even though that’s the kind of trade-off most people come to regret.

Chasing outcomes 

There’s a bigger difficulty. For most of us, goals are outcomes: we want to become a lawyer, say, or get married. 

Until you’ve achieved them, though, you can’t know if your goals will improve your life. And even if the present-you would find it fulfilling, it’s impossible to know if future-you will. As life coach and consultant Stever Robbins points out, many of us set life goals in our teens or 20s. But, he says, would you really trust your life to a 20-year-old?

Plus, when you achieve one goal, you immediately… move on to another. The same is true of short-term goals. “When you get there, it’s like, What’s next?,” Shapiro says. “We’re in a spiral of trying to find out the next thing to keep us in action.”

That focus on outcome alone feeds into a hamster-wheel mentality. The Bhagavad Gita, the fundamental Hindu text, communicated the downside of this perpetual motion 2,200 years ago: “Those who are motivated only by desire for the fruits of action are miserable, for they are constantly anxious about the results of what they do.”

Meanwhile, some of the world’s most successful people know that you don’t need to focus on an outcome to achieve it. Take Oprah Winfrey, the US’s third-wealthiest self-made woman. To get to a $3.1 billion net worth, you might imagine that money must have been a main goal. Her secret? "The reason I've been able to be so financially successful is my focus has never, ever for one minute been money,” she’s said.

In fact, focusing on the outcome may make you even less likely to meet it. 

“These self-help books… give you the advice that if you want to lose weight, post a picture of yourself on the fridge that shows you at a weight you’d really like to be again,” says Alexandra Freund, a University of Zurich psychology professor who researches goal pursuit. “What I find in my research is that it’s actually better, in terms of achieving these outcomes, if you do not focus on them.” 

Instead of focusing on your goal, the key may be to focus on process – and forget the outcome entirely

Even visualisation – picturing the outcome you want – can be damaging. Researchers found that people who visualise their goals are less likely to achieve them, perhaps because they trick their brain into thinking they’ve already done the work. (What works much better is ‘mental contrasting’, whereby you visualise not only a positive outcome, but the negative reality you want to change). 

Take weight loss. If you’ve ever dieted, the findings of this study won’t surprise you: when participants who wanted to lose weight slipped up, they were more likely to say ‘I blew it’… and have another piece of cake. But when those who focused on the process of eating more healthily gave into temptations they were more likely to compensate afterwards. As a result, the people who focused less on their goal were more likely to meet it. 

Other research has made similar findings. Study participants who missed a savings goal were more likely to overspend later than those who hadn’t, while subjects who missed a deadline for an assignment were more likely to fail to complete it at all. 

Instead of focusing on your goal, the key may be to learn to focus on process – and forget the outcome entirely. 

Blinders on 

To make matters more confusing, you can forget setting yourself nice, easily attainable targets to achieve your objectives and shield yourself against the negative effects of slipping up. Research shows that tough, specific goals get better results than easy, vague ones. Meanwhile, Freund and others have found that the only participants who got a happiness boost from hitting a target were those who thought it would be difficult to reach.

Easy goals also can limit us. “The weird thing about a goal is that, when you meet it, it doesn’t matter whether you meet it or exceed it by a ton. If your goal is a 5% increase in market share, and you had the opportunity to go further, you find people slack off,” Ordonez says. 

That relates to the other risk of goal-setting: its focus. 

Of course, that’s the point. If your goal is to be a doctor and someone asks if you want to train as a carpenter, you’ll say no. But a blinkered focus on a singular ambition can have unintended consequences.

Corporate incentives may boost performance, but employees also become more likely to cheat, act selfishly and feel demotivated

In her research on goal-setting, Ordonez found that corporate incentives may boost performance, but employees also become more likely to cheat, act selfishly, feel demotivated and even be less likely to learn what they’re supposed to. What she and her co-researchers have found is so compelling that, they wrote in one recent paper, “We offer a warning label to accompany the practice of setting goals.” 

“The first couple of times I presented this to executives, I was waiting for them to say, ‘Go jump off a bridge, this is crazy academic stuff’,” Ordonez says. “Instead, they were like ‘Oh, yeah – this happens all the time’.” 

One example was a call centre that set a goal for its employees: keep the average call time under two minutes. The result? Employees would call a number and, when the person answered, immediately hang up. They met their goal.

That isn’t the only kind of ‘shortcut’ that outcome-oriented goals can encourage. You might think that, if you’ve never gone for a run before, there might be no better motivator than saying you’ll run a marathon in six months. Not necessarily. “You have to be very careful when you apply goals,” says Ordonez. “There’s research showing that if you don’t really know how to complete the task, giving someone a goal that early on messes up their performance. They don’t take the time to learn, practice other strategies and discover the best methods, because they’re just focused on that goal – and they don’t do very well.” 

Rewards can perform a weird sort of behavioural alchemy: they can transform an interesting task into a drudge. They can turn play into work – Daniel Pink

People also can end up taking shortcuts when it comes to ethics. As this analysis outlines, when people were given performance-oriented goals, they acted in more competitive, selfish ways. (Volkswagen is just one recent real-life example as is Wells Fargo).

That’s particularly true if the goals are seen as unrealistic. The tougher the goal is, the more it mentally tires us out, says Ordonez, and makes us more likely to be unthinking about the consequences. In one study, she found that participants who were given a tough goal to tackle first from a series of tasks acted more unethically than those who started with easier goals. That pattern continued even as the difficulty of their goals lessened.

Creativity crush 

Some goals – if they’re tied to rewards – can also hinder creativity and problem-solving. 

In his book Drive, Daniel Pink points to Harry Harlow’s 1940s monkey experiments. When researchers gave monkeys a puzzle, the animals happily tried to solve it. But when the monkeys were offered a reward for figuring it out, they made more errors and solved fewer puzzles.

Something similar happened with artists. In one study by Harvard Business School professor Teresa Amabile, a panel of artists and curators chose the best works of artists – and, without knowing the context, ended up rating works artists had done without a commission as far more creative than commissioned works. 

“Rewards can perform a weird sort of behavioural alchemy: they can transform an interesting task into a drudge. They can turn play into work,” Pink writes. “And by diminishing intrinsic motivation, they can send performance, creativity, and even upstanding behaviour toppling like dominoes.” 

One direction 

Of course, it’s human nature to have aims. Even wanting to re-evaluate your relationship with goals is a goal. So, what do you do? 

For one, be more flexible and go easier on ourselves about tasks and deadlines if we slip up.   

But we also may want to change our whole approach. 

Use goals as a compass, and not as a GPS – Lisa Ordonez

“Use goals as a compass, and not as a GPS,” says Ordonez, echoing a metaphor preferred by Robbins, Shapiro and others. “If you allow goals to guide your general direction as a compass, then when things change, you can realign much easier because you know that goal. Or if the weather changes, and now there’s somewhere else you want to be, you don’t have to miss those things.” 

One idea is to question your motivations, says Robbins. Want be a lawyer? Why? Because you think law is fascinating? So, is it more that you want to do something fascinating, and you think law might be it. Set your compass to “something fascinating, maybe law-related” instead, says Robbins. 

Another way to think of it is as a hike, says Freund. “Of course, you want to get to a certain goal; you want to end at a certain place” she says. 

“But if you just do that, you will probably not enjoy the hike very much. It’s almost the opposite of what you want to do when you go hiking. Mostly, hiking is about enjoying the movement, enjoying the particular challenges and enjoying the landscape.”

In other words, prioritise the journey (the process) not just the destination (the outcome).”

The people who had the least extraordinary lives were the ones who had managed to adhere closest to their plans – Stever Robbins

So instead of focusing on where you want to be in five years, set a goal based on the experience you want to have along the way, says Robbins, who has written about this approach. Setting off on a course that seems painful and unfulfilling? Choose a different goal.

If that sounds frightening, says Robbins, consider what he’s noticed from his interviews with successful people: “The people who had the least extraordinary lives were the ones who had managed to adhere closest to their plans.”

That doesn’t just make the process more fun. It opens you up to more opportunities – and more success. After all, if you take numerous actions in the general direction you want to go, you’ll have a higher probability of getting there than if you make one, laser-focused plan that tries to anticipate the future, Shapiro points out.

If you really want to find fulfilment, you have to do something else, too. “Give yourself entirely to your work, yes. But let go of the outcome. Be alike in success and defeat,” writes Stephen Cope in his book The Great Work of Your Life. That, some say, is the best way to ensure you’re focused on the process and the journey – and, ironically, to wind up getting the outcome that suits you best. For most of us, it’s also the hardest part.

But maybe that shouldn’t be surprising. Despite how it sounds, it turns out that living with fewer goals, and with purpose, direction and openness instead, may be an even bigger challenge than sticking to a plan. But it can be more freeing and fulfilling.

To comment on this story or anything else you have seen on BBC Capital, please head over to our Facebook page or message us on Twitter.

If you liked this story, sign up for the weekly bbc.com features newsletter called "If You Only Read 6 Things This Week". A handpicked selection of stories from BBC Future, Culture, Capital and Travel, delivered to your inbox every Friday.