Overpopulation. It is a word that makes politicians wince, and is often described as the "elephant in the room" in discussions about the future of the planet.

You often hear people citing overpopulation as the single biggest threat to the Earth. But can we really single out population growth in this way? Are there really too many people on our planet?

It is clear to all of us that the planet is not expanding. There is only so much space on Earth, not to mention only so many resources – food, water and energy – that can support a human population. So a growing human population must pose some kind of a threat to the wellbeing of planet Earth, mustn't it?

Not necessarily.

"It is not the number of people on the planet that is the issue – but the number of consumers and the scale and nature of their consumption," says David Satterthwaite, a senior fellow at the International Institute for Environment and Development in London. He quotes Gandhi: "The world has enough for everyone's need, but not enough for everyone's greed."

The global impact of adding several billion people to these urban centres might be surprisingly small

The number of "modern human beings" (Homo sapiens) on Earth has been comparatively small until very recently. Just 10,000 years ago there might have been no more than a few million people on the planet. The one billion mark was not passed until the early 1800s; the two billion mark not until the 1920s.

As it stands now, though, the world's population is over 7.3 billion. According to United Nations predictions it could reach 9.7 billion people by 2050, and over 11 billion by 2100.

Population growth has been so rapid that there is no real precedent we can turn to for clues about the possible consequences. In other words, while the planet might hold over 11 billion people by the end of the century, our current level of knowledge does not allow us to predict whether such a large population is sustainable, simply because it has never happened before.

We can get clues, though, by considering where population growth is expected to be strongest in the years ahead. Satterthwaite says that most of the growth over the next two decades is predicted to be in urban centres in what are currently low and middle-income countries.

It is not the number of people on the planet that is the issue – but the number of consumers and the scale and nature of their consumption

On the face of it, the global impact of adding several billion people to these urban centres might be surprisingly small. This is because urbanites in low- and middle-income countries have historically consumed little.

The emissions of carbon dioxide and other greenhouse gases give us a good indication of how high consumption is in a city.

"We know of cities in low-income nations that emit less than one tonne CO2-equivalent per person per year," says Satterthwaite. "Cities in high-income nations [can] have six to 30 tonnes CO2-equivalent per person per year."

Citizens of more affluent nations leave a much greater footprint on our planet than people living in poorer countries – although there are exceptions. Copenhagen is the capital of a high-income nation – Denmark – while Porto Alegre is in upper-middle-income Brazil. Living standards are high in both cities, yet per capita emissions are relatively low.

Satterthwaite goes on to say that if we look at an individual's lifestyle, the differences between wealthy and non-wealthy groups are even more dramatic. There are many low-income urban dwellers whose consumption is so low that they contribute almost nothing to greenhouse gas emissions.

People living in high-income nations must play their part if the world is to sustain a large human population

So a world with a human population of 11 billion might put comparatively little extra strain on our planet's resources. But the world is changing. Low-income urban centres may not continue on low-carbon development trajectories.

The real concern would be if the people living in these areas decided to demand the lifestyles and consumption rates currently considered normal in high-income nations; something many would argue is only fair. If they do, the impact of urban population growth could be much larger.

This fits with a general pattern that has played out over the past century or so, explains Will Steffen, an emeritus professor with the Fenner School of Environment and Society at the Australian National University. It is not the rise in population by itself that is the problem, but rather the even more rapid rise in global consumption (which of course is unevenly distributed).

This leads to an uncomfortable implication: people living in high-income nations must play their part if the world is to sustain a large human population. Only when wealthier groups are prepared to adopt low-carbon lifestyles, and to permit their governments to support such a seemingly unpopular move, will we reduce the pressure on global climate, resource and waste issues.

A 2015 study in the Journal of Industrial Ecology looked at environmental impact from a household perspective. It puts consumption firmly in the spotlight.

If we change our consumption habits, this would have a drastic effect on our environmental footprint as well

The analysis showed that household consumers are responsible for more than 60% of the globe's greenhouse gas emissions, and up to 80% of the world's land, material and water use. What's more, the researchers found that the footprints are unevenly distributed across regions, with wealthier countries generating the most impacts per household.

Diana Ivanova at the Norwegian University of Science and Technology in Trondheim, the author of the study, explains that the finding comes from simply changing our perspective on who is responsible for emissions associated with producing consumer goods. "We all like to put the blame on someone else, the government, or businesses," she says.

For instance, consumers in the west might argue that countries that produce many consumer goods, such as China, should take responsibility for the emissions needed to make them. Ivanova and her colleagues argue the consumers themselves are just as responsible. "If we change our consumption habits, this would have a drastic effect on our environmental footprint as well."

By this reasoning, there needs to be a fundamental change in the core values of developed societies: away from an emphasis on material wealth, and towards a model where individual and societal well-being are considered most important.

Even if those changes occur, it seems unlikely that our planet could really sustain a population of 11 billion. So Steffen suggests that we should stabilise the global population, hopefully at around nine billion, and then begin a long, slow trend of decreasing population. That means reducing fertility rates.

Creating a sustainable population is as much about boosting women's rights as it is about reducing consumption of resources

There are actually signs that this is already beginning to occur, even as population numbers continue to rise. The rate of population growth has been slowing since the 1960s and the UN Population Division's world fertility patterns show that, worldwide, fertility per woman has fallen from 4.7 babies in 1970-75 to 2.6 in 2005-10.

However, it could still take centuries for any meaningful reductions to happen, says Corey Bradshaw at the University of Adelaide in Australia.

The trends are so deeply set, he says, that even a dramatic catastrophe might not change their course. In a 2014 study, Bradshaw concluded that if two billion people died tomorrow – or if every government adopted controversial fertility policies such as China's recently-ended one-child policy – there would still be as many if not more people on the planet by 2100 as there are today.

What is urgently needed, then, is ways to speed up the decline in fertility rates. One relatively easy way to do so might be to raise the status of women, especially in terms of their education and employment opportunities, says Steffen.

If some or all of us consume a lot of resources, the maximum sustainable population will be lower

The UN Population Fund has calculated that 350 million women in the poorest countries did not want their last child, but did not have the means to prevent the pregnancy. If these women's needs were met, it would have a significant impact on global population trends. According to this reasoning, creating a sustainable population is as much about boosting women's rights as it is about reducing consumption of resources.

So if a world population of 11 billion is probably unsustainable, how many people, in theory, could Earth support?

Bradshaw says that it is nearly impossible to say what this number would be, because it is entirely dependent on technologies like farming, electricity production and transport – and on how many people we are willing to condemn to a life of poverty or malnutrition.

Many people argue that we are well over a sustainable number already, given the lifestyle choices many of us have made and our reluctance to change them. In support of this, they point to the problems of climate change, the biodiversity extinction crisis underway, mass ocean pollution, the fact that one billion people are already starving and that another one billion people have nutrient deficiencies.

In the early 20th Century, the global population problem was as much about the fertility of soil as the fertility of women

A 2012 UN report summarised 65 different estimated maximum sustainable population sizes. The most common estimate was eight billion, a little larger than the current population. But the estimates ranged from as few as two billion to, in one study, a staggering 1,024 billion.

These estimates all depend on so many assumptions that is difficult to say which is closest to the truth.

Ultimately the real determinant is how we choose to run our society. If some or all of us consume a lot of resources, the maximum sustainable population will be lower. If we find ways to each consume less, ideally without sacrificing our creature comforts, Earth will be able to support more of us.

Changes in technology, which are often wildly unpredictable, will also affect the maximum population.

In the early 20th Century, the global population problem was as much about the fertility of soil as the fertility of women. George Knibbs, in his 1928 book The Shadow of the World's Future, suggested that if global population reached 7.8 billion, there would have to be much more efficient use of its surface.

Just three years later Carl Bosch won a Nobel Prize for helping develop chemical fertilisers, the production of which probably did more than anything to fuel the dramatic 20th-Century human population growth.

In the very distant future, technology could lead to much larger sustainable human populations if some people could eventually live off planet Earth.

Earth is our only home and we must find a way to live on it sustainably

In the few decades since humans first ventured out into space, our ambitions have jumped from simple stargazing to the living away from Earth and inhabiting other planets. Many eminent thinkers, including physicist Stephen Hawking, say colonising other worlds is critical for the ultimate survival of our species.

However, even though NASA's Kepler mission has discovered a large number of Earth-like planets, we do not know much about them and they are all very far beyond our reach. So a move to another planet does not offer an imminent answer to our problems.

For the foreseeable future, Earth is our only home and we must find a way to live on it sustainably. It seems clear that that requires scaling back our consumption, in particular a transition to low-carbon lifestyles, and improving the status of women worldwide. Only when we have done these things will we really be able to estimate how many people our planet can sustainably hold.

If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.