"Daddy, what's that?" says Nicholas, aged six, pointing at a large red-brown insect that has just fallen off a cast iron gatepost. A bearded man in spectacles bends down and, after several seconds of peering intently at the bug, replies, "I don't know."

Hardly an unusual scenario, you might think. Yet it's not every day that Max Barclay, beetle collection manager at the Natural History Museum in London, UK, fails to recognise a large, brightly coloured insect next to his office.

That was in March 2007, and Barclay soon discovered that the mystery bug was cropping up in huge numbers in London and other European cities. However, experts were unsure and divided as to its true identity. It took a couple of years before Barclay solved the puzzle in consultation with colleagues.

It was a plane tree bug (Arocatus longiceps), an insect that feeds on the seeds of plane trees. It has spread rapidly through the cities of central and western Europe in recent years.

With their tarmac and concrete surfaces, and clouds of pollutant-filled vehicle emissions, our towns and cities might not seem ideal habitats for wildlife. Yet some species, including the plane tree bug, are actually more common in cities than in apparently more natural settings. What is it about these species that makes them so fond of city living?

In the UK at least, the most famous urban colonizer is the red fox, which now occurs in its highest densities in towns and cities. A survey published in 2014 found that the distribution of urban foxes had changed markedly over 25 years.

Their agility allows them to exploit the 3D urban environment

Following appeals made during a television series, members of the public submitted more than 17,000 reports of sightings of foxes in urban areas to a website set up by Dawn Scott of the University of Brighton and her colleagues.

Of 65 urban areas in which few or no foxes had been reported in a previous study in 1987, 59 had sightings reported in 2012. Their geographical distribution had also changed, with many more reported in northern and eastern regions.

Scott says foxes are very nimble creatures. "Their agility also allows them to exploit the 3D urban environment, jumping over fences and climbing on roofs to find food, hiding under sheds to rest and make dens."

But they are far from the only wild animals that thrive in cities.

Badgers are sometimes just as common in cities as they are in the countryside. They have adapted their behaviours to survive along dense housing, industry and roads: for instance, they live in smaller groups with smaller territories.

They're generalists so they are better able to cope with change

Scott says both foxes and badgers are able to cope with different and changing environments.

"They are omnivorous and have adapted their foraging behaviours to be able to exploit more anthropogenic food sources," she says. "They're generalists so they are better able to cope with change than other animals that are specialised to specific habitats."

What underlies this adaptability? There's probably no single answer, but research on birds suggests it helps to be smart.

Alexei Maklakov of Uppsala University in Sweden and his colleagues examined this question for a study published in 2011.

They examined records of 82 common bird species that occur in and around 12 cities in France and Sweden. Those with larger brains, relative to their bodies, were more likely to successfully breed in cities. These successful brainiacs included tits, crows and wrens.

It's not just big animals like foxes and birds that thrive in cities

Maklakov says the added burden of running a larger brain might be worth it in environments where learning and adaptation are favoured. These bigger brains act as a "cognitive buffer", allowing the animal to survive changing conditions by adapting its behaviour.

In line with that, there is evidence that urbanisation is driving increases in brain size. A 2013 study showed that white-footed mice and meadow voles from cities had greater cranial capacities than their country cousins.

Maklakov's colleague Niclas Kolm is now taking this idea a step further, and breeding animals – specifically fish – with different brain sizes to see how it affects their behaviour.

But it's not just big animals like foxes and birds that thrive in cities. Plenty of insects do too.

One of the most familiar to gardeners is the black vine weevil (Otiorhynchus sulcatus), which damages many ornamental plants including rhododendrons, ferns and begonias.

It's a very successful insect which is not a very good competitor

Originally from Italy, this flightless beetle is believed to have been spread first by the Romans and later by the British Empire. Nowadays it is common in most urban gardens in the UK but is rare in the countryside, with the exception of the tops of Scottish mountains.

That offers a hint about why it does so well in towns: there aren't many other insects around. "In both urban environments and on mountain tops, the competition is very limited," says Barclay. "It's a very successful insect which is not a very good competitor."

This lack of competition opens the door for the black vine weevil to reproduce in numbers that would be impossible in more diverse settings.

Another reason why some species are more common in towns and cities is that their food sources are more abundant in urban settings.

Humans introduce non-native plants such as rosemary, lavender and leylandii in their highest densities in urban settings. So it is no surprise that the animals that eat them are also to be found there.

Food supply is also a big reason for the success of urban foxes

"With increasing international trade, the herbivores associated with these non-native plants eventually get here," says Barclay. "When they do, it's the equivalent of them entering a room full of money. They have all of this habitat that no other species are exploiting, so they become highly numerous."

The box tree moth (Cydalima perspectalis) is a good example. A native of East Asia, it was first recorded in Europe in 2007 and has now become common in London and other cities in the southern UK. But it is rarely found in the countryside.

Food supply is also a big reason for the success of urban foxes. "The quality of the habitat in terms of food resources is higher compared to rural environments," says Scott.

Another reason why some species are more common in cities is the "heat island effect". Towns and cities are warmer than the areas surrounding them, because their surfaces are made of Tarmac, stone and other materials that absorb and store heat.

British gardeners enjoy the challenge of introducing ever more exotic plants

This effect is particularly significant in Britain, which is on the northern edge of the ranges of many plants and insects. These species occur more often in warmer urban environments.

This helps explain why the oak processionary caterpillar (Thaumetopoea processionea), a pest that defoliates oak trees and causes allergic reactions in humans, has been found almost exclusively in London.

Plants also exploit the heat island effect.

British gardeners enjoy the challenge of introducing ever more exotic plants, which helps explain the preponderance of things like palm trees, olive trees and tree ferns in British cities. These imports sometimes bring their neighbours with them.

Large cities are environments where it helps to be able to cope with a rapid pace of life

"Quite often, other little things from the Mediterranean hitch-hike in on the root balls of these larger imported plants and end up surviving only in warmer urban environments," says Fred Rumsey of the Natural History Museum.

The nettle Urtica membranacea is one such unexpected arrival. It can cope with the byproducts of exhaust fumes.

But perhaps crucially, it is an annual species, growing in winter and flowering in spring.

Large cities are environments where it helps to be able to cope with a rapid pace of life. Annual species, which flower and produce their seeds quickly, do better than the more sedate biannuals.

There is another group of plants that we might expect to do well in cities

"People cut laws, scrub walls and tidy up flower beds," says Rumsey. "So plants that can germinate and create seed in a short space of time stand a much better chance of thriving."

For example, Oxford ragwort was originally grown in the Oxford Botanic Garden from seeds collected in the 17th century from Mount Etna on the island of Sicily. It germinates and flowers very rapidly, and is now common throughout UK towns.

It is believed to have been spread by the railways during the Industrial Revolution, partly because its tiny fruits were carried to new areas in the carriages, and partly because the rocks in the clinker beds used in construction were similar to the lava soils of its native habitat.

Such accidental imports aside, there is another group of plants that we might expect to do well in cities: chasmophytes, which ordinarily live in cracks in exposed rocks and cliffs.

Chasmophytes do well on walls, bridges and buildings. Some, like buddleias, are known to cause damage with their roots.

It is capable of lifting paving slabs and even breaking through asphalt

Others that do well clinging to garden walls are snapdragons and red valerian.

The Serbian bellflower (Campanula poscharshkyana) is reported to be rare in its native Dinaric Alps, but now abundant on the walls of gardens in most British towns.

But to find some true naturals at city living, we have to look beyond plants and animals.

The common ink cap fungus (Coprinopsis atramentaria) is often found on lawns and in disturbed habitats such as urban wastelands. It copes beautifully with the hard surfaces of towns and cities, since it is capable of lifting paving slabs and even breaking through asphalt.

Its Latin name, Serpula lacrymans, means "the creeping maker of tears"

The fungi's stalks are composed of vertically-arranged tubes called "hyphae". They are made of chitin, which is also found in the exoskeletons of insects and crustaceans.

This material's strength, and the arrangement of its fibres in a helix, means the tubes only grow at their tips, providing extraordinary combined force.

Dry rot fungus is not as strong as C. atramentaria, but it is equally at home in the urban jungle. Its Latin name, Serpula lacrymans, means "the creeping maker of tears" – which is rather apt since it is the most damaging of several fungi that cause dry rot in timber in buildings.

S. lacrymans produces white hyphae that can look like cotton wool. These bundles sometimes contain droplets of water, which the fungus transports over long distances to make it easier to decompose wood.

Pulling these examples together, we can draw up a laundry list of factors that can allow a species to thrive in cities.

For those that aren't natural city dwellers, adaptability is the key

"There are two types of species that do well in urban environments," says Maklakov. "There are those for whom urban environments are ideal for them just by chance, so they don't need evolutionary or behavioural adaptations, and those for whom the conditions are hostile, so they do need to adapt to survive."

Those that naturally flourish include anything that prefers to live on the materials humans make use of, such as rock and wood, and those that like the heat. Animals that can't cope with competition may also do well.

For those that aren't natural city dwellers, adaptability is the key. Animals that can switch to new food sources, or use man-made structures as shelters, will do well. This is more relevant for larger animals like foxes and birds, and bigger brains might well be a factor.

Perhaps the most profound lesson nature's urban dwellers offer is a salutary one, about the complex webs of connections between different organisms, and the way humans have disturbed them.

Urban ecosystems are far less diverse than wild ones

When we build high streets, housing estates and roundabouts, we cause havoc for many species. "For a variety of reasons, most species are excluded by human modifications of the environment," says Barclay.

The urban species are the exceptions to this rule. "For those few things that can cope with the modifications, there is less competition," says Barclay. "Finding a place that is difficult to eke out a living in can make sense, if it's somewhere that your natural parasites and predators prefer not to eke out a living."

Still, while there are plenty of individual organisms, urban ecosystems are far less diverse than wild ones.

"If you go to Trafalgar Square, you might see 1000 pigeons, 2-3 starlings, 1-2 sparrows and that's about it," says Barclay. "Whereas if you go out into some similarly-sized patches of countryside you might find 50 species of birds."

It only works for a minority, but for those few species that find the urban lifestyle suits them, it seems city living is paradise.

This story is a part of BBC Britain – a new series focused on exploring this extraordinary island, one story at a time. Readers outside of the UK can see every BBC Britain story by heading to the Britain homepage; you also can see our latest stories by following us on Facebook and Twitter.