A former resident and BBC correspondent returns to the city to find out how it has changed since the handover and his time there.

During the final minutes before light, the harbour is shrouded and all its sounds muffled. Waiting for the Star Ferry I see the winking lights of tug boats as they fuss around a cruise ship. I can hear fog horns and grumbling engines and then countless lesser sounds: anchor chains grinding, a Chinese military helicopter heading for the New Territories, small waves slapping at the sides of fishing boats. At this hour, the city smells of the night’s dead heat mingled with oily water.

I feel a pulse of joy to be back. Hard to think that this was a city I first came to in a state approaching despair. Switching roles from BBC Southern Africa correspondent to cover Asia, I found myself longing for the open spaces of the continent I'd left behind. And then, in the last year of British imperial rule, my first child was born here. I had an intimate connection with the place. I made Chinese friends and learned that it wasn't a question of there being two Hong Kongs - one brash and commercial, the other Confucian and Cantonese. There were many Hong Kongs, all intertwined, a place where the traveller could be confounded by difference and cosseted by the familiar all in the same day.

Now I feel a childish enthusiasm crossing Victoria Harbour on the Star Ferry. Tubby and unlovely, with hard wooden seats, the ferry boats have plied these waters since the late 19th century. An Indian, Dorabjee Nowrajee, who made his fortune selling opium into China, set up the Kowloon Ferry Company in 1888. But the old drug dealer was not immune to the romance of the Hong Kong waters. A lover of poetry, he named his new ferry service after some lines of Tennyson, from Crossing the Bar, which contemplate the afterlife:

Sunset and evening star,
And one clear call for me!
And may there be no moaning of the bar,
When I put out to sea...

The long departed Mr Nowrajee understood that while Hong Kong might gain its vitality from the ruthless pursuit of profit, it was also a place of spiritual questing and romantic longing.

Within minutes of disembarking on the "Hong Kong side", I plunge into Wan Chai market, where plastic packets of dried seafood - some with medicinal qualities - are sold alongside knock-off Calvin Klein underwear. A little over £1 will see you happily munching starfish while draped in the finest pants a mainland sweatshop can produce.

On Bowrington Road, one of the small streets that bisect the area, there is a row of fish stalls where the produce flips, wriggles and claws in large tanks. The eternally elastic vowels of the Cantonese language bounce around the lanes as the traders harangue suppliers, and shoppers demand only the fattest fish and are told in return that they will have to pay more. The local Cantonese are perhaps the most enthusiastic consumers of fish anywhere in the world.

It is striking how many of the stalls are run by people who in the west would be dismissed as too old to work. An elderly woman pushes through the crowds hauling behind her a cart filled with vegetables. Dressed in the ageless garb of the Hong Kong grandmother - a dark, work-worn smock and trousers - she disperses all comers with a loud cry. A group of the elderly are taking a short break on the corner of Spring Garden Lane, gossiping raucously and resembling not an image of old age but of boisterous teenagers, determined to give no ground to the crowds thronging around them.

Noodles for breakfast
There is a noodle shop nearby where I try to have breakfast whenever I visit Hong Kong. Dickie Kwong's place always feels like a heightened version of the tumult outside. The diners are packed into a small room facing onto the street. Posters of the house specialities adorn the walls. Noodles with fish balls, noodles with pork, noodles with chicken etc. There is a delicate choreography of elbows and chopsticks as diners take care not to encroach on their neighbours' territory. In the corner Dickie and his cooks are enveloped in steam rising from vats of boiling water and broth. Every few seconds a heap of ivory white noodles is scooped into a bowl and dispatched to a diner.

Dickie offers good food at a price recession-hit Hong Kongers can afford. Like many of his customers, he grew up on stories of hard times. His parents fled the Communist takeover of China in the 1940s. When 1997 and the handover to China came around, he was one of those who felt trepidation about the future. Would the Communists kill the city's enterprise and his own dreams of prosperity with it? It didn't happen and these days he is a happier man. "Business is good. Booming, yes," he says. There was bungling by the new administration in the early days but massive demonstrations saw off an attempt to bring in draconian security laws. This success boosted local confidence. "Now we feel there is a good future," Dickie tells me. "I don't worry for this recession. Hong Kong always comes back."

One of the best places to reflect on the city's changing fortunes is just across the road from the race course at the Happy Valley cemetery. Here, the colonial graves testify to the endurance that has sustained Hong Kong. The roots of ageing West Indies mahogany trees circle the graves of sailors, soldiers and missionaries. All died in the service of the most improbable of all imperial adventures: the making of a great city on a barren rock in the South China Sea.

Walking across the graveyard I hear the city's traffic dwindle to a low hum. A local woman is cleaning the graves with all their carved pieties and lingering whispers of empire. I come across the grave of Henrietta Hall Shuck, the first American woman missionary to China, who died in Hong Kong in 1844, aged 27, broken by overwork and the climate. A nearby monument recalls the British and American sailors of the ships USS Powhatan and HMS Rattler who died fighting pirates off the coast of Hong Kong in the 1850s; another marks the graves of the 95th regiment, swept by fever in 1848: Nine Sergeants, Eight corporals, Four drummers, 67 privates, Four women, Four children.

So much of British and Chinese history is recorded here. For this is not only the resting place of dead colonials - on many of the stones are traced the details of China's tormented relationship with the West. It is all here: opium and exploitation, bravery, brutality and sacrifice.

Temples, and fortunes told
It puts the traveller into a properly reflective state of mind for a visit to Man Mo temple above Central business district. To reach Man Mo you first pass through the world of Gucci, Rolex and Starbucks, where wealth shimmers on the glass walls of the skyscrapers. I used to be repelled by the relentless materialism of this area until I started to see it as the extraordinary result of colliding historical impulses: the imperial urge to trade, the native river people's culture of barter, and the powerful desire to succeed of the refugee Chinese who fled here from the trials of civil war.

There is an escalator which climbs towards the temple from just above Central, gliding through the streets of Mid Levels past numerous new restaurants and bars where expatriate businessman, shirt-sleeved and tie-less in the afternoon heat, brag of deals and speculations.

At the corner of Ladder Street a small crowd has gathered at the entrance to Man Mo. Some are offering imitation money to be burned in the temple furnace in supplication to their ancestors; others place offerings of fruit, fast food, drinks and flowers inside the door. Coils of incense hung from the roof fill the crimson and gold rooms with a cloying smoke.

There is a small shop attached to the temple and in the corner a lugubrious figure sits reading his newspaper. Mr Ng is the temple fortune-teller. He seems very bored but not so pressed by tedium that he finds the prospect of my company a joy. He is polite but there is a discernible hum of irritation. Perhaps he has had to deal with too many lumbering westerners.

"What do you want to know?" he asks.

I am momentarily flummoxed. I had thought he would simply read my palms and recite a glorious future.

"You must have some thought!" he snaps.

"Er, I would like to know if my son will be happy in his life," I say.

"What do you mean happy? Happy in work or in love? In what happy?" he shoots back. I settle for an area that is always fertile ground in education-obsessed Hong Kong. "Happy in his studies," I reply.

Mr Ng nods and produces several coins, which he proceeds to shake. He then makes some calculations in Chinese. "Your son must not have a closed mind. This is what you need to work against," he says firmly. And that is the end of that. I am relieved of £8 and dispatched with the ghost of a smile.

Across the road from Man Mo, the antique dealers and bric-à-brac sellers of Cat Street are doing a modest trade. There is a handful of mainland Chinese browsing here, noticeably better dressed than in the old days when their badly cut hair and cheap suits ignited sniggers from the locals.

Here you find a vast array of Maoist memorabilia for sale. There are Chairman Mao watches, copies of the Little Red Book in every size, statues of Mao and Mao badges. Mr Ming is tending a stall so Maoist, it looks like a mini-museum of the Cultural Revolution.

"Who buys this stuff?" I ask.

"Only the foreigners, only the tourists. Not the Chinese," he tells me.

"But he killed millions and yet people still buy it?" I ask.

At this, Mr Ming laughs. It is a long laugh as if I have entirely missed the point. Business, after all, is business.

At some point in Hong Kong, no matter how careful you are to keep to quiet streets, the density of the city will start to close in. There are more than 6,000 people to every square kilometre and there can be days when it sounds as if every one of them is talking at the same time. So when the claustrophobia strikes, I head for the islands. There are 262 outlying islands and several are served by regular ferries. For me the pick of them all is one of the smallest and most remote.

To reach Kat O, up near the border of the Chinese mainland, you need to take a taxi to the pier at Wong Shek, about an hour's journey from Hong Kong. Riding down the hill towards the harbour with my friend and translator, Choi Li Hung, we pass cattle grazing by the roadside. There are no high-rises out here and the roads are narrow and winding with glimpses of the South China Sea at every other turn.

The day is gloriously sunny with a warm wind blowing across the Pearl River delta. To make the final stage of our trip to Kat O, Hung has arranged a speedboat taxi. There are several speedboat owners who will take a small group to Kat O and wait to bring you back for around £80.

Of course there are cheaper and easier islands to visit. A round trip to Lamma Island, much nearer but also far busier at weekends, is roughly £2, while the island of Cheung Chau with its numerous seafood restaurants and hiking trails can be reached in 30 minutes for a similar price. So why travel further and spend more?

Quite simply because Kat O is one of the world's most spiritually refreshing places. From the moment we head out of the harbour I feel a growing lightness. It is partly to do with the joy of emptiness after so much crush, but also the certainty that out on the pastoral edges of Hong Kong, I am beyond the reach of all vexation. As we pass the stilt houses of the fish farmers, I let my hand fall into the water. It is warm and clear and shoals of tiny fish flicker back and forth beneath the boat.

Retreat to the island
Kat O covers an area of just 2.3 sq km and was named Crooked Island by the British because of its twisting coastline. It once had a population of several hundred; now there are about 50 people living here. The young have gone to Hong Kong or to the cities of Britain. What remains is a small society of Hakka and Tanka fishing clans, eking a living along the coves and inlets, their villages dotted with abandoned houses into the recesses of which the jungle steadily creeps. The village stretches out along either side of a short pier. At the head of the pier is a shop cum restaurant belonging to Wing Gei and his wife Aida. A sign tells visitors that if the owners are away they can take a drink and leave the money.

Gei is drinking tea with a friend when Hung and I approach. He smiles, jumps to his feet and offers us seats. Cups of steaming tea appear. Gei has lived here for most of his life, apart from a brief period trying to do business in Hong Kong. "I am a quiet man," he explains. "I am not aggressive and I am trusting. It's very aggressive out there, you know." He waves a hand towards the coast. Gei tried a few different businesses but could not make them work, so he came home to the island. "I like it here because it is quiet. I don't even lock my door at night. I know everybody. Can you understand that living in a place becomes part of your nature?"

I nod. At that moment the village policeman appears. He smiles and goes off in the direction of some boats. "What does he do?" I ask. There is no crime on the island, Gei explains. I wonder if his presence is intended to deter the people-smugglers across the border in China. So called "snakehead" gangs have been responsible for smuggling thousands of illegal migrants out of China into Hong Kong every year. Mr Gei's friend chips in, "In the old days, lots of people used to try to get to Hong Kong from China by swimming across here. Not all of them made it of course: we used to see the bodies. But if they did make it across, the families here used to feed and help them."

Further along we meet an old couple who are roasting cashew nuts in an outdoor oven. The man's name is Woo Tan and he is 73 years old. He sells me a bag of nuts for a pittance and refuses to accept a tip. Woo has one of those marvellously austere Chinese faces, a countenance forged by hard work and pride. Do I know that the Hakka people have been on this island for more than 300 years, he asks. His people can trace their presence here back to the Qing dynasty. The Hakka are renowned for their resolute determination to preserve their traditions and their ability to survive in the least promising of environments. Migrating from the north to the coastal regions of South China, they established a thriving fishing and farming culture.

We say goodbye to the old man and climb to the top of the island where we watch black kites circling while a solitary fisherman steers his boat into the shallows. Across the water in China, a container port looms in the heat haze. An old woman with a huge and fierce-looking dog passes by. 'Don't worry,' she says, "the dogs here don't bite."

Hunger eventually brings us back to the pier and the welcoming smile of Wing Gei. A pot of soup is produced, thick with vegetables and ginger, followed by a plate of chickens' feet and, after that, the speciality of Kat O, cuttlefish balls. This combination of moulded pork fat and thick white cuttlefish takes some chewing but, served in a good broth of spring onions, is delicious. Our little group slurps and burps happily. And then the boatman drums his fingers on the table and smiles. Island time is running out. Gei says we are welcome to stay, but he knows we cannot and accepts our refusal with a sad shrug. "I promise you I will be back," I tell him. I will Mr Gei. I most definitely will.

Originally published May 2009

The article 'Where worlds collide' was published in partnership with Lonely Planet Magazine.