The Hamptons may be best known for their palatial beachfront houses and the whiff of old money – but there’s a lot more to Manhattan’s favourite getaway.

It’s early evening on Wainscott Beach, a well-hidden strip of white sand and pounding shoreline a few miles beyond the village of East Hampton. A decidedly chilly breeze is gathering strength. During the summer season, hurricanes arrive in this exposed part of Long Island as reliably as the Cannonball trains from New York City, but tonight, at least, there’s plenty of warmth on offer from a blazing firepit and a circle of tall, open-flame torches. They glow steadily brighter as the sunlight fades.

For the Wiesenmaier family, sitting around a long, fold-down picnic table, drinking wine from plastic cups and preparing for a clambake of lobster tails and slow-roasted potatoes cooked by two chefs on an enormous outdoor grill, this pristine corner of the US eastern seaboard has tonight been transformed into America's most exclusive restaurant.

'It's hard to beat, right?' laughs Kurt Wiesenmaier, a 27-year-old New Yorker, who has organised the dinner for his mother's birthday. He adds that although his parents have owned a second home in nearby Three Mile Harbour for decades, 'They've never done this before. I don't know why - I mean, look at the view: it's so good, you can take a swim in it.'

Aside from this small gathering, the beach is empty; nothing but pristine sand dunes and surf, all the way to the almost Northumbrian-looking Georgica Pond. The pond is in fact a coastal lagoon, where Bill Clinton set up a 'summer White House' during the worst days of the Monica Lewinsky affair. The seclusion isn't an accident: unless you've been given specific directions for navigating East Hampton's identical-looking back roads, many of them lined by towering, maze-like hedges, access to the beach can be near impossible to find. Parking is also a challenge. A resident's permit helps greatly, the cost of which could easily run to, oh, £20 million, or whatever the going price is for one of the iconic, white-painted, shingle-sided beach houses.

Current permit-holders include Sir Paul McCartney, Steven Spielberg and Lloyd Blankfein, chief executive of Goldman Sachs. For everyone else, there are bicycles, which can be parked anywhere. No-one bothers with locks: you could pretty much leave your wallet and keys on the seat, return a couple of days later, and they'd be exactly where you'd left them.

All of which, of course, is exactly what has come to be expected of the Hamptons, a broad term applied to the preppy towns, villages and hamlets at the East End of Long Island, originally settled by the Montaukett tribe of Native Americans, then seized by English colonists in the mid-17th century. This semi-rural peninsula of New York state, which juts out into the Atlantic in two thin forks, has become world-famous for its extraordinary wealth and privilege, and the high society of polo games, lawn parties and sports cars that go with it.

But it's not just the scale of the wealth that's notable in the Hamptons, it's the age of it. In some cases, the money is so old it predates the US itself. Take the Gardiner family, for example: they bought their private island from the Montauketts (for some powder and shot, a few Dutch blankets and a dog, allegedly) some 372 years ago, and were duly awarded a royal warrant from Charles I, which, as far as anyone knows, remains in effect today. In theory, this means they can perform trials and hangings at their leisure.

Disappointingly, the Gardiners have never found a reason to exercise this right, not even when Captain Kidd tried to bury some pirate treasure on their land; or, a few centuries later, when Ernest Hemingway came ashore for a spot of game hunting and, so it's been claimed, let off a round at what he thought was the glint of a deer's eye... but turned out to be a passing car.

This incident brings to mind a lesser-known fact about the Hamptons: it's a place where conservative Wall Street tycoons have long co-existed with artists, many of whom love the delicate, ever-changing light that results from the region being almost entirely surrounded by sea. Jackson Pollock, the abstract expressionist - who presumably didn't have an interest in his stockbroker neighbours - resided in the hamlet of Springs until he died there in a car crash. Today, his disciples can take painting classes at The Art Barge in Amagansett, founded by Victor D'Amico, the first director of education at New York's Museum of Modern Art. Likewise, Truman Capote and Kurt Vonnegut summered close to Wainscott Beach, fetching supplies every day from the doll's-house-sized Sagaponack General Store. It's still there to this day - a place to buy a coffee, a copy of The New York Times and an egg sandwich, before resting on the wooden bench outside to plot a new bestseller.

The reason for the Hamptons' enduring popularity is pretty simple. It has a breeze. This is vital for Manhattanites in August, when the city becomes a steaming, belching, flesh-roasting hellpit. And while the East End of Long Island may seem unwelcoming for those without a banking dynasty, the young and the broke simply join forces and share rental properties, of which there are plenty, and travel back and forth from the city via train or the Hampton Jitney bus service. Meanwhile, the older and richer buy estates and buzz over the traffic in their helicopters, a sore point for those along the route, who've recently started to complain about the noise. Tales about bankers racing each other from Wall Street to the beach - with one enterprising weekender beating a helicopter-owning colleague by flying overhead in his Gulfstream business jet and parachuting out at the right moment - are thought to be mostly apocryphal.

As with any ritual, of course, the summer exodus to the Hamptons has its own set of rules. Hotels should be avoided - they're either very expensive or dives; and one of the best, Sag Harbor's American Hotel, has only eight guest rooms. Also, it is highly desirable to be south of the highway - the highway being Route 27, aka the Montauk Highway, because it means fewer insects and an easy walk or cycle to the beach. This rule is of such importance there's even a nail-polish colour by Essie named South of the Highway in its honour. And, finally, the summer begins on Memorial Day, a US public holiday which falls on the last Monday in May (after which it is socially acceptable to wear white), and ends on Labor Day, the first Monday in September.

The basic geography of the Hamptons can be mastered with relative ease, because all the towns are on the same eastbound road (Route 27). Having a car is ideal, although cycling is sometimes necessary because of parking restrictions. Also, each Hampton has its own reputation: Southampton for new money; East Hampton for old money; and Montauk for hippies who don't care either way, mainly because they're more concerned about catching the large-scale surf at nearby Ditch Plains. By and large, there's some accuracy to all this.

If anything, the sickly economy has made the Hamptons more accessible than ever, especially when it comes to renting a property for, say, a couple of weeks instead of having to commit to an entire season. The end of the boom days also means that locals' favourite spots, such as Duryea's Lobster Deck, at the edge of a rotting, abandoned pier in Montauk - the New England Clam Chowder and Lobster Roll are essential summer pleasures - are now relatively peaceful on weekdays. The same goes for Amagansett Farmers Market, with its vat of freshly-made bread pudding (eaten for breakfast) and its outdoor seating area, which is also home to free-range chickens. Likewise, ordering The Godfather, a monstrous sandwich with four different meats, mozzarella, roasted peppers, pepperoncini and extra virgin olive oil, from Villa, a classic deli near East Hampton train station, is a ten-minute operation, as opposed to an hour.

But there's no danger of the Hamptons coming down to earth any time soon. It's still a place where hopelessly urbanised weekenders and second-homers are willing to pay handsomely for rustic experiences, provided only too happily by locals whose other sources of income (such as whaling and potato farming) have long vanished.

Take that clambake on Wainscott Beach, for example. A company named Claws on Wheels is one of a few that will take care of everything - using their employees' residents' permits to literally drive a grill onto the beach - for £80-£100 per head. For parties under 50, guests can legally bring their own alcohol. 'For people who were born and bred here like me, this used to be a very basic, simple thing to do,' says Emma Beudert, 24, who manages the operation from a small industrial estate. 'Now it's still simple, but it's also a luxury.' And it isn't cheap. Kurt Wiesenmaier's bill was nearly £1,300, all in. That's modest, however, compared with another Claws on Wheels customer, who recently entertained 200 friends on the beach. The total? £50,000.

Not everything in the Hamptons is high-class though. Especially not at Cyril's Fish House, a shack on Route 27, owned by 68-year-old Irishman, Cyril Fitzsimons. He came to America 45 years ago and enlisted in the US Marine Corps. 'I wanted to fight in a war, and Vietnam was the only one available,' he explains from under his straw hat. 'I was a platoon sergeant, Charlie company, 1st battalion, 5th Marines. My tour lasted 14 months.'

The food at Cyril's is cheap, fried and not very memorable. But the drinks are strong, and the locals - among them a few Long Island fishermen, known as Bonackers, or Bubbies (they once had their own dialect) - can't get enough of it. Above the bar hangs a New York license plate which reads 'HAMMERED'. Another sign boasts: 'Cyril's... a sunny place for shady people'.

Then there's the banner in the car park, which makes sure everyone knows that 'Eastern Long Island Police Respect Cyril'. Perhaps that's why so many people seem to have no problem staggering from the bar to their convertibles on a Saturday night.

Every day during the summer, the man himself can be found sitting near the front of house, chain-smoking Marlboro reds. With his ginger moustache, sun-destroyed hair and huge, rhinestone-studded watch, he's become a Hamptons icon. Nevertheless, conversation with Cyril isn't advisable beyond 3pm, by which time he has made the most of his own hospitality. And as soon as the season ends, he's gone: he flies to Anguilla in the Caribbean, where he owns two other establishments. He came to this part of Long Island, he says, to 'meet the rich and famous', which is precisely what he did. Visitors to his shack have included President Richard Nixon: 'I was a big fan.'

Another escape from the old-money Hamptons can be found at Sunset Beach, a bar, restaurant and hotel on Shelter Island, which lies between the north and south forks. To get there, you take a drive-on ferry, which crosses an absurdly narrow channel of water and seems to arrive before it has even set sail. Between the French bistro menu, the boozy, well-dressed crowd and the sunset views from the roof-deck over Pipes Coves, you could almost be in St Barts. The food is excellent - the branzino (Mediterranean sea bass) is perfect - although it should be for the Manhattan prices. The only problem is claustrophobia, which sets in with alarming speed when a ferry separates you from the mainland. After dinner, it's a relief to be making for the lights of Sag Harbor ahead.

In spite of the impression given by the New York Post's Page Six gossip column, it's not really the partying Manhattan girls and their Wall Street heir boyfriends who rule the Hamptons during the summer these days. No, it's their parents - the kind of people who drive their Aston Martins over Labor Day weekend to Bridgehampton's Hampton Classic Horse Show, an event that can be traced back to before WWI. The grey hair doesn't just come from New York; Hollywood studio and agency bosses fly in for the season too. Many of them belong to families that have summered here for generations. They eat at restaurants like Nick and Toni's or Della Femina in East Hampton. And yet tagging along as an outsider is surprisingly easy and affordable. Or at least easier and more affordable than it has been in a long time.

While a table at the Hampton Classic in the shade next to the showjumping arena costs around £3,200, access to the bleachers (a sloped area of uncovered seating) is free. Meanwhile, a table at Nick and Toni's can easily be forsaken for wine and cheese tasting at the more relaxed Wölffer Estate Vineyard, which attracts a similar crowd.

Even some of the Hamptons' most lavish estates can be enjoyed via private garden tours advertised in the local newspaper, The East Hampton Star. And then there are the kind of shops that could only ever survive in a place where an ex-Beatle and the head of the world's most powerful investment bank occasionally need to decorate a bedroom. The best is an antiques emporium owned by Brian Ramaekers, located on Main Street in Bridgehampton.

Ramaekers, quietly-spoken, bespectacled and in his late sixties, is a friend of the American lifestyle magazine publisher Martha Stewart. His current inventory features a full-size tack horse from 1880 (£7,500), vintage jewellery and a series of framed, flapper-style wool bathing suits from the 1920s (£540-£890).

'I love the Hamptons in the summer,' he says. 'The weather, the light, the isolation.' Then he catches what he's saying. 'And by isolation, I mean the sense of isolation, while having all the culture of the city, too.'

Anywhere else it might be a contradiction. Here, it sums things up perfectly.

Chris Ayres writes for The Los Angeles Times and is a bestselling author. His books include Death by Leisure: A Cautionary Tale (£7.99; John Murray).

Correction: An earlier version of this article misstated the inspirational setting for the fictional town of East Egg in The Great Gatsby. It was actually inspired by Long Island's north shore.