Too much togetherness can sap productivity and creativity.

Editor's note (28 April 2015): In honour of Cubicle Day, BBC Capital brings back this insightful story on the backlash against open-plan offices.

As more employers adopt an open design plan with few private offices — or even cubicles — Pankaj Arora and his young information technology innovation team at Microsoft are bucking the trend.

They discussed an open floor plan, but decided they could be most productive in their own small offices. When they need to collaborate, they simply open their doors and roll their desk chairs out into the corridor, which they have dubbed the “hallway of knowledge.”

“It was important to be able to hunker down and focus behind closed doors, but be in close proximity to each other so we could collaborate,” Arora, leader of the Modern IT Innovation Group, said. “We never see the doors as barriers to communication, just as barriers to noise.”

Like Arora and his team, many workers today crave a haven for privacy and concentration. Collaboration is critical, of course, especially at technology companies such as Microsoft that thrive on innovation, but too much togetherness can sap productivity and creativity.

Many people are growing frustrated with the din and distractions of the modern office, where they sit at crowded desks or tables without any physical barriers separating them from their neighbours. Call it the backlash against the open office plan — but don’t expect most employers, who have invested in new floor plans and reduced their real estate costs, to backtrack on the noisy shared spaces.

Too open

More than two-thirds of US employees are unhappy with noise levels at work, and 53% say other people disturb them when they try to focus, according to a 2013 survey by Gensler, a global design firm. 

Surprisingly, the Gensler study found that people reported doing more focus work than they did five years earlier — 54% of their time vs. 48% — but less collaboration — 24% versus 30%.

“If we spend more than half of our time on work that requires concentration, we need to be sure we have some quiet places in the office; otherwise performance will suffer,” said Janet Pogue, a principal at Gensler. Already, workplace performance has dropped 6% since 2008, driven largely by the inability to focus, Gensler’s research found.

Finding a compromise

One solution to the noise problem is to compromise by designing workspaces that balance the need for interaction with colleagues and quiet time to focus and reflect — or just make a personal phone call. To that end, some employers have begun building more small rooms into their design plans.

These “huddle rooms” or “focus booths” can provide solitude when workers need a refuge from ringing phones and chatty colleagues. They tend to be small and usually accommodate one person or a small group. They might include a work table and chair, or simply a standing desk with just enough room for a laptop and phone.

Steelcase, an office furniture and design company, is even marketing a series of “quiet spaces” for “introverts” because it says they represent one-third to one-half of the workforce. Some of the private rooms include comfortable sofas for relaxing and enough floor space for stretching or meditating.

At Microsoft, some employees are stationed in “neighbourhoods” with open layouts and various rooms for team meetings, private phone calls and quiet, individual work. The neighbourhoods are located away from high-traffic corridors to minimise visual distractions, and they are designed for a single team because hearing other groups of employees doing unrelated work could be disruptive.

“The world has put all the focus on collaboration with people thrown in a big room together, but you have to be more thoughtful than that,” said Martha Clarkson, Microsoft’s global workplace strategist. “When you put people in a communal environment, it won’t work if you don’t provide privacy. They need alternative spaces for thinking time, whether it’s focus rooms, lounges, patios or outside walking areas.”

Workplace design experts recommend that some private rooms be reservation only, with others available on a first come-first served basis.

Although a mix of open and private areas may sound like an ideal solution to the noise problem, some companies fail to provide enough quiet zones to meet everyone’s needs. In a 14-country study this year by Steelcase, only about half of the employees said their workplaces offer private enclaves. The highest numbers of respondents with private rooms in the workplace were in India (71%), China (70%), the US (68%), Canada (65%), and the UK and Netherlands (64%). About 45% of French, German and Belgian respondents had private rooms at work, while only 32% of those in Turkey and 39% of those in Russia did.

Escaping the office loudmouth

Despite growing concerns about privacy and distractions, some employers are taking the open office concept even further with so-called “free-address” workspaces where employees don't even have assigned seats. Such offices might seem like a recipe for even more chaos and cacophony, but at least an employee won’t get stuck sitting next to the office loudmouth day in and day out.

When CBRE, a Los Angeles-based commercial real-estate firm began creating free-address workspaces in its offices around the world, the change was met with some resistance. “There were a lot of concerns at first about things like brokers and attorneys needing privacy and the prestige factor in having an office,” said Lew Horne, president of CBRE’s southern California region. While CBRE told employees they simply had to get over any ego needs for a corner office, the company addressed privacy issues by creating a variety of soundproofed focus rooms.

Millennials don’t mind

Not surprisingly, the very social millennial generation doesn’t seem to mind the hubbub in open-plan workspaces. Millennials, born in the 1980s and 1990s, see the same drawbacks to open layouts — noise, distractions, lack of privacy and crowding — as older people. But they strongly believe that the opportunities to socialise, work in teams or get help from colleagues outweigh the negatives, according to studies of office workers in Finland.

“Those born after 1980 liked an environment that supported collaboration and socialising more than older generations,” said Peggie Rothe, a researcher at Aalto University in Finland and co-author of the studies. “Privacy preferences were similar across the generations, but those born in the 1950s valued it a bit more than the others.”

No matter what other millennials say, however, Microsoft’s Arora and his young IT team are clinging to their private offices. “For some people, an open floor plan may well be the best way to collaborate, but we think it can also be very restrictive, such as when you need to focus on a problem or do a conference call,” Arora said. “We believe we enjoy the best of both worlds with our offices for focusing and our hallway of knowledge for interacting.”

To comment on this story or anything else you have seen on BBC Capital, head over to our Facebook page or message us on Twitter.