On 20 September 1984, The Cosby Show introduced us to the Huxtables; the doctor/attorney duo of Heathcliff and Clair and their five impossibly well-behaved kids – Sondra, Denise, Theo, Vanessa and Rudy.

Built on humour and moral messages, the programme dealt with everything from ear piercings (Theo), getting your children to eat vegetables (Rudy) and your eldest daughter dating someone undesirable (Sondra). With instructions for life dished up with tough love, viewers could never mistake the Huxtables for the Brady Bunch.

"Black audiences greeted The Cosby Show with a certain amount of euphoria” says Mark Anthony Neal, a professor of African and African American Studies at Duke University in North Carolina. “It was a regular opportunity to see black life portrayed in a 'responsible' manner."

The family sitcom drew on Bill Cosby’s many creative endeavours, as a stand-up comedian, actor, writer and musician. While The Cosby Show ranked as the number one TV show for five seasons from 1985 to 1990, it was not without its critics.

"The root of the hip-hop generation’s displeasure with The Cosby Show was not simply that the show wasn’t political, but rather the show served the political function of diverting attention away from the harsh realities of Reagan-era social policies," says Neal. "In effect, the Huxtable family was posited as the 'model' black family, overriding the legitimate criticisms of Reagan-era attacks on social policies that were enacted to address social inequities the show helped obscure."

Yet some argue that The Cosby Show’s cosy middle-class family built on an evolving tradition of African-American representation on television. “In the early days, roles on the whole were limited to gangsters, pimps, drug dealers, some kind of servitude, or buffoons” says Tim Cogshell, film critic on NPR affiliate, KPCC.

While The Amos ‘n’ Andy Show – a television programme adapted from radio – was one of the first to have a predominantly black cast in the early ‘50s, it wasn’t until the late ‘60s that a multi-dimensional view of African Americans began to take shape on screen. Airing from 1968 to 1971, Diahann Carroll took the lead as a widowed nurse in Julia. By 1972, Bill Cosby took the reins with his animated series Fat Albert and the Cosby Kids and Sanford and Son starred comedian Redd Foxx.

Good Times, in 1974, documented the struggles of the Chicago-based Evans family. Putting raw and controversial subjects on screen, the plotlines examined urban life through humour. Yet there was a behind-the-scenes battle to keep negative stereotypes out.

“Trying to get something decent done was like trying to pull elephant teeth with tweezers” says Eric Monte, screenwriter and creator of Good Times.

“Nobody wanted to do that horrible stereotypical stuff being perpetuated by the white writers. John Amos and Esther Rolle, who played James and Florida Evans, they’d get insulted, just like I did.”

For Monte, it was more than just a creative fight. “Good Times was all we had to represent my black culture on air. White people had dramas, talk shows, news. All we had was Good Times and Sanford and Son. I wanted the world to see our real culture.”

The Jeffersons, which premiered in 1975, went some way in breaking down the popular media stereotypes of African Americans.  Louise and George Jefferson, along with their son Lionel, entertained audiences with their antics after coming into money and moving to Manhattan. Through comedy it tackled tough topics such as suicide and gun control.

In the ‘80s and ‘90s, black television exploded, as networks fought to create positive programming for an engaged demographic. A Different World – a spin-off of The Cosby Show followed, along with The Fresh Prince of Bel Air starring Will Smith, Martin with Martin Lawrence and Living Single with Queen Latifah.

Today, black lead characters feature in a range of programmes. Screenwriter Shonda Rhimes has dominated this space with award-winning shows such as Scandal, starring Kerry Washington, and also Grey’s Anatomy, Private Practice and How to Get Away with Murder, starring Viola Davis from The Help.  This season will also see the launch of the new sitcom, Blackish, starring Tracee Ellis Ross, Anthony Anderson and Laurence Fishburne.

Cinema’s black voices

But for screenwriters such as Good Times’ Eric Monte, creating for television was never enough. The real power, he says, is in the big screen. He produced one of the seminal titles in black film, Cooley High (1975), a coming-of-age movie that explored the tragedies and triumphs of two high school friends growing up in Chicago. “It had people lined up around the block” says Monte. “It was only advertised on black radio, but Cooley High was the biggest hit after Jaws that year.”

Much like television, there has long been an African-American presence in US cinema. There are the painful scenes in DW Griffith’s Birth of a Nation in 1915; the race-themed films of Oscar Micheaux through the ‘20s; Hattie McDaniel’s historic Academy Award for Gone with the Wind and Paul Roberson’s powerhouse baritone in Show Boat in the ‘30s.  In 1943, Lena Horne starred as Selina Rogers in Stormy Weather and by the late ‘50s Dorothy Dandridge and John Justin explored the social taboo of interracial relationships in Island in the Sun.  

The civil rights struggle and the social upheaval of the 1960s marked a profound shift in how black America was portrayed on screen. While Dr Martin Luther King Jr and Malcolm X marched and demonstrated, artists such as Ruby Dee, Ossie Davis and Harry Belafonte waged their own campaigns – using the movies as their messenger. Sidney Poitier was at the forefront, becoming the first black man to win a Best Actor Oscar, in 1964.

“Sidney Poitier almost single-handedly represented black folks and what was going on in the community” says Tim Cogshell. “As a doctor in Guess Who’s Coming to Dinner, tackling interracial relationships and a teacher in To Sir with Love Poitier broke down the stereotypes.”

If the reserve of Sidney Poitier marked the ’60s, the ‘70s was all about being up front: enter the blaxploitation era.   Independent filmmakers took control of Hollywood’s portrayal of black America, going one step further with the ownership of content.

Already an established director following the success of Watermelon Man, in 1971, Melvin Van Peebles became the man who could do everything. He wrote, produced, directed, starred in, edited and penned the score for Sweet Sweetback's Baadasssss Song, a film about a black man’s fight against white supremacy. This message spoke to the then-growing Black Panther Party and became essential viewing for members.  

“Sweet Sweetback's established a black voice in cinema,” says Cogshell. “It became an event. Black folks would leave the suburbs and travel into the city to see what this film was all about.” Along with Sweetback’s, Shaft by Gordon Parks was credited with starting the blaxploitation genre.  Others benefited from the trend, such as Rudy Ray Moore with Dolemite. A number of women also came to the fore: Pam Grier as Foxy Brown, Tamara Dobson as Cleopatra Jones and Teresa Graves in Get Christie Love!

Through the ‘80s, black America continued to see itself on screen with Robert Townsend’s Hollywood Shuffle being one of the most notable films of this decade. It examined the roadblocks to pursuing a career in Tinseltown. But it was Spike Lee’s She’s Gotta Have It and Do the Right Thing which marked the start of what film critic Roger Ebert called the ”Black New Wave”.

“That era … with Lee, John Singleton, Matty Rich, the Hughes Brothers; they were all actively creating and owning their work” says Gil Robertson, president of the African American Film Critics Association. “It’s one thing to be hired as the director.  But are your words there? How involved are you in the creation of that story?”

So what happened to that ‘New Wave’?  

‘Business as usual’

A recent report from the University of Southern California’s Annenberg School of Communication and Journalism warns of a “scarcity” of African Americans in Hollywood.  Even in a year with films such as 12 Years a Slave, The Butler and The Best Man Holiday in 2013, 17 of the top grossing films of the year had no black actors. And there’s no better news for filmmakers.  Just five black directors took the helm of any of the top grossing movies of the year. And none were women.

"2013 was not a banner year – it was business as usual," said Professor Stacey L Smith, study author and leader of Annenberg's Media, Diversity & Social Change Initiative. "Any momentum gained in terms of accolades or even awareness gained on this issue in 2013 – we probably won't see the fruits of that conversation for a few years."

Rather than jump through film and TV executives’ hoops, some content creators are now taking their work straight to the audience via the web. Could the internet be the next home of the Black New Wave? Los Angeles-based Black and Sexy TV, co-founded by Numa Perrier, Dennis Dortch, Jeanine Daniels and Brian Ali Harding, creates content that is unapologetically black.

Black and Sexy TV’s web series The Couple has been picked up by HBO, though the founding team insists this wasn’t their main goal. “An HBO executive saw the show, watched all of the episodes, loved it and called us the next day. But we’re still in the game of building our own platform,” says Perrier.

“The web is a great place for us, and it speaks to what Ruby Dee and her contingency did in the ‘60s and ‘70s. Every generation has done its best to work together to figure out a way to break through. We’re another generation. We’re doing that with the tools we have. And we just happen to have really great tools right now.”

The likes of Numa Perrier have inherited a very different media landscape from the one The Cosby Show was born into 30 years ago. Cultural shifts, including the rise of social media, the democratisation of distribution through the internet and advances in recording technology, now mean there is no shortage of ways for black America to be represented on television and in film.  In many ways the challenge for content producers now is, how do they build on the foundation of the many black programmes that have gone before? While The Cosby Show didn’t tackle race issues directly, it quietly cracked open the door to change; both the way black people are perceived by others, and how black people view themselves. Quite an achievement for a show with a lead character with such awful taste in sweaters.

If you would like to comment on this story or anything else you have seen on BBC Culture, head over to our Facebook page or message us on Twitter.