Manchester City's 100% league record was emphatically ended by a defeat at Tottenham, who remain unbeaten so far this season.
Liverpool came from behind to win, Arsenal had a dramatic and controversial late goal at Burnley to thank for three points, while Chelsea beat Hull City.
Stoke moved off the bottom thanks to a draw at Manchester United and Leicester City were held by Southampton, while Sunderland, Watford and West Ham drew with West Brom, Bournemouth and Middlesbrough respectively.
Do you agree with my team of the week? Or would you go for a different team? Why not pick your very own Team of the Week from the shortlist selected by BBC Sport journalists and share it with your friends?
Pick your Team of the Week
Pick your XI from our list and share with your friends.
Goalkeeper - Lee Grant (Stoke City)
It was only his second appearance in top-flight football and his performance was super. He showed absolutely no signs of nerves and always looked in command.
This was by no means a virtuoso performance by Stoke but their current position in the table demanded a result and it was Grant's performance that was the determining factor.
If substitute Anthony Martial had not entered the fray and produced such a spectacular finish Manchester United would still be playing now without scoring, such was their wastefulness in front of goal. But you can't blame Lee Grant for that.
Right-back - Kyle Walker (Tottenham)
I don't think I saw Manchester City winger Jesus Navas go past Walker once in the 65 minutes he was on the pitch, and eventually Walker saw him off.
Walker is so full of running and confidence at the moment it's scary. It was his interception that started the move that resulted in Tottenham's penalty.
I must say Walker's athleticism reminds me of my former team-mate, ex-Spurs and England full-back Danny Thomas. Now there was an athlete.
Centre-back - Toby Alderweireld (Tottenham)
Alderweireld was outstanding against Manchester City. His tackle on Fernandinho five minutes into the second half was a statement of intent. It was a clear that Spurs were determined to start the second half the way they had finished the first and that was to remain totally in control of the match.
Last season when Spurs went on their impressive run of games, it was Alderweireld and Jan Vertonghen who stood out at the back. They now have Kyle Walker and Danny Rose alongside them and both have a year's more experience under their belts. Let's see if it makes a difference.
Centre-back - Jan Vertonghen (Tottenham)
This was a Tottenham side without Harry Kane or Mousa Dembele and they still managed to batter Manchester City. The main reason was due to the performance of the two centre-backs, Vertonghen and Toby Alderweireld.
I would never have believed that Spurs could have kept a clean sheet against City the way they have been playing this season under Pep Guardiola. Granted, City were without their talisman Kevin de Bruyne but if Guardiola hadn't been so obstinate and hard-headed, he'd have left the door open for Yaya Toure to return to the team and who knows what might have happened.
No Manchester City manager, however celebrated, can have a player of Toure's ability and justify his exclusion with a result, not to mention a performance, like this.
Left-back - James Milner (Liverpool)
A player with a cool head and a decent right foot is quite capable of digging a team out of a very tight situation - and that is precisely what Milner did with his re-enactment of the Antonin Panenka penalty kick. Of course, there will be a generation of youngsters who will begin to copy the technique without having a Scooby Doo where the idea originated or under what circumstances.
However, none of that will interest Swansea manager Francesco Guidolin whose team continue to play well but are destroying their chances of survival in the Premier League by panicking in their own penalty area.
What will help Guidolin is understanding the basic principle that it doesn't matter if you are Milner or Panenka, if you can't handle the pressure you can't possibly be successful at the highest level.
Right midfield - Willian (Chelsea)
What a wonderful finish by Willian against Hull City. The Brazilian was voted Chelsea's best player last season and quite rightly. He seldom misses a game through injury and provides a touch of class in the Chelsea midfield to complement the more energetic members of the engine room like Nemanja Matic and N'Golo Kante.
My only criticism with him is that he doesn't make a fuss. A typical example of this was when Hull centre-back Curtis Davies clearly chopped Willian down in the penalty area and referee Anthony Taylor waved play on.
Willian raised his arms in disbelief - a reaction I have seen before from him when a clear miscarriage of justice has occurred - but, in my view, it is not enough. Players know when a referee has got something badly wrong and not bringing it to his attention does you no favours.
Centre midfield - N'Golo Kante (Chelsea)
Kante might not be the finest finisher in the world - in fact, he's not a finisher at all - and that point was clearly demonstrated in the game against Hull. In the end, Chelsea beat the Tigers comfortably but it was the running power of Kante that provided the platform for a very important victory for manager Antonio Conte that stopped the side's run of defeats.
Kante is every footballer's dream team-mate. Anyone who can put in the amount of unselfish running Kante provides each week should be rewarded by his team-mates with a percentage of their bonuses.
Left midfield - Dimitri Payet (West Ham)
"Could this be his moment?" asked BBC commentator Jonathan Pearce. Well, it certainly was. France international Payet waltzed around five Middlesbrough defenders before calmly planting the ball past keeper Victor Valdes. What a fabulous goal.
The problem with West Ham is that they can't keep relying on Payet by waiting for a moment of inspiration or a wonder goal to pull them out of the mire. The Hammers may have the passion of Mark Noble and the commitment of Winston Reid but they need a genuine goalscorer to share the load with Payet if they intend to retain their status in the top flight.
If Andy Carroll can stay fit for a decent run of games, that might do the trick.
Striker - Troy Deeney (Watford)
The more I see Deeney, the more I start asking myself "why has this lad not been selected for England?" His performances for Watford were immense last season and he appears to be having the same impact on his team again. He was inspirational against Bournemouth and always appears to be a threat in the opposition penalty area.
With a little bit of luck England caretaker manager Gareth Southgate might look beyond appearances or the size of the club and pick a good, honest, down-to-earth footballer. We could do with a few honest individuals in the game at the moment.
Striker - Diego Costa (Chelsea)
Costa's finish against Swansea was as cold and calculated as it gets. Not only did it destroy any impetus Hull had of getting back into the game, but it signalled that Chelsea, after two bad defeats by Liverpool and Arsenal, were back on the march.
I could have picked Crystal Palace striker Christian Benteke or Everton forward Romelu Lakaku instead this week, but only one striker actually won the game for his team and that was Costa.
Striker - Son Heung-min (Tottenham)
What an inspired move by Mauricio Pochettino to play Son up front with Harry Kane injured. Does it work? You bet your sweet life it does.
Son is mobile, sees the pass and is capable of playing it. The through ball for Dele Alli to score was simply world-class and the sort you would expect from a Luka Modric or an Ivan Rakitic.
The only flaw in this otherwise superb team performance was that Son and Erik Lamela found themselves squabbling like a couple of schoolchildren in a playground over who was taking the penalty.
If Spurs have any chance of doing themselves justice in any of the competitions this season, then all their players need to behave in the best interests of the team and not themselves. An unedifying spectacle if ever I saw one.