Swann and Broad took three wickets each to leave the hosts reeling at 76 for six in their second innings, trailing England by 156 runs with one day left.
Ian Bell made a fine 141 to lead the touring side to a huge total of 575 for nine declared and off-spinner Swann dismissed Ashwell Prince and Hashim Amla before tea.
Jacques Kallis, AB de Villiers, JP Duminy and Graeme Smith then fell in rapid succession after the interval and only a gritty stand between Mark Boucher and Morne Morkel enabled South Africa to reach the early close without further loss.
Smith and Prince survived the opening burst in relative comfort but Swann's second delivery deceived Prince (16) by going straight on, catching the inside edge of the bat and rebounding off the pad to short-leg where Bell took a low reflex catch. The off-spinner then bowled Amla for six as the batsman drove loosely and the ball turned sharply through the gate.
Broad struck three times after tea when Kallis (three), De Villiers (two) and JP Duminy (nought) all made misjudgments trying to leave the ball.
Kallis shouldered arms and had his off stump knocked over by an in-swinger which also seamed off the pitch.
De Villiers left an in-swinger and was given out lbw after asking for an umpire decision review (UDR) and Duminy lasted one delivery, withdrawing his bat too late and deflecting the ball on to his stumps.
Smith fell three overs later when he played across a straight delivery from Swann and was adjudged lbw.
SCOREBOARD
SOUTH AFRICA (I Innings): 343 all out.
ENGLAND (I Innings, O/n: 386/5):
Strauss b Morkel 54
Cook c Kallis b Morkel 118
Trott c Boucher b Morkel 18
Pietersen lbw Harris 31
Collingwood c Boucher b Duminy 91
Bell c Boucher b Steyn 141
Prior b Duminy 60
Broad c Kallis b Duminy 20
Swann c Prince b Steyn 22
Anderson (not out) 1
Onions (not out) 2
Extras (lb-10, W-6, nb-1) 17
Total (9 wkts decl, 170 overs) 575
Fall of wickets: 1-71, 2-104, 3-155, 4-297, 5-365, 6-477, 7-536, 8-565, 9-569.
Bowling: Steyn 34-6-94-2, Ntini 29-4-115-0, Morkel 31-6-78-3, Kallis 14-1-43-0, Harris 38-4-146-1, Duminy 24-1-89-3.
SOUTH AFRICA (II Innings):
Prince c Bell b Swann 16
Smith lbw Swann 22
Amla b Swann 6
Kallis b Broad 3
De Villiers lbw Broad 2
Duminy b Broad 0
Boucher (batting) 20
Morkel (batting) 7
Total (6 wickets; 32 overs) 76
Fall of wickets: 1-27, 2-37, 3-40, 4-44, 5-44, 6-50
Bowling: James Anderson 7-1-24-0, Graham Onions 4-1-12-0, Graeme Swann 12-3-22-3, Stuart Broad 9-2-18-3.