Sri Lanka bowled the home side out for 241 just before the close on the fourth day of the second test to level the series at 1-1.
It was Sri Lanka's first test victory in the country in nine attempts and followed their innings defeat at Centurion in the first match.
South Africa, chasing a record 450 for victory, collapsed dismally after lunch, slumping to 133 for six before AB de Villiers and Dale Steyn resisted for two and a-quarter hours.
Herath was the inspiration for Sri Lanka, taking five for 79 in 30.3 overs, to give him nine wickets in the test and the man of the match award.
Pace bowler Dilhara Fernando (2-29) could also take credit for bowling South Africa out in two and a-half sessions.
De Villiers and Steyn added 99 for the seventh wicket and appeared set to see South Africa through to stumps before Herath returned with a second new ball that was just three overs old.
He quickly trapped De Villiers lbw for 69 with an arm-ball and Morne Morkel then slogged at part-time spinner Tillakaratne Dilshan, missed and was out leg-before for five.
With bad light threatening to end play for the day, Herath removed Steyn for 43 and last man Marchant de Lange (0) in the space of three balls.
Jacques Kallis had completed the first pair of his test career as South Africa lost five wickets for 50 runs in the afternoon session.
Kallis faced six balls without getting off the mark, before top-edging a sweep at Herath into his helmet, from where the ball looped to short-leg.
Playing in his 149th test, Kallis was initially given not out by umpire Steve Davis but Sri Lanka had the decision overturned on appeal.
Jacques Rudolph (22) edged a wide ball from Thisara Perera into the slips, where Mahela Jayawardene took another fine catch just after lunch.
Hashim Amla reached 51, but dashed off for a quick single to mid-on off Herath, which was turned down by Ashwell Prince, leaving him stranded.
Prince was out for seven when a lifter from Fernando forced him to edge a catch into the slips. Mark Boucher was trapped lbw by Herath, also for seven.
South Africa had comfortably reached 37 without loss before Fernando extracted steep bounce and forced captain Graeme Smith (26) to edge a slip catch to Jayawardene.
Earlier, Steyn took two of Sri Lanka's last three wickets as the tourists added just 23 runs to their overnight total before they were dismissed for 279 in their second innings.
Steyn ended with figures of five for 73 in 20 overs to complete the 17th five-wicket haul of his test career.
The third and final test is in Cape Town from January 3.
