It took late interventions from midfielder Anton Lienert-Brown, who scored a try after sustained pressure to put his side to within a point, and then an incredible breakout try for halfback Brad Weber, his second of the night, to put them in front in a match which caught fire in the second half.
It is a victory which propelled them from second to last on the table to 11th and within striking range of a playoff place. The Chiefs haven't been all that convincing this season, but their performances over the last fortnight suggest they may have found a groove that could provide problems for the remainder of the regular season and perhaps the finals series.
They found themselves in trouble after the break after a relatively straightforward opening stanza which saw Weber scoring a nicely constructed try and loose forward Mitch Karpik going over from close range.
But, while the Chiefs showed a fair bit of grit in their fightback to draw against the Highlanders in Dunedin last weekend, this Sharks team are made of some stern stuff too. They showed their true pedigree when they drew with the Crusaders in Christchurch a week ago, a match they probably should have won but for Mitchell Hunt's heroics after the 80 minutes for the home side.
Here they were more expansive and looked better for it, although they still relied on their 21-year-old first-five Curwin Bosch, a goalkicker with ice in his veins and a finely tuned radar in his right boot.
Bosch kicked all his side's points last weekend and tonight he kicked five from five and offered a bit around the field besides. He has pace and vision and looks a real prospect for South Africa.
And yet, while they battled back from a halftime deficit to lead 23-17 at one point following the sinbinning of Harris for not releasing a tackled player when under pressure in the red zone, an act deemed by referee Nic Berry as cynical, they didn't quite have the quality to close it out.
In the first half, lock Hyron Andrews probably surprised himself and the wider Waikato when he ran in from 40m out to score his side's first try, and with Bosch kicking everything going the Sharks were never out of it.
Andrews' locking partner Ruan Botha scored one of his own after the break from close range and the Sharks had a handy lead following Harris' temporary dismissal.
They would have dared to believe at this point but the Chiefs had other ideas, with forwards Luke Jacobson and Tyler Ardron putting in huge performances for the home side and Lienert-Brown striking exactly when needed.
From the restart, replacement midfielder Alex Nankivell went on a big break and fed halfback Weber who scampered over for a converted try.
It appeared to close the gate on the match, but the Sharks weren't finished and threw everything they had in an attempt to score a famous victory, but it wasn't to be. The Chiefs are back and appear to be going places at last.
Chiefs 29 (Brad Weber 2, Mitch Karpik, Anton Lienert-Brown tries; Marty McKenzie pen, 3 cons), Sharks 23 (Hyron Andrews, Ruan Botha tries; Curwin Bosch 3 pens, 2 cons). Halftime: 17-13