The two countries traded two tries each in a magnificent, open first half before Pollard’s boot edged the Springboks to victory in a contrastingly tense and bruising second period where only three more penalties were added.
Pete Horne and Hamish Watson went over for Scotland while Jesse Kriel and Pollard were South Africa’s try scorers, with Pollard adding two conversions and three penalties.
Greg Laidlaw kicked 10 points for Scotland.
It was another close call for South Africa, who lost to England by a point and then scrapped past France over the previous two weekends. Scotland, meanwhile, lost for the second time in three weeks after losing to Wales in Cardiff.
South Africa were first to score in the sixth minute through centre Kriel after Pollard had cut through the home defence with a darting run to set up the chance.
Huw Jones’ enterprising attacking play then set up Scotland’s first reply after 19 minutes with a clever switch pass putting Pete Horne over at the end of sweeping move halfway up the field.
But within two minutes Scotland paid the price for over-adventurous play with ball in hand in their own 22 as they were snagged and Springbok captain Siya Kolisi forced a turnover that set Pollard away for a try to make it 14-7.
After both sides traded penalties, Scotland then caught their visitors napping at a lineout near the line with a short ball that saw Hamish Watson burst over untouched for a try and, with Laidlaw converting, the two sides were back level at 17-17.
A Pollard penalty made it 20-17 at half-time but Laidlaw ensured parity soon after the break when Willie le Roux was penalised and sin binned after a botched attempted interception was ruled a deliberate knock down.
But while South Africa were man short, Pollard put over another penalty to become South Africa’s third highest test point scorer with 336 career points, and restore a 23-20 lead.
KICKABLE CHANCES
But he missed two kickable chances after that to further extend the lead.
Elton Jantjies took over the kicking duties after coming on as a substitute to put over a long-range effort with nine minutes to go and extend the lead to six points.
It was only a second defeat in 11 home games for Scotland whose recent victims at Murrayfield have included Australia, England and France.
"It was pretty frustrating but we couldn't generate any speed of ball in the second half," said Laidlaw.