Last batsman Mick Lewis hit a full toss from Andrew Hall through midwicket for two off the first ball of the final over to secure Australia's win.
Boeta Dippenaar hit 101 as South Africa made 246 for nine after winning the toss, with Shaun Pollock giving the innings a late burst with an unbeaten 53 off 33 balls.
It didn't look nearly enough when Adam Gilchrist and Simon Katich raced to 87 in a first wicket partnership for Australia, especially when leading bowler Pollock left the field with a back injury after bowling only five overs.
But four wickets fell quickly, including a spell of three wickets for two runs in ten balls.
Then Andrew Symonds, playing in his first match of the series after recovering from a hip flexor muscle injury, powered 76 off 71 balls as Australia seemed to take control again.
There was another twist, with Symonds and Brett Lee falling off successive deliveries as Australia slipped to 218 for eight. Nathan Bracken was bowled with six still needed but Stuart Clark and Lewis saw Australia through.
Fast-medium bowler Roger Telemachus kept South Africa in the game, taking three for 34. He dismissed Damien Martyn and captain Ricky Ponting in the space of three balls and bowled Symonds when he came back to bowl at the end of the innings.
Ponting said he was disappointed with some of the shots played by his batsmen.
"South Africa had us on the ropes at times.
"We played well below our best and there is some work to do before Sunday."
South African captain Graeme Smith said the loss of Pollock was a major blow. "But the other guys stood up well, although it was disappointing in the end."
South Africa started the match badly after Smith won the toss and batted, with Nathan Bracken dismissing Smith and Herschelle Gibbs for a single apiece to remove two of the host country's most dangerous batsmen.
Dippenaar played virtually a lone hand until he was joined by Pollock, who reached his fifty off 28 balls. The pair put on 72 for the seventh wicket off 55 balls before Dippenaar was the first of three dismissals off the last four balls of the innings.
Dippenaar was clearly struggling towards the end of his innings and was put on a drip. He did not field and was not well enough to receive his man of the match award.
Australia bowled with great discipline to keep the home side under pressure for most of the innings.
AB de Villiers made an aggressive 38 off 45 balls, including two sixes off fast bowler Lee, but he played one ambitious shot too many against accurate medium-pacer Mick Lewis and was low.
Both teams were strengthened by the inclusion of star all-rounders, with Jacques Kallis returning for South Africa and Symonds playing for Australia for the first time in the series.
Kallis, playing his first match since January 15 after returning home early from a tour of Australia because of a tennis elbow injury, made 21 before he was caught at deep midwicket off a Symonds full toss. He later held a diving catch in the deep and took two wickets.
While wickets fell at crucial times, Dippenaar batted soundly to reach his fourth One-day international century off 141 balls with nine fours.
Bracken and Lewis were the best of the bowlers, taking two for 36 and two for 38 respectively.