India's bowlers set up the victory by restricting South Africa to 126 for nine on a surprisingly lively pitch before captain Virender Sehwag blazed to a quick 34.
India were heading for an easy win in their first 20-overs international but there were some jitters towards the end as left-hander Dinesh Mongia was out for 38 to left-arm spinner Robin Peterson, leaving India with 19 to win off the last 15 balls.
It came down to nine off the last over, bowled by Peterson, but Dinesh Karthik hit the first ball for six over square leg and India were able to get the remaining three runs in singles.
"The bowlers did a fantastic job," said Sehwag. "Everybody's happy. We've got the confidence and we'll take this to Sunday's game."
India play South Africa in the fifth and final one-day international at Centurion Sunday. South Africa lead the series 3-0 with one match rained off.
Karthik, who finished with 31 not out, was given out by umpire Brian Jerling, caught at slip by Graeme Smith off Albie Morkel, when he had six. Karthik stood at the crease before starting to move towards the dressing room.
The umpires consulted and the decision was referred to television umpire Karl Hurter, who gave Karthik not out when replays showed the ball had bounced between hitting the bottom of his bat and going to Smith.
Smith won the toss for the fourth successive time against the tourists and decided to bat on a pitch which had a green tinge and gave the bowlers plenty of assistance.
Zaheer Khan set the tone with a superb spell of two for 15 in four overs, including the wicket of Smith for the fourth time.
Smith, who made only one run in three innings in three one-day internationals, falling to Khan on each occasion, made 16 and looked in reasonable form until he was trapped leg before by the last ball of Khan's spell.
Ajit Agarkar started his spell with a rare maiden over, which included the wicket of Herschelle Gibbs, but had to leave the field with a groin injury three balls into his third over when he had figures of one for ten.
Left-handed Albie Morkel hit three sixes and top-scored with 27 off 18 balls. One of his sixes, off Harbhajan Singh, sailed over midwicket and out of the ground into a neighbouring block of flats.
Morkel and Johan van der Wath (21) put on 37 off 23 balls for the sixth wicket in the only productive partnership of the innings.
India became the last of the ten Test nations to participate in the shortest form of international cricket. South Africa were playing in their fourth match, having won one and lost two.
South Africa rested six of the players who have given them an unbeatable 3-0 lead in a one-day international series, including star performers Jacques Kallis, Shaun Pollock, Mark Boucher and Makhaya Ntini.
Rain started to fall in the 12th over and it would probably have been enough to stop play in a normal one-day international but the umpires kept the players on the field and the Indians continued uncomplainingly.
The rain eased before the end of the innings and the weather was fine during India's innings.
Sachin Tendulkar had another disappointing innings, being bowled off an inside edge for 10 by Charl Langeveldt, but Sehwag was in sparkling form, hitting five fours and a six in his 29-ball innings, before he was run out following a mix-up with Mongia.
Mongia shared stands of 43 for the second wicket with Sehwag and 37 for the fourth wicket with Karthik before slogging Peterson to deep mid-on.
South Africa 126 for 9 in 20 overs ( A Morkel 27, Van der Wath 21, Justin Kemp 22; Agarkar 2 for 10, Zaheer 2 for 15) lost to India 127 for 4 in 19.5 overs ( Dinesh Mongia 38, Virender Sehwag 34, Dinesh Karthick 31 n o; Langeveldt 2 for 20, Peterson 1 for 11)