Kallis's well-paced innings saw South Africa overhaul an India total of 242 for eight built around a third wicket stand of 158 between Sachin Tendulkar (99) and Rahul Dravid (74).
South Africa needed four off the last over, bowled by part-time left-arm spinner Yuvraj Singh.
But all-rounder Vernon Philander (17 not out), in only his second match at this level, settled any lingering nerves by lofting Yuvraj for four as the Proteas won with three balls to spare.
Tendulkar was left just 50 runs away from becoming the first batsman in history to score 15,000 one-day international runs - a landmark he could reach during the remaining games of this series.
However, in sight of what would have been his 42nd hundred at this level, the 34-year-old Tendulkar was run out, going for an ill-judged second, by cover sweeper Morne van Wyk's throw to wicket-keeper Mark Boucher.
Tendulkar and India captain Dravid repaired the innings after Sourav Ganguly and Gautam Gambhir had fallen cheaply to fast bowler Andre Nel, who led the attack with three wickets for 47 runs.
In reply, AB de Villiers and van Wyk shared a first-wicket stand of 56 as conditions became increasingly cold and overcast.
That ended when de Villiers, four balls after being dropped by Tendulkar at first slip off RP Singh, edged a square-drive off the backfoot against the left-arm quick and wicket-keeper Dinesh Karthik, diving to his right, held a fine catch.
Teenage leg-spinner Piyush Chawla struck with his 11th ball when van Wyk, who'd made 44, got a leading edge, playing across the line, gave Rohit Sharma a gentle catch at cover.
And the 18-year-old had his second wicket when dangerman Herschelle Gibbs, trying to dab the ball down on five, was bowled by a ball that didn't turn as much as he expected.
The Proteas needed under a run-a-ball when Mark Boucher was lbw for 23 to left-arm quick Zaheer Khan despite the ball pitching outside leg stump.
However, Kallis was still in, his fifty up in 65 balls with three fours.
But at the other end Andrew Hall (18) drove Chawla to Dravid at extra-cover and South Africa, with 10 overs left, needed 51 to win with four wickets standing. Chawla, like Nel, finished with three for 47.
South Africa, despite Kallis saying Monday they wanted to bring on a spin bowler in a bid to remedy a longstanding area of weakness, had omitted off-break specialist Thandi Tshabala from their side.
Tshabala's place was taken by medium-pacer Charl Langeveldt, whose eight overs cost 59 runs.
Other changes to the Proteas team that beat Ireland by 42 runs here Sunday saw Nel replace fellow quick Dale Steyn and Andrew Hall take over the all-rounder's spot from Justin Kemp.
Tendulkar completed his fifty with a single off Kallis in 95 balls with six fours, Dravid getting there in 71 with three boundaries.
India though lost six wickets for 61 runs in the last 10 overs.
Dravid played on to a Nel yorker, Tendulkar exited and Yuvraj Singh (13) drove Hall straight to Kallis at mid-on.
India, whose squad had been struck down by 'flu on the eve of this match, made two changes from the side that beat Ireland by nine wickets here Saturday.
Powar replaced all-rounder Ajit Agarkar and Zaheer came in for fellow quick Shanthakumaran Sreesanth.