Chasing a 321-run victory target, the hosts were bowled out for 224 with more than a day to spare.
A century partnership between Vaughn van Jaarsveld and Justin Kemp was the only notable resistance India faced as the visitors chalked up a much needed win after being whitewashed 4-0 in the One-day series against South Africa.
Once Van Jaarsveld was run out, the South Africans fell apart and handed India a boost before they head into the three-Test series starting in Johannesburg on Friday.
All rounder and man of the match Irfan Pathan again proved to be a saviour with the bat adding an unbeaten 40 to his first innings 111 not out to push the target beyond 300 and out of the opposition's reach.
Seamer Alfonso Thomas picked up two more wickets on Saturday to finish with 7-56 in the second innings, with India bowled out for just 142.
Apart from Pathan, captain Vangipurappu Laxman (31) was the only other Indian batsman to score more than 20. Wasim Jaffer, former captain Saurav Ganguly, who scored 83 in the first innings, and Harbhajan Singh were all dismissed without scoring in the second innings. For Jaffer, it was his second successive duck in the match.
Zaheer quickly put the skids under the run chase by bowling the Rest XI skipper, Jacques Rudolph, with a superb inswinger first ball of the innings and added the scalps of internationals J-P Duminy and Justin Kemp to finish with 3-36 from 12 overs.
Vikram Singh collected another two victims to add to his first innings trio to finish the match with 5-76 and a good chance of earning a starting berth at the traditionally seamer-friendly Wanderers.
A fifth wicket stand of exactly 100 between Kemp (53) and Van Jaarsveld (66) briefly threatened to take the game away from India but van Jaarsveld's optimistic call for a single ended the stand.
Off spinner Harbhajan (3-39) ensured a swift and painless end.