First innings destroyer Shanthakumaran Sreesanth struck three crucial blows as South Africa, set to make 402 to win, reached 163 for five at the close of play.
Sreesanth, who took five for 40 when the host nation were bowled out for 84 in the first innings, took three for 47 to put his team well on the way to their first Test win in three tours of South Africa.
Ganguly said the bowling of Sreesanth and the other Indian seamers had been "brilliant" but added: "The fields Rahul have set have been outstanding."
Ganguly, who made 51 not out and 25 in a low-scoring match, cautioned that the match was not yet over. "But we should win. If we don't win we shouldn't get back to the dressing room, to be honest."
Ashwell Prince made a carefully-crafted 54 not out to lead South Africa's otherwise unconvincing resistance. He was the only home batsman to top 30 in either innings.
Prince will resume Monday with Mark Boucher (23 not out) with South Africa still needing 239 with five wickets standing and two full days to play.
All-rounder Shaun Pollock is the only batsman of any note still to come but he pledged that South Africa would fight to the last. "The guys believe in themselves in any situation. It's a tough ask but we will come out and fight."
South Africa's top three in the order failed for the second time in the match after India were bowled out for 236 in their second innings. They made a collective 32 runs over two innings.
An improbable quest to become only the fourth team to score more than 400 to win in the final innings of a Test got off to a disastrous start when Herschelle Gibbs was out for nought in the only over South Africa had to face before lunch.
Gibbs was caught at first slip off Zaheer Khan, completing a "pair" for the match.
Then it was Sreesanth's turn as he took the wickets of captain Graeme Smith, Hashim Amla and Jacques Kallis as India chipped away at the South African batting during the afternoon.
The other wicket to fall was that of AB de Villiers, who was needlessly run out when he called for a risky single and was beaten by a direct hit from mid-on by Zaheer Khan.
Sreesanth, who lists breakdancing as a hobby, was the day's outstanding personality. He provided a moment of comic drama towards the end of India's innings. After Andre Nel advanced threateningly down the pitch after a bouncer, Sreesanth hammered the next ball for a straight six and ran down the pitch twirling his bat to taunt the South African fast bowler.
His action made him a target for the spectators and he seemed to delight in playing up to them between overs when he fielded on the fine leg or third man boundary.
He appealed passionately for a catch behind the wicket when Prince was on 40 but ran up to the batsman to shake his hand when he reached 50.
He also bowled superbly, as he had in the first innings, keeping the ball up to the bat and taking advantage of swing and seam movement on a pitch which played easier than on the first two days.
Sreesanth had the out-of-form Graeme Smith caught at backward point for 10 when the South African captain chased a wide delivery. He followed up by having Hashim Amla caught behind for 17.
Kallis and Prince put on 50 for the fourth wicket before Sreesanth struck again after tea, inducing Kallis to drive at an out-swinger and edge a catch to third slip.
Prince batted for 218 minutes, faced 148 balls and hit five fours. He gave a half chance on 22 when a mistimed pull off Khan fell just out of reach of Wasim Jaffer at midwicket.
Earlier VVS Laxman and Khan ensured South Africa would have to chase a daunting target when they put on 70 for the eighth wicket to provide the bulk of the 90 runs that India added for the loss of their remaining five wickets Sunday.
Laxman made a watchful, polished 73 off 134 balls with 12 fours, while Khan provided enterprising support in hitting 37 off 80 deliveries.
India, first innings, 249
South Africa, first innings, 84
India, second innings (overnight 146-5)
Fall: 1-20, 2-37, 3-41, 4-61, 5-119, 6-147, 7-148, 8-218, 9-219
South Africa, second innings
Fall: 1-0, 2-22, 3-34, 4-84, 5-120
Umpires: M Benson (ENG), D Harper (AUS). TV umpire: K Hurter (RSA). Match referee: R Mahanama (SRI)