The disciplined Prince, who came to the crease after South Africa had been reduced to 28 for three, batted for four-and-a-half hours before play was halted for the day 17 overs early due to bad light.
India squandered a chance to dismiss Prince for 41 when a delivery from medium pacer Vikram Singh was edged to first slip but Sachin Tendulkar spilled the chance.
Tendulkar injured his hand in attempting to take the catch, and was taken to hospital for x-rays. He remained off the field for the rest of the day.
Left-arm fast bowler Zaheer Khan struck in the third over of the match when Graeme Smith, who scored five, top edged a pull shot and fell to a well judged catch by Tendulkar running back from first slip.
Hashim Amla became Khan's next victim when he was trapped in front for one, and South Africa lost their third wicket when AB de Villiers edged a drive off fast bowler Shanta Sreesanth to Tendulkar at first slip to be dismissed for nine.
Herschelle Gibbs and Prince consolidated in a partnership of 94 that lasted until an hour after lunch.
Sreesanth ended the stand when an increasingly aggressive Gibbs, who hit 13 fours in his 63, slashed at a short delivery and was spectacularly caught behind by wicketkeeper Mahendra Dhoni, who dived full length to take the catch in front of first slip.
Mark Boucher and Prince took South Africa to the 13th over after tea, when Sreesanth sent two of Boucher's stumps cartwheeling.
Boucher's 53 was part of a fifth-wicket stand of 100 he shared with Prince in which the runs flowed faster than at any other stage of the day's play.
Shaun Pollock scored 11 before cutting hard at a delivery from Singh and being smartly caught by Virender Sehwag at point in the over before the enforced close.
Five balls later a googly by Kumble dismissed Andrew Hall leg-before for a duck.
Three balls after that Kumble bowled Andre Nel with a top-spinner, also for nought.
Debutant Morne Morkel, who was not out on nought, faced one ball before the players left the field.
South Africa suffered a setback before the start when key allrounder Jacques Kallis was ruled out with a painful back. Kallis was replaced by allrounder Andrew Hall.
India retained the team that won the first Test by 123 runs to record India's first Test win in South Africa.