South Africa crashed to 122 for five at close of play in reply to a West Indian total of 408 in which Chanderpaul made a patient century.
Daren Powell took three for 40 and Jerome Taylor two for 28 as South Africa were left with their last two recognised batsmen, AB de Villiers and Mark Boucher, at the crease.
Each wicket was greeted by joyous celebrations by the tourists and Chanderpaul admitted it had been "a long time" since the West Indies had such a good day in Test cricket.
Former captain Chanderpaul said the spirit in the team was good.
"It's the sort of guy Chris is. He's nice and relaxed and just chirping people up. Helping the young players and being the person he is, it's good for the team," said Chanderpaul.
South African vice-captain Ashwell Prince said it was disappointing that three batsmen, including himself, had reached 20 and then got out.
"There were a few soft dismissals, including myself," he said.
But Prince said South Africa had the ability to fight back.
"It's the sort of pitch where you can survive for a long time but if you are trying to score quickly. It's a bit slow. Surviving is not the problem."
Prince said a key objective was to build partnerships and quench the West Indian fire.
"We know what the Windies are like. If they get their tails up, especially if they are bowling, they've got quite a bit to say, but as soon as you get a partnership going it goes a bit more quiet. There's still a lot of cricket to be played."
While South Africa's bowlers were notably unable to strike with the new ball after the West Indies were sent in to bat Wednesday, their Caribbean counterparts were on the mark almost immediately.
Powell struck with the fifth ball of the innings when Herschelle Gibbs, on nought, edged a catch to wicketkeeper Denesh Ramdin.
South African captain Graeme Smith made an aggressive 28 but was then trapped leg before wicket when he played across a full delivery from Taylor.
In his next over Taylor claimed the prize wicket of Jacques Kallis, who had made five centuries in his previous four Test matches.
Kallis failed to score before he tried to withdraw his bat from a lifting delivery by Taylor but instead steered a catch to third slip.
Powell came back for a second spell, changing ends, and with his first ball sent Hashim Amla's middle stump flying with a superb yorker. Amla had looked in good form in making 29 off 30 balls.
Prince and De Villiers put on 33 for the fifth wicket and seemed to have steadied the innings until Powell struck again three balls after a drinks interval when Prince edged a catch to second slip. Half South Africa's batsmen had been dismissed for 96.
Boucher was hit on the gloves by a bouncer from Powell but he and De Villiers survived until the close when South Africa, who started the match as hot favourites, still needed 87 runs to avoid the follow on.
Chanderpaul made 104 for the West Indies before he was ninth man out, bowled by Andre Nel after a patient vigil which lasted 398 minutes during which he faced 254 balls and hit 12 fours.
It was Chanderpaul's 17th Test century and his fourth against South Africa. He also made his seventh consecutive score of 50 or better, equalling a world Test record set by fellow West Indian Everton Weekes in 1948 and 1949 and repeated by Andy Flower of Zimbabwe in 2000 and 2001.
He attributed his run of success to hard work and a determination to capitalise on good starts.
"I've been thinking about it the past couple of years. If you get an opportunity you want to carry on and get big scores."