Lanky fast bowler Shahadat Hossain picked up a career-best 6-27 to help Bangladesh bowl out their mighty rivals for 170 inside 61 overs, a first innings advantage of 22 runs.
The hosts, who made 192 in their first knock, swelled the overall lead to 147, reaching 125-4 at stumps on an already deteriorating pitch at the Sher-e-Bangla Mirpur stadium.
Young Junaid Siddique led the charge with an unbeaten 64 off 158 balls, including five fours, as Bangladesh raised hopes of only their second ever Test victory.
The opener put on 56 runs for the fourth wicket with captain Mohammad Ashraful (24), who earlier plotted a bizarre dismissal of South African batsman A.B. de Villiers.
Aftab Ahmed, batting on 13, was giving Siddique company when stumps were drawn for the day amid frowns and creased foreheads in the visiting camp.
Bangladesh, who lost all their previous four Tests to South Africa by innings margins, have won just one of their 51 Tests -- against minnows Zimbabwe.
The South Africans tried to stage a comeback through Dale Steyn (3-38) who cleaned up the Bangladeshi top order in a three-wicket burst.
Opener Tamim Iqbal (two) was done in by a late inswinger while Shahriar Nafees (16) was trapped leg before soon after.
Habibul Bashar, the former captain playing in a national record 50th Test, was out leg-before for just two as Bangladesh crashed to 29 for three.
Ashraful then became the first victim of Makhaya Ntini in the match when he edged one behind to Mark Boucher for 24 which included one four and a six.
But Siddique held anchor at one end to ensure there were no further setbacks for the host team.
Bangladesh owed their performance to Hossain as well as the move by Ashraful to bring himself into the attack and claim the crucial wicket of de Villiers (46).
De Villiers, dropped in the preceding over by Ashraful on 30, pulled at a ball which had bounced twice before reaching his bat.
While Ashraful promptly made amends for the earlier miss by taking the catch, de Villiers stood his ground believing that the ball was dead.
But umpire Steve Bucknor raised his finger, rightly so, as the rules state that a ball will be deemed legitimate if it does not bounce more than twice before hitting the bat.
It was an important breakthrough for the hosts as de Villiers was growing in confidence, having hit seven fours and a mighty six over the mid-wicket in his 73-ball knock.
Hossain then returned to clean up Johan Botha (25). who had stoutly resisted for 91 balls, and tail-ender Morne Morkel.